Facebook open-sources deep-learning modules

389 points by darklrd 10 years ago | 71 comments
  • DannyBee 10 years ago
    Sadly, this has facebook's new, horrible patent clause.

    For those who aren't aware, it says

    1. If facebook sues you, and you counterclaim over patents (whether about software or not), you will lose rights under all these patent grants.

    So essentially you can't defend yourself.

    This is different than the typical apache style patent grant, which instead would say "if you sue me over patents in apache licensed software x, you lose rights to software x" (IE it's limited to software, and limited to the thing you sued over)

    2. It terminates if you challenge the validity of any facebook patent in any way. So no shitty software patent busting!

    • jawns 10 years ago
      Here's a link to the patent clause: https://github.com/facebook/fbcunn/blob/master/PATENTS

      DannyBee, I'm having a hard time envisioning what sort of scenario might trigger this, and what the repercussions might be. Any chance you could explain?

      • DannyBee 10 years ago
        Do you mean the first part, or the validity provision? as a reply points out, it is incredibly broad.

        Assuming you mean the first part (the validity provision seems "obviously bad" to me), let's lay out some scenarios:

        You make your important software dependent on this software. Congrats, facebook can now sue you with impunity because you can't afford to lose these patents.

        Worse, the fact that they have given you a grant does not prevent them from suing you, and the second you counterclaim, you lose. Note also that the grant specifically does not cover "the Software in combination with any software or other technology provided by you or a third party." So if you take this stuff and run it in a pipeline, and facebook has a patent on running it in a pipeline, you don't get that patent and they can still sue you.

        (Again, in the apache case, you'd only ever lose the patent grant if you sued facebook over the software in question).

        Ignoring direct cases, facebook can also use this as leverage during acquisitions that use their software (Nice software, shame if you were to lose rights to it), etc.

        There are potentially an infinite number of ways to use this kind of stuff.

        Also note this kind of thing has been looked at in the past. There is an entire discussion here: http://mail-archives.apache.org/mod_mbox/archive-license/200...

        from when v2 of apache license was being written, etc.

        There are reasons the Apache grant has become pretty much the standard - most people and corporations find it a pretty good balance of various concerns (being able to defend yourself in patent lawsuits, etc).

        What Facebook has done is entirely one-sided. It benefits Facebook only, and even then, it generally only helps them in situations where they want to sue people. None of this really helps defensively. If they wanted it for defense, they'd write it differently to cover defensive situations.

        • vonmoltke 10 years ago
          > You make your important software dependent on this software. Congrats, facebook can now sue you with impunity because you can't afford to lose these patents.

          I have been rattling the clause jawns posted around in my head, and I cannot see this. I agree that the clause is onerous and I would not incorporate software that had such a clause into one of my products without a backup plan. I do not see how this gives Facebook the ability to sue anyone with impunity. The clause applies to alleging patent infringement BY Facebook or its subsidiaries (i.e., you are suing them) or if Facebook is specifically suing you for patent infringement. Clause section b only applies if your defense is based on invalidating a Facebook patent; saying, "We don't actually infringe that" does not trigger this clause.

        • frabcus 10 years ago
          How can they claim this text file applies?

          There is also a file with the industry-standard name LICENSE (https://github.com/facebook/fbcunn/blob/master/LICENSE), with a standard looking BSD license in it.

          Surely somebody using it could claim that was the file that applied, and this other non-standard whimsical "PATENTS" file was irrelevant?

          Or can people just bury random licenses deep inside copyrighted works, and expect them to apply?

          • mcintyre1994 10 years ago
            Does that BSD license grant a patent license though? It doesn't seem to. Otherwise arguing their "PATENTS" file doesn't apply means you have no right to any patents exercised by the software anyway.

            All it does is grant rights, albeit flimsy ones that can be taken away in stupid circumstances. There's no real use arguing it doesn't count because then you're just saying you never had those right and are infringing in exactly the same way.

          • SixSigma 10 years ago
            > The license granted hereunder will terminate, automatically and without notice, for anyone that makes any claim ... by ... assertion or other action ... alleging .. that any right in any patent claim of Facebook is invalid or unenforceable.

            That sounds pretty chilling considering that could mean that anything any of your employees say about any other Facebook patent could wreck your business, if you choose to use Facebook code from Github.

            • j_s 10 years ago
              The most interesting part to me is whether or not such claim is related to the Software - so any patent suit against any software, product or service of Facebook or any of its subsidiaries or affiliates terminates patent license to this software... seems like they went for the nuclear defense!
            • bitL 10 years ago
              Does this apply in continental Europe as well? I doubt so. You can't give away your rights via any form of contract.
              • jongraehl 10 years ago
                As long as you're doing your training offline and shipping the resulting learned models, I don't see this is a big risk. If you're planning on shipping software that can learn online, then sure, use something else for that, or ask for a better deal from FB.

                I wonder what would happen if every company offered patented algorithms under these terms. Still wouldn't stop patent trolls, I'd guess.

                • tim333 10 years ago
                  As I read it is basically says you can use Facebook's patents unless you turn around and sue them in which case the deal's off. I'm not sure it's that awful - I mean how many companies are planing to use Facebook's patents and then sue them?
                  • DannyBee 10 years ago
                    You should read it again :) It requires no lawsuits to lose patents As written, if you make blog posts saying their patents are crap, you've lost.

                    (and the other provisions are not defensive, they don't require you sue facebook to lose patents)

                  • mcintyre1994 10 years ago
                    This does look terrible but I'm not sure about number 2 - can you quote the part of the clause that suggested that?
                    • DannyBee 10 years ago
                      "The license granted hereunder will terminate, automatically and without notice, for anyone that makes any claim (including by filing any lawsuit, assertion or other action) alleging ... (b) that any right in any patent claim of Facebook is invalid or unenforceable."

                      (the entire rest of the ... is clause (a))

                      • mcintyre1994 10 years ago
                        Oops, not sure how I misread that, it's pretty clear! This unfortunately seems to reduce these projects in usefulness to pretty much awesome and interesting research - completely unusable in any business though. I don't see how any business can be expected to make all employees keep to these conditions - way too fragile and easy for an employee to screw the company.
                    • swatow 10 years ago
                      I understand the concern. For reference, what are the worst patents that Facebook owns? Do they own patents that should not have been granted?
                      • 10 years ago
                      • bpodgursky 10 years ago
                        I appreciate that Facebook open-sources their libraries, but they have an awful habit of dumping source code on Github and then continuing to develop it internally without pushing those changes back out.

                        For example, Facebook open-sourced Thrift, then proceeded to develop it internally, and recently released out a fork of thrift, not merging anything back into the original tree.

                        Likewise, they open-sourced Corona a few years ago, their Hadoop fork, and I'm pretty sure that branch has been abandoned too.

                        So I'm really hesitant to start using any projects out of Facebook unless they develop some processes to actually maintain these projects.

                        • choppaface 10 years ago
                          Agree that their stewardship has been lacking (especially in comparison to say Apache, Docker, etc), but for research code this is a pretty nice release in terms of the scope of problems solved. For research it's more important to have an implementation of what worked (and what /didn't/) than a polished solution + well-kept community because a lot of the current code will probably get thrown away eventually anyways. The service here is research done rather than software provided.
                          • peterhunt 10 years ago
                            That was years ago. Things have changed around there: https://github.com/facebook
                            • tdaltonc 10 years ago
                              They've done a good job keeping up PlanOut.

                              https://github.com/facebook/planout

                              • amelius 10 years ago
                                From the article:

                                > We hope that these high-quality code releases will be a catalyst to the research community and we will continue to update them from time to time.

                                • 10 years ago
                              • nightski 10 years ago
                                Too bad it is Torch. Working in the lua environment is not enjoyable at all. Every error becomes a long procedure of looking deep into the source of the framework since there is no type information or stack traces to go off whatsoever. You are constantly guessing about the shape of the data or what parameters are supposed to look like.
                                • smhx 10 years ago
                                  there's clear and well stack-traces at the lua level. there's great debugger (in fblualib, called fb.debugger). there's an inbuilt profiler into LuaJIT. too bad you've had a bad experience.
                                  • nightski 10 years ago
                                    Thanks I am always willing to give it another shot with more tools.
                                  • vonnik 10 years ago
                                    try http://deeplearning4j.org/

                                    they just partnered with canonical: http://insights.ubuntu.com/?p=9373

                                  • vonnik 10 years ago
                                    This is a recruiting move. Facebook is building its DL team, and to do that, they need to grow the community using their tools.
                                    • publicfig 10 years ago
                                      I see no issue with open sourcing tools to recruit developers. Honestly, that's one of the best methods I can think of.
                                      • vonnik 10 years ago
                                        I'm not against it. It's a great way to recruit devs. Just wanted to point out that they would likely be recruiting from the community.
                                      • ghayes 10 years ago
                                        And yet we should applaud the move. I will always appreciate a company moving proprietary technology into the public sphere.
                                        • kylebrown 10 years ago
                                          Now they just need to release the datasets as well. The datasets are far more important than the algos.
                                      • contingencies 10 years ago
                                        Paper with proper summary: http://arxiv.org/abs/1412.7580
                                        • ajtulloch 10 years ago
                                          This paper describes one part of the fbcunn release (the fast convolution layers implemented via FFT, with the source available at https://github.com/facebook/fbcunn/tree/master/src/cuda/fft). There's a lot more in fbcunn if you want to check it out.
                                          • Hydraulix989 10 years ago
                                            In my experience, the fully connected layers are the bottleneck. The other issue was the alternating compute-heavy convolution and the IO-heavy pooling. I'm curious how this FFT implementation stacks up against cuDNN (what's the speedup like for just the convolutional layers? and then what's the overall speedup like?).
                                      • technologia 10 years ago
                                        We've been using Torch for a while at work, really appreciate Facebook doing this. Some of us had a hard time adjusting to Lua, but it was well worth it.
                                        • chubot 10 years ago
                                          I'm curious what your team experience was with Lua.

                                          IIRC, Lua is even more liberal than Python in that if you mistype a variable name, it will silently give you 'nil' rather than throwing an exception.

                                          Also, the prototype system is a little wonky. It seems overly flexible and makes you do too much work yourself, which would make it hard on a large team all getting started at once.

                                          If Torch gets popular I wonder if something like TypeScript or Facebook's own Flow would start to make sense.

                                          • psykotic 10 years ago
                                            > IIRC, Lua is even more liberal than Python in that if you mistype a variable name, it will silently give you 'nil' rather than throwing an exception.

                                            Like most things in Lua, that is easy to customize. You can just add an __index metamethod to environment tables. There are various 'strict' modules which do just that.

                                            • technologia 10 years ago
                                              Thats what we love about Lua, it gives us control of the side effects.
                                            • technologia 10 years ago
                                              So some of our team were a bit disoriented with the lack of reference counting and for others it was their first foray into functional programming.

                                              Our team first picked up the Little Schemer and then jumped directly to lua.

                                            • muser 10 years ago
                                              Could you provide some steps how you could bring everyone one board? We are still having python/R migration issues.
                                              • technologia 10 years ago
                                                Most of our people came from a stats background using R or python (more focused on Scipy/numpy than any other package).

                                                A few of us had some experience with Lua before with game programming, so we took it on ourselves to teach the others Scheme (almost like an intro to programming class) and then teach how Lua worked in comparison.

                                                I guess we lucked out because they were able to pick it up quickly after the education on functional programming.

                                              • Houshalter 10 years ago
                                                Torch is really cool. I wish I could use it more. Sadly it isn't supported on Windows.
                                                • technologia 10 years ago
                                                  Initially we had to use Windows for some client requirements, because of that we used Vagrant+Docker to deploy torch. You might want to try that if you can use VMs.
                                              • UXDork 10 years ago
                                                Does anyone know what Facebook is using the AI for?
                                                • raverbashing 10 years ago
                                                  Basically everything in FB has one main goal: Ads

                                                  Secondary goals might be: Spam detection and extracting info from the graph/pictures, etc

                                                  FB identify people from your Social Graph in Pictures, for example

                                                  • amelius 10 years ago
                                                    The software runs on Nvidia hardware. Not sure if all their servers are fitted with one or more GPU cards...

                                                    I think it is more likely that this is just research software.

                                                    • Houshalter 10 years ago
                                                      Deep neural networks are usually trained on GPUs which can provide a huge speed up, and is much cheaper. However they can easily be run on CPUs after being trained. I don't see why they wouldn't have access to GPUs in production though.
                                                      • raverbashing 10 years ago
                                                        Well, it depends, you can definitely have servers with NVidia hardware, and you don't need to have all of them with it. (Amazon offers GPGPU servers on EC2 for example)
                                                      • pkfrank 10 years ago
                                                        Also, identifying trends in users engaging in predatory behavior, etc.
                                                        • WillNotDownvote 10 years ago
                                                          "Basically everything in FB has one main goal: Ads"

                                                          Is this actually true, or a glib answer? Ads certainly don't feel like much of the FB experience, nowhere near the level of Google, or even Twitter. They're negligible, really.

                                                          Whereas shaping the feed and identifying people in pics, etc, is central to the FB experience.

                                                          • dragonwriter 10 years ago
                                                            > Ads certainly don't feel like much of the FB experience, nowhere near the level of Google, or even Twitter.

                                                            Facebook creates an artificial distinction between labeled ads (which are, of course, obvious ads) and posts with paid reach (which are also ads, but presented as normal content), exactly for the purpose of maximizing the quantity of ads while minimizing the impression of content being dominated by ads.

                                                            • raverbashing 10 years ago
                                                              Not necessarily Ads in the traditional sense, but FB also sells content reach (Promoted Content) and targeting

                                                              Shaping the feed is certainly one part of it. But I guess that part of the success of FB is that they don't make it, as you said, not make the ads feel as much part of the experience.

                                                              • choppaface 10 years ago
                                                                Categorization is critical to boosting precision in information retrieval tasks. Deep networks for cropped object recognition are an important source of (inferred) content categories (especially when other context, e.g. photo captions, are not useful). While a good image categorizer would be very useful for ad targeting and auctions, the technology can also boost precision for news feed, graph search, etc., which improves user experience.
                                                                • samolang 10 years ago
                                                                  Agreed. You can't show ads unless you have users. Optimizing advertisements is definitely second fiddle to optimizing the user experience.
                                                                  • kylestlb 10 years ago
                                                                    Someone with more context can answer this better but I would guess that selling behavioral data to ad agencies is a thing?
                                                                    • 10 years ago
                                                                      • 10 years ago
                                                                    • anantzoid 10 years ago
                                                                      To further personalise the user feed.
                                                                    • frik 10 years ago
                                                                      Wow great. I dreamed of an AI platform based on LuaJIT a few years ago. Torch and FB's deep-learning module sound amazing!