EleutherAI announces it has become a non-profit
259 points by stellaathena 2 years ago | 105 comments- dvt 2 years agoNot sure why the AI community has a weird obsession with being non-profit (or not). It's not like non-profits can't be evil, and it's not like OpenAI has done anything particularly egregious, either.
What we do need is more weights being released in the public domain (hard to find even on Huggingface), easier ways to train models locally, better pruned models for embedded device inference (e.g. running on a Jetson Nano), easier ways to fine-tune for specific contexts, and so on. My big gripe, and for obvious reasons, is that we need to step away from cloud-based inference, and it doesn't seem like anyone's working on that.
- stellaathena 2 years agoYou're in luck! EleutherAI has trained and released open source weights of several LLMs, including GPT-Neo (2.7B parameters), GPT-J (6B parameters), and GPT-NeoX (20B parameters). This last model is currently tied for second on the list of the largest open source LLMs in the world.
We also developed VQGAN-CLIP and CLIP-Guided Diffusion, techniques for doing text-to-image synthesis that don't require training and can easily be run locally for inference.
- ImprobableTruth 2 years agoI don't want to be too cynical, but OpenAI used to be more open too until they decided releasing weights was too dangerous (/not profitable enough?), what guarantee is there that Eleuther doesn't also close their doors at some point?
- jerpint 2 years agoYes but in their wake they still left great contributions (CLIP, whisper, etc). So far they still seem net positive, even if they’re now for profit
- stellaathena 2 years agoI mean, ultimately there isn’t one. I’m just providing examples of how we fulfill the things that the OP says they want, as they seem unaware of our work.
But I’m confused by the anti non-profit vibes in this comment section. We aren’t saying that becoming a non-profit makes us ethical people, that would be a silly argument. But people do realize that the alternative would be to become a for-profit entity right?
We’re still the same community-driven open collaborative research lab we’ve always been. But incorporating allows us to do things like hire full time staff, enter organizationally binding legal agreements, and protect our members. Between the options of becoming a for-profit and a non-profit, the later seems clearly better suited for our goals.
- jamilton 2 years agoThere is no guarantee, but it sounds like if they didn't incorporate they wouldn't be doing much of anything, public or not.
- 2 years ago
- jerpint 2 years ago
- ImprobableTruth 2 years ago
- josephjacks 2 years agoSam doing a 180 on non-profit to for-profit was egregious, IMHO. Having said that, plenty of extremely profitable organizations masquerade (literally and figuratively) as non-profits.
Also: https://twitter.com/elonmusk/status/1626516035863212034
- versteegen 2 years agoIt's disruptive technology and an enormous amount of value is likely to be created. I would much rather the profits go to a nonprofit which can distribute them charitably rather than them giving them away freely to the megacorps which actually have the resources to run these massive models (edit^2: at scale). I do think the 100x cap on return on Microsoft's $1B investment was way too high (we don't know what the cap is on later investments but it's meant to be reducing over time).
Edit: Plus it's still the case that OpenAI aren't putting profit above all else. E.g. they just released the ChatGPT API and it's 10x cheaper per token than GPT3! And their Charter does count for something, it's binding on them.
- ketzo 2 years agoA nonprofit wouldn't have to distribute profits to charity. It just can't distribute profits to private or public shareholders (because it can't have them). A nonprofit is obligated to reinvest any earnings towards its mission.
Not remotely an expert, this is just my basic understanding. "Nonprofit" is a pretty abused term.
- photochemsyn 2 years agoI just looked up the the top CEO salaries of non-profit organizations in the USA - as of 2020, they ranged from about $16 million to $8 million. A non-profit appears to simply be a 'non-stock corporation' and can have the same ridiculouly pyramidal compensation structures that shareholder corporations do.
Overhauling the legal definition of non-profit to require the highest-paid employees to make no more than ~10X as much as the lowest-paid employees would make a lot of sense.
- bayindirh 2 years agoWell, "they are doing something amazing, so ethics can be suspended for T amount time in the name of value" is equally egregious from my point of view.
- 2 years ago
- ketzo 2 years ago
- dragonwriter 2 years ago> Having said that, plenty of extremely profitable organizations masquerade (literally and figuratively) as non-profits.
Non-profit doesn’t mean the organization isn’t supposed to have surplus revenue, it means it doesn’t exist to return profits (e.g., via a claim on the assets of the company) to stockholders or some other beneficiary with a claim on them.
- mb7733 2 years ago>Having said that, plenty of extremely profitable organizations masquerade (literally and figuratively) as non-profits.
That's not a contradiction. A non-profit can be profitable. The profit just needs to be reinvested back into the mission of the organization.
- gowld 2 years agoOpenAI is a nonprofit that ownes a for-profit. Same as Mozilla. OpenAI was never a charity.
A privately owned for-profit company can do whatever it wants, good or evil. OpenAI is not publicly-owned.
Did you donate money to OpenAI?
A nonprofit is an organization that uses its income and profits for the organization's main goal that supports the mission. On the other hand, a charity is a type of nonprofit that engages in activities aimed at improving lives in the communities.
NFL was a non-profit trade association until ignorant whiners made it bad PR and they changed it to shut up the whiners.
- RobotToaster 2 years ago>plenty of extremely profitable organizations masquerade (literally and figuratively) as non-profits.
Isn't Ikea technically a non-profit? through some weird structuring?
- leesec 2 years agoCapped profit*
- nwoli 2 years agoCapped at 100x investment which would be the most profitable company in history lol
- nwoli 2 years ago
- zitterbewegung 2 years agoSort of meta that Sam did something that Elon Musk would do…
- versteegen 2 years ago
- vikp 2 years agoIf you want more open research and weights, you should be happy about this announcement. Incorporating as a nonprofit doesn't guarantee that an organization will act ethically, but it does make it more likely. Nonprofits have more restrictions around how they can spend their money. They also pay less, so the teams is more likely to be mission-aligned.
The profit motive pushes organizations to keep their work secret (which is what has happened with OpenAI).
- braingenious 2 years ago> Not sure why the AI community has a weird obsession…
Can you elaborate on this “obsession”? Aren’t a lot of AI things made/implemented by for-profit companies? Google, Meta, Midjourney, etc?
- __loam 2 years agoYeah lol. The obsession is understanding this is going to be pretty impactful technology and thinking it might be nice to not have that be closed source for once
- __loam 2 years ago
- fl0ps 2 years agoI'm with you there on hoping for more concentration on local training and inference, especially on something like the new Orin (open to donations, NVidia!).
I think there's still something of a financial disincentive to promote selfhost over cloud capabilities for just about every party involved except the selfhoster. NVidia loses out if they're selling a few Nano or Orin platforms relative to the much pricier datacenter cards sold in huge lots. The cloud hosters are making up for the cost of hosting in terms of end-user pricing, snarfing all that lovely customer data and likely selling analytics, and probably other measures I can't understand yet. And the large companies that fund research and initial model development want to know what's being done with their models so as to gain any possible competitive advantage. They can't necesarily guarentee that intel from a self-hoster. Almost nobody is willing to spend the time explicitly to make it easier for the individual dev at a small lab or at home to do this because that's essentially a donation and not a business expense that might yeild obvious returns.
- bootsmann 2 years ago> easier ways to train models locally, better pruned models for embedded device inference (e.g. running on a Jetson Nano), easier ways to fine-tune for specific contexts
I think there are steps being taken in this direction (check out [1] and [2] for interesting lightweight transpile / ad-hoc training projects) but there is a lack of centralized community for these constrained problems.
[1] https://github.com/emlearn/emlearn [2] https://github.com/BayesWitnesses/m2cgen
- underlines 2 years agoI was looking for open alternatives to self hosted, or crowd hosted finetuned LLMs like ChatGPT and found LAION Open Assistant. Then found resources to further optimize inference as well as training:
- Open source fine tuned assistants like LAION Open-Assistant [1]
- inference optimizations like VoltaML, FlexGenm Distributed Inference [2]
- training optimizations like Hivemind [2]
1 https://github.com/LAION-AI/Open-Assistant
2 https://github.com/underlines/awesome-marketing-datascience/...
- warent 2 years agoIt seems like a rift between "Enlightenment" and "Authoritarian High Modernism." People want this progress to benefit reason and humanities, not contribute to our backsliding. Being nonprofit is at least symbolic of enlightenment.
- xwdv 2 years ago> Not sure why the AI community has a weird obsession with being non-profit (or not)
A lot of times, I find the obsession isn’t a desire for collaboration or anything noble, it’s as simple as this: people want to use the end products for free or very cheaply.
- Bloating 2 years ago> It's not like non-profits can't be evil
There's no shortage of non-profits that exist to pay saleries. As an example, growing up in the bible-belt, and I can confirm that religion can be good business.
- chologrande 2 years agoI can't help but wonder if non-profit makes it easier to get access to or train on certain datasets/models.
- dustingetz 2 years agotaxes!
- stellaathena 2 years ago
- rahidz 2 years agoI hope Eleuther will continue to focus on the development of truly open-source models, which seems to be where the big gap is. The quality gap between Neo-20X and even the original GPT3 is still rather large, and making headway there would be amazing.
- riku_iki 2 years agothere will always be gap, since corps can grab open model, and add resources to improvements on top of it.
- Taek 2 years agoThis is not the case with Stable Diffusion. The open source community is moving faster and innovating faster, and the open models are by far the best. Midjourney is a contender, but it doesn't hold a candle to the best open models
- wokwokwok 2 years agoLet’s be honest though; the number of people willing to devote their time to tireless effort generating porn (or suggestive images) is >> the number who will ever be willing to do it for text models.
Add the size barrier (an order of magnitude > SD for text models) and it’s pretty questionable the open source text models ecosystem is going to go anywhere.
Hardly anyone has done anything interesting with any of the existing ones and they’ve been out for longer than SD has.
Text models just aren’t that interesting to people.
- riku_iki 2 years agoI think this space is in initial choke chaos.
Eventually commercial entities will build pipelines which will absorb open models and build on top of them using proprietary tech, data and expensive compute.
- wokwokwok 2 years ago
- Taek 2 years ago
- riku_iki 2 years ago
- jackblemming 2 years agoThey're the real OpenAI! Great team.
- behnamoh 2 years agoIf only they had chosen a better name... "Eleuther" is hard to pronounce correctly for someone who doesn't know the word.
- Dalewyn 2 years agoOne measure of likelihood for success is how well the name rolls off the tongue.
- ftxbro 2 years agohow do you pronounce it correctly
- spapas82 2 years agoIf it's from the greek word for freedom (ελευθερία) then i guess elefther: ele (like electric) f the (like thesis) r.
- danuker 2 years agoI couldn't find the IPA even in Wiktionary, but I found out where it comes from: https://en.wikipedia.org/wiki/Eleutheromania
- layer8 2 years agoLike the Eleuthera island in the Bahamas I would think: https://youtube.com/watch?v=3-F-7WKXG14
So Eh-Luther AI.
- bilsbie 2 years agoI’d been saying E Luther. Is that all wrong?
- RobotToaster 2 years agoI assume similar to luthier?
- spapas82 2 years ago
- coffeebeqn 2 years agoWhich is everyone
- ttctciyf 2 years agoEverytime I see it, I think of John Fowles' The Magus.
See, for example: https://pzo31b.p3cdn1.secureserver.net/wp-content/uploads/20... (via https://www.fowlesbooks.com/1966-fowles-letter-to-high-schoo... )
- ttctciyf 2 years ago
- Dalewyn 2 years ago
- behnamoh 2 years ago
- supermatt 2 years agoWhile I appreciate what Eleuther have done - why are they highlighting "research" consisting of removing a bit of whitespace from some source code... is it April 1st already?
Heres my addition! Or subtraction, i guess... (-4 chars by changing call to AdaptiveAvgPool2d):
from torch.nn import* def c(h,d,k,p,n):S,C,A=Sequential,Conv2d,lambda x:S(x,GELU(),BatchNorm2d(h));R=type('',(S,),{'forward':lambda s,x:s0+x});return S(A(C(3,h,p,p)),*[S(R(A(C(h,h,k,1,k//2,1,h))),A(C(h,h,1)))for _ in[0]*d],AdaptiveAvgPool2d(1),Flatten(),Linear(h,n))- 2 years ago
- jamilton 2 years agoJust having a bit of fun.
- 2 years ago
- bilsbie 2 years agoI followed along on their discord for a while but I wasn’t sure how to gradually increase my engagement to get involved.
- victorbjorklund 2 years agoAwesome. Remember your great work in GTP-Neo
- whitten 2 years agoWhat was special about GPT-nano ?
- v64 2 years agonot nano, Neo, and its successor NeoX, one of the largest public GPT models available https://blog.eleuther.ai/announcing-20b/
- victorbjorklund 2 years agoVery big LLM. Was almost as good as GTP-3. And it was open source.
- v64 2 years ago
- whitten 2 years ago
- victor9000 2 years agoI really appreciate how the work of EAI has truly focused on making models accessible to the general public.
- eachro 2 years agoHow do they pay for their compute?
- stellaathena 2 years agoWe have a number of donors including Hugging Face, Stability AI, Nat Friedman, Lambda Labs, and Canva that make our work possible. We also have some orgs that provide sponsorship for computing resources specifically: Stability AI, CoreWeave, and Google Research.
- ftxbro 2 years agoare you bottlenecked by engineering
- stellaathena 2 years agoYes, we are currently bottlenecked primarily by engineering manpower.
- stellaathena 2 years ago
- ftxbro 2 years ago
- ronsor 2 years agoDonations and sponsors such as Stability AI, based on the announcement in their Discord server
- mperham 2 years agoWith money. A non-profit can still have plenty of revenue.
- stellaathena 2 years ago
- valzam 2 years agoEasy to become a non-profit if you've never made any money...
- kobalsky 2 years agobold claim to make on a site linked to growing companies by burning money.
- kobalsky 2 years ago
- hinkley 2 years agoNow I have Lenny Kravitz songs stuck in my head.
- quartzbox 2 years agoEleuther has been at the forefront of AI, without them none of this would be possible ,thanks and kudos
- return_to_monke 2 years agosorry for killing the mood. great article and great progress.
but. didn't openai start as a nonprofit, too?
- loa_in_ 2 years agoOpenAI has some complicated setup about this. Part of it is non-profit and part of it isn't and there's some rules they laid out on how to keep it non profit first from what I've gathered
- infinityio 2 years agoto my knowledge - started 100% non-profit, the split into partial profit came later when they started having a chance to make money
- infinityio 2 years ago
- loa_in_ 2 years ago
- whitten 2 years agoAs a three letter acronym (TLA) aficionado, the page uses TPU wwhich isn’t explained.
- inconceivable 2 years agotensor processing unit. a processor designed to do large, fast matrix operations.
- archgoon 2 years agoTPU - Tensor Processing Unit. They're essentially Google GPU's.
- inconceivable 2 years ago
