What if the Winners of Generative A.I. are Open-Source?

Scoops by Semianalysis and The Information shed light behind the curtain at OpenAI and Google.

Hey Everyone,

While OpenAI decided to be a commercial secretive startup with funding in the Billions on the favors and deep pockets and Microsoft, is that even the optimal approach for innovation in Generative A.I.?

Information Scoop: OpenAI’s Losses Doubled to $540 Million as It Developed ChatGPT. May try to raise $100 Billion to develop AGI.

As reported by The Information, we now have a glimpse behind the curtain: In OpenAI’s case, the figure totals $540 million in losses last year as the startup developed ChatGPT. Then there is the supposed $700,000 a day simply to run ChatGPT.

SemiAnalysis

had a scoop recently that claimed Google is worried about Open-Source disrupting them.

OpenAI had mentioned in early 2023 that and warned that the research strategy that led to the development of the AI model has reached its limits in terms of scalability and huge models. It’s believed GPT-4 has about 1 Trillion parametres.

The widespread suggestion is that open-source models are improving rapidly because people are able to iterate on top of each other's improvements and try out a much larger number of experiments than a small team at a single company possibly could. So the incredible talent at places like Google DeepMind and OpenAI may not be able to match the Open-Source community in the long-run.

I think here we have to give Meta A.I. some credit for its contribution to the Open-Source movement as yet another alternative to competition in LLMs and Generative A.I. innovation.

Source: https://lmsys.org/blog/2023-03-30-vicuna/

The supposed leak out of Google is a bit vague though as SemiAnalysis notes:

While our models still hold a slight edge in terms of quality, the gap is closing astonishingly quickly. Open-source models are faster, more customizable, more private, and pound-for-pound more capable. They are doing things with $100 and 13B params that we struggle with at $10M and 540B. And they are doing so in weeks, not months.