The Biggest Open-Source Week in the History of AI

The last week of March, 2024 will go down as a unique moment for Open-source LLMs. China's open-source scene hits the ground running.

Hey Everyone,

While covering AI this week I noticed something peculiar. Open-source LLMs had a coming out party. 🎉 I’m not usually one to brag or exaggerate, but what we saw this week was unusual.

The Hugging Face folk were noticing it too:

Like Omar Sanseviero (X, here).

To follow what HuggingFace notices bookmark this page.

• DBRX, by Databricks (MosaicML really)

• Jamba, by A21 Labs

• Qwen1.5, by Alibaba Cloud

• Samba-CoE v0.2 by SambaNova Systems

• Starling-LM-7B-beta by NexusFlow (Berkley)

• xAI’s Grok 1.5

• Mistral’s 7B v2

• Wild 1-bit and 2-bit quantization with HQQ+ by Mobius Labs, see this paper too.

• Earlier in March we saw, SaulLM-7B for Law.

If you believe this is a good overview of Open-source innovation at this time, support the author and share it.

Of course many predicted that 2024 would be a great year for Open-source AI, but also Agentic AI and AI devices are showing promise as 2024 trends in AI. But in my mind, Jamba, DBRX and Samba-CoE are all incredibly unusual and specific launches of Open-source LLMs demonstrating a pivotal moment in the diversification and proliferation of these accessible and decentralized AI models.

Billionaire Elon Musk said Friday his artificial intelligence company xAI's chatbot Grok-1.5 “should be available” to the public next week, after the chatbot became open-source and officially entered the rapidly growing AI chatbot market.

The current generative AI revolution wouldn’t be possible without the so-called large language models (LLMs) being optimized constantly and globally simultaneously in a more or less decentralized manner. However it’s open-source LLMs that are making many new things possible in how companies train their proprietary data on their own models to have full control and customization.

But behind every AI tool or feature, there’s a large language model (LLM) doing all the heavy lifting, many (to now most) of which are open-source. As these periods like March, 2024 occur, new possibilities will emerge. The density of launches of quality open-source LLMs in March, 2024 is peculiar.

“In my mind Jamba, DBRX and Samba-CoE are all incredibly unusual and specific launches of Open-source LLMs demonstrating a pivotal moment in the diversification and proliferation of these accessible and decentralized AI models.”

Gap Between Open-Source and Closed-Source is Narrowing