What if World Models and Quantum Computing complemented LLMs?
The race to a unified AI of everything.
Good Morning,
I envision a world where LLMs are complemented by many types of AI, Quantum computing and a convergence of new kinds of computing architectures. For all the book smarts of LLMs, they currently have little sense for how the real world works and there’s a growing list of “world model” AI startups trying to fill the gap. People won’t trust a one dimensional AI, more on this at the end.
While Generative AI models have their uses, I believe the next decade will fill in some of the physical AI gaps, and allow for things that were previously not possible. Yann LeCun, Fei-Fei Li and others are working towards this end. The likes of Google, Meta, Tencent, Nvidia and others are of course also developing world models.
Robotic brain startups like Physical Intelligence (π) are building general purpose models for physical AI that are basically foundational models for the physical world to help robots learn among other things. Language isn’t enough and models trained on text have inherent limitations. The token based way of predicting the world is just one perspective. A world-model based AI could have advantages over LLMs that understand casualty, and maintain mental models that are more human-like and grounded. This could enable more efficient decision making AIs, to actually make the world a better place inspired by cognitive neuroscience and grounded in empirical experience.
Yann LeCun, Gary Marcus, Packy McCormick and many others make a lot of good points about the inherent limitations of LLMs and the merits of World Models, and I think this is important to point out even as Silicon Valley fuels capex that isn’t rational compared to what the future AI will actually be like. Of course that is just my opinion.
My guest contributor today is James Wang of Weighty Thoughts Newsletter, a VC on AI and DeepTech. 📚 His book What you Need to know about AI, recently won this year’s American Legacy Award Best Nonfiction . It was also a finalist in the business technology category. Congratulations to James.
Weighty Thoughts ∘˙○˚.•
My Weights Thoughts of the day ⚗️
A little Quantum tangent on our World Models exploration today:
Nvidia recently announced Nvidia Ising that the company says might be the world’s first open AI models to accelerate the path to useful Quantum computing. It’s exciting because it’s being piloted by the likes of Academia Sinica, Fermi National Accelerator Laboratory, Harvard John A. Paulson School of Engineering and Applied Sciences, Infleqtion (that is now a public company under the ticker INFQ 0.00%↑) , IQM Quantum Computers, Lawrence Berkeley National Laboratory’s Advanced Quantum Testbed and the U.K. National Physical Laboratory (NPL).
 | A guest post by
|