State of AI Report 2024 Summary

Part I, Consolidation, emergence and adoption. 🚀

Hey Everyone,

2024 was the year that frontier labs like Anthropic and OpenAI converged, and in fact Claude outdid ChatGPT in many areas while both frantically building Enterprise AI products.

This was a long report and took me two entire days to come up with. It’s not just a reader-friendly summary of the State of AI report but contains tons of additional insights, tangents and grounded in more realism.

(☝—💎For a limited time get a Yearly sub for just $5 a month✨— ☝).

The State of AI Report analyses the most interesting developments in AI and it’s by Nathan Benaich and Air Street Capital. They are based in the UK so it may have some European perspectives a bit more highlighted.

Was this Newsletter forwarded to you? Sign up easily. Check out the full body of my work.

---

As an emerging tech analyst (with several related Newsletters) I make it my business to dive deeply into such reports and report back to you on what I find most salient. Air Street Press has a Substack and the section I like to follow the most is called Guide to AI. The annual report also has a dedicated Twitter/X handle.

Out of the $79.2 billion total raised by cloud firms, 40% of all funding went to generative AI startups, Accel said. Read more.

I go on many tangents in this State of AI report summary, but try to return mostly to the report. I’m not a VC firm whose job it is to hype up the industry and I am a critical observer.

AI is eating software

Much of the growth of funding in cloud is being driven by excitement around AI. Much of capex in BigTech is being eaten by datacenters and AI. Some Generative AI sales is impacting SaaS and Cloud software revenue. It’s a weird period in 2024.

The cloud industry has been having a tough time beyond AI, with enterprise software budgets squeezed by macroeconomic and geopolitical risks. Oddly, not a single company in Accel’s Euroscape index has seen revenue growth of more than 40% per year this year, compared with 23 businesses achieving the feat in 2021. So what’s really going on?

IPOs will tell the Full Story

With so much news and AI news overload, it’s getting harder to find and get well balanced summaries. We’ll have a lot better data on the actual business of the new crop of Generative AI Startups actually start to go public as the IPO market begins to pick up again likely in 2025 or 2026.

Even just two years after ChatGPT went viral, adoption of Generative AI remains fairly murky and inconsistent both for consumers and the Enterprise sector. This ambiguity is not well covered in this State of AI report. Take all data around Generative AI as such, with a grain of salt especially if the makers of the reports have money on the line like a VC.

The hype of the first two years since ChatGPT was impressive, but the pace of acceleration and what remains will be the key.

“The only AI report in the universe has landed, covering LLMs, science, the US vs China, NVIDIA's reign of terror, which start-ups actually make money, regulators, economic impact, technical safety, and more.” - Alex Chalmers, State of AI Report team. (source)

🟣 Generative artificial intelligence startups are getting 40% of all the venture capital funding that flows into cloud companies, according to venture investors Accel.

✨ Google NotebookLM Audio Overview on State of AI Reports, 2023 and 2024.

🎧 Listen to 19 minutes.

• An AI generated “overview” of a State of AI report that is itself an overview on the progress of AI in 2024. The NotebookLM synthetic podcast is actually citing the slides (references) so I decided to share it.

🟢 The new Blackwell B200 GPU and GB200 Superchip promise significant performance gain over the Hopper architecture of H100 fame. NVIDIA claims it can reduce cost and energy consumption 25x over an H100. In a mark of NVIDIA’s power, every major AI lab CEO provided a supporting quote in the press release.

It’s a New World, Not Jetsons but “Jensens” and the Taiwanese

Blackwell Economics - Nvidia’s big surge

Amid growing demand for its hardware to power demanding gen AI workloads, every major lab depends on NVIDIA for its hardware. Its market cap hit $3T in June, only the third US company to reach this milestone (following Microsoft and Apple). Following blowout earnings in Q2, its position looks as unassailable as ever. See Slide 86 for more Industry related insights.

Race you to the top? Compute Index: NVIDIA A100 clusters

The number of large-scale NVIDIA A100 GPU clusters has stayed constant as the industry focuses its dollars on the H100 and shinier Blackwell systems.

Emperors of the AI Clusters

A LinkedIn post by Peter Gostev, highlights pretty nuts bit of trivia from SemiAnalysis and Dylan Patel - Meta was building a data centre under the old specifications, but as Nvidia released GB200s, it turned out that they needed data centres with liquid cooling - which would unlock 9x inference and 3x training uplift vs H100s.

So in the space of c.18 months, Meta have demolished their data centre and have re-built a new one on the same spot, that could take advantage of the new speeds. If they hadn't done so, they might have been disadvantaged for years to come. I did not independently verify the story.

This decision was part of a strategic shift towards developing new AI-ready designs that can accommodate the increasing demands of artificial intelligence applications. The old design was deemed insufficient for the future needs of the company, particularly as the industry moves towards higher power densities necessary for AI workloads. Read about Datacenter anatomy.

Generative AI Companies are Scaling Faster