Nvidia is the undisputed king of AI chips. But thanks to the artificial intelligence he helped build, the champion may soon face growing competition.
Up-to-date AI runs on Nvidia’s designs, and this momentum has propelled the company to a market capitalization well over $4 trillion. Each recent generation of Nvidia chips enables companies to train more powerful AI models using hundreds or thousands of processors networked across huge data centers. One reason for Nvidia’s success is that it provides software that helps program each recent generation of chips. Maybe soon it won’t be such a diverse skill.
The so-called startup Wafer trains AI models to perform one of the most challenging and crucial tasks of AI – optimizing code to run as efficiently as possible on a specific silicon chip.
Emilio Andere, co-founder and CEO of Wafer, says the company performs reinforcement learning on open-source models to teach them how to write kernel code, which is software that directly interacts with the hardware in the operating system. Andere says Wafer also adds “agent harnesses” to existing coding models such as Anthropic’s Claude and OpenAI’s GPT to enhance their ability to write code that runs directly on chips.
Many well-known technology companies now have their own chips. Apple and other companies have been using custom silicon for years to improve the performance and efficiency of software running on laptops, tablets and smartphones. At the other end of the scale, companies like Google and Amazon produce their own silicon to improve the performance of their cloud computing platforms. Meta lately he said would apply 1 gigawatt of computing power thanks to a recent chip developed in cooperation with Broadcom. Implementing a custom silicon chip also involves writing a lot of code to make it run smoothly and efficiently on the recent processor.
Wafer works with companies like AMD and Amazon to assist optimize the software to run efficiently on their hardware. The startup has so far raised $4 million in seed funding from Jeff Dean of Google, Wojciech Zaremba of OpenAI and others.
Andere believes his company’s AI-driven approach has the potential to challenge Nvidia’s dominance. Many high-end chips now offer similar raw floating-point performance – a key industry benchmark for a chip’s ability to perform straightforward calculations – to Nvidia’s top silicon.
“The best AMD hardware, the best [Amazon] Trainium equipment, the best [Google] “TPUs provide the same theoretical drawbacks as Nvidia GPUs,” Andere told me recently. “We want to maximize intelligence per watt.”
Performance engineers with the skills needed to optimize code to run reliably and efficiently on these chips are expensive and in high demand, Andere says, while Nvidia’s software ecosystem makes it easier to write and maintain code for its chips. This makes it difficult for even the largest tech companies to operate on their own.
For example, when Anthropic partnered with Amazon to build AI models in Trainium, it had to rewrite its model code from scratch to make it run as efficiently as possible on the hardware, Andere says.
Of course, Anthropic’s Claude is one of many artificial intelligence models that currently have superhuman abilities at writing code. So Andere believes it won’t be long before artificial intelligence starts taking advantage of Nvidia’s software advantage.
“The moat is in the programmability of the chip,” Andere says, referring to libraries and software tools that make it easier to optimize code for Nvidia hardware. “I think it’s time to reconsider whether this is actually a strong moat.”
In addition to making it easier to optimize code for different silicones, AI may soon make it easier to design chips yourself. Recursive intelligencea startup founded by two former Google engineers, Azalia Mirhoseini and Anna Goldie, is developing recent ways of designing computer chips that employ artificial intelligence. If the technology is successful, many more companies will be able to move into chip design, creating custom silicon chips that run their software more efficiently.