In September, Amazon said it would invest up to $4 billion in Anthropic, a San Francisco startup working on artificial intelligence.
Soon after, an Amazon executive sent a private message to an executive at another company. He said Anthropic won the deal because it agreed to build its AI using specialized computer chips designed by Amazon.
Amazon, he wrote, wanted to create a viable competitor to chipmaker Nvidia, a key partner and king in the all-important field of artificial intelligence.
The boom in genetic artificial intelligence over the past year has revealed just how dependent the big tech companies are on Nvidia. They can’t build chatbots and other AI systems without a special kind of chip that Nvidia has mastered in recent years. They’ve spent billions of dollars on Nvidia’s systems, and the chipmaker isn’t keeping up with demand.
So Amazon and other industry giants—including Google, Meta, and Microsoft—are making their own AI chips. With these chips, the tech giants could control their own destiny. They could cut costs, eliminate chip shortages, and eventually sell access to their chips to businesses that use their cloud services.
While Nvidia sold 2.5 million chips last year, Google spent $2 billion to $3 billion building about a million of its own AI chips, said Pierre Ferragu, an analyst at New Street Research. Amazon spent $200 million on 100,000 brands last year, he estimated. Microsoft said it had started testing its first AI chip.
But that job is a balancing act between competing with Nvidia while working closely with the chipmaker and its increasingly powerful CEO, Jensen Huang.
Mr. Huang’s company accounts for more than 70 percent of AI chip sales, according to research firm Omdia. It provides an even larger percentage of the systems used to create artificial intelligence. Nvidia’s sales are up 206% over the past year, and the company has added about a trillion dollars in market value.
Nvidia’s revenue is a cost to the tech giants. Orders from Microsoft and Meta made up about a quarter of Nvidia’s sales over the past two full quarters, said Gil Luria, an analyst at investment bank DA Davidson.
Nvidia sells its chips for about $15,000 each, while Google spends an average of just $2,000 to $3,000 for each of its chips, according to Mr. Ferragu.
“When they encountered a salesman holding them over a barrel, they reacted very strongly,” Mr Luria said.
Companies are constantly courting Mr. Huang, trying to be at the forefront of his brands. He regularly appears on stage at events with their CEOs, and companies are quick to say they remain committed to their Nvidia partnerships. All intend to continue offering his brands alongside their own.
While the big tech companies are moving into Nvidia’s business, it’s moving into its own. Last year, Nvidia launched its own cloud service where businesses can use its chips and is funneling chips to a new wave of cloud providers, such as CoreWeave, that compete with the big three: Amazon, Google and Microsoft.
“The tensions here are a thousand times the usual game between customers and suppliers,” said Charles Fitzgerald, a technology consultant and investor.
Nvidia declined to comment.
The artificial intelligence chip market is projected to more than double by 2027, to about $140 billion, according to research firm Gartner. Venerable chip makers like AMD and Intel also make specialized AI chips, as do startups like Cerebras and SambaNova. But Amazon and other tech giants can do things smaller competitors can’t.
“Theoretically, if they can reach high enough volume and can lower their costs, these companies should be able to provide something that’s even better than Nvidia,” said Naveen Rao, who founded one of the first artificial intelligence chips. ups and later sold it to Intel.
Nvidia makes so-called graphics processing units, or GPUs, which it originally designed to help render images for video games. But a decade ago, academic researchers realized that these chips were also very good at building the systems, called neural networks, that now drive genetic artificial intelligence
As this technology took off, Mr. Huang quickly began modifying Nvidia’s chips and related software for artificial intelligence, and they became the de facto standard. Most of the software systems used to train AI technologies were adapted to work with Nvidia’s chips.
“Nvidia has great chips, and more importantly, they have an incredible ecosystem,” said Dave Brown, who runs Amazon’s chip efforts. That makes using a new kind of AI chip “very, very challenging” for customers, he said.
Rewriting software code to use a new chip is so difficult and time-consuming that many companies don’t even try, said Mike Schroepfer, a consultant and former chief technology officer at Meta. “The problem with technological development is that so much of it dies before it even starts,” he said.
Rani Borkar, who oversees Microsoft’s hardware infrastructure, said Microsoft and its ilk needed to make it “seamless” for customers to move between chips from different companies.
Amazon, Mr. Brown said, is working to make switching between chips “as simple as possible.”
Some tech giants have succeeded in making their own chips. Apple designs the silicon in iPhones and Macs, and Amazon has deployed more than two million of its own traditional server chips in its cloud data centers. But such achievements take years of hardware and software development.
Google has the biggest lead in AI chip development. In 2017, it introduced the tensor processing unit, or TPU, named after a type of computation vital to building artificial intelligence. Google has used tens of thousands of TPUs to build AI products, including its online chatbot, Google Bard. And other companies have used the chip via Google’s cloud service to build similar technologies, including the high-profile start-up Cohere.
Amazon is now using the second generation of Trainium, its chip to build AI systems, and has a second chip built just to serve AI models to customers. In May, Meta announced plans to work on an AI chip tailored to its needs, though it’s not yet in use. In November, Microsoft announced its first AI chip, Maia, which will initially focus on powering Microsoft’s AI products.
“If Microsoft makes its own chips, it makes exactly what it needs at the lowest possible cost,” Mr. Luria said.
Nvidia’s rivals have used their investments in high-profile artificial intelligence start-ups to fuel their chip usage. Microsoft has committed $13 billion to OpenAI, maker of the chatbot ChatGPT, and its Maia chip will serve OpenAI technologies to Microsoft customers. Like Amazon, Google has invested billions in Anthropic and also uses Google’s AI chips.
Anthropic, which has used chips from both Nvidia and Google, is one of the few companies working to build artificial intelligence using whatever specialized chips they can find. Amazon said that if companies like Anthropic used Amazon’s chips on a larger scale and even helped design future chips, doing so could lower costs and improve the performance of those processors. Anthropic declined to comment.
But none of these companies will overtake Nvidia anytime soon. Its chips may be expensive, but they are some of the fastest on the market. And the company will continue to improve their speed.
Mr. Rao said his company, Databricks, trained some experimental AI systems using Amazon’s AI chips, but built the biggest and most important systems using Nvidia chips because they provided higher performance and played nicely with a wider range of software.
“We have many years of hard innovation ahead of us,” Amazon’s Mr. Brown said. “Nvidia is not going to stand still.”
