OpenAI’s Jalapeño AI chip brings new 'threat' to Nvidia margins as custom silicon gains ground

Nvidia's near-monopoly over the most advanced AI chips is under "threat" as OpenAI's and other tech giants announce custom-built semiconductors, analysts told CNBC.

OpenAI announced that its first AI chip, the Jalapeño, had "industry-leading speed and efficiency," as it unveiled the semiconductor on Tuesday. Google, AWS and Meta are all also developing their own AI chips.

Nvidia has seen its share price rocket amid the data center buildout, which has created huge demand for its chips in both model training and inference: how AI systems run day-to-day tasks.

But hyperscalers and AI companies are increasingly gaining ground in developing their own silicon to power AI systems.

The Jalapeño chip, designed for inference, shows that a "hyperscaler-designed chip can now match or beat Nvidia's Blackwell-class GPUs on inference efficiency," Adrien Sanchez, technology analyst at Yole Group, told CNBC.

He added that, while Nvidia still owns the "vast majority" of AI compute and has ecosystem lock-in to its software platform CUDA, OpenAI's new chip is a "threat to Nvidia's inference margins, which is the field growing the most at the moment."

Nvidia has been approached for comment.

OpenAI announced the first benchmarking results from its Jalapeño, saying it would allow users to get "faster responses, more responsive agents and more reliable access" as demand grows.

The new chip is being developed with Broadcom. It will be deployed within OpenAI's compute infrastructure by the end of the year, and OpenAI said it was already working on the semiconductor's generations two and three.

OpenAI announced Jalapeño in June, saying it would be "built from the ground up for current and future LLMs across the industry."

Jalapeño is an "impressive achievement, most of all in terms of efficiency," Alexander Harrowell, senior principal analyst at Omdia, told CNBC.

"In a large-scale deployment, this would save power, cooling, and power distribution infrastructure, and contribute a lot to their unit economics," he added.

OpenAI's custom chip could reduce its reliance on Nvidia over time for inference workloads, TrendForce Analyst Fion Chiu told CNBC.

But for more compute-intensive workloads, like large-scale model training and frontier AI workloads, "we believe Nvidia GPUs will remain important given their broad programmability, performance, software ecosystem, and ability to handle a wide range of workloads," Chiu said.

Research firm SemiAnalysis said it visited OpenAI's labs to benchmark Jalapeño. It found the chip beat Blackwell on performance per watt in nearly all tested scenarios. But it added that the comparison was "somewhat incomplete and unfair" because Jalapeño uses newer HBM4 memory. Nvidia's Rubin platform is a better like-for-like comparison as it also uses HBM4.

"Jalapeño is really competing against chips like Rubin that also use HBM4," SemiAnalysis analysts said in a blog post titled "OpenAI Jalapeño: Better Than Nvidia Blackwell" on Tuesday.

"Vera Rubin systems are starting to ship to customers right now, while it will still be some time before OpenAI has anything beyond engineering samples of Jalapeño," the blog post added.

OpenAI is one of several AI companies looking to build custom silicon.

In April, a slew of deals were announced for custom AI chips, also known as application-specific integrated circuits (ASICs).

Google unveiled new chips for AI training and inference, which it calls tensor processing units (TPUs). Meta said it agreed to deploy 1 gigawatt of custom AI chips using Broadcom technology as part of a multi-GW deal. Anthropic said it committed to spending more than $100 billion on AWS tech over the next 10 years, including current and future generations of Amazon's custom AI chips, Trainium.

"Omdia expects custom ASIC chips like Jalapeño to exceed GPUs in volume by 2028, although revenue will take much longer as GPUs are considerably more expensive," said Omidia's Harrowell.

"This is the biggest competitive threat to NVIDIA, as about half the capital expenditure on AI infrastructure comes from hyperscale cloud providers who either have a custom chip program or could reasonably have one," he added.

Startups including Cerebras, SambaNova, D-Matrix, Etched and Fractile are also developing chips for AI.

OpenAI has been a huge buyer of Nvidia's GPUs as it trains and runs huge AI models. Having its own chip could affect that relationship.

Sanchez said the AI lab had been "one of the largest single consumers of Nvidia GPUs," and Jalapeño "raises the stakes for Nvidia's largest customer relationship specifically."