NVIDIA's Vera CPU heralds a redefinition of computing architecture from a traditional, task-isolated paradigm to an orchestrated, agentic intelligence ecosystem, shifting the role of the central processing unit (CPU) from a general-purpose multitasker into a specialized conductor designed specifically to manage the chaotic, multi-step reasoning loops of AI agents.

Recently, Wccftech got an opportunity to sit down with NVIDIA's Vera team for an informative tête-à-tête on the emerging role of the CPU in the rapidly growing agentic AI economy. So, delve in as we discuss the rebirth of single-threaded performance, the staggering scale of CPU-controlled AI sandboxes, and the evolving CPU-to-GPU ratio.

After years of cloud infrastructure prioritizing massive core counts at the expense of per-core speed, agentic AI is forcing a reversal, and NVIDIA's Vera is a manifestation of this paradigm shift

Question: Can you provide a brief overview of what the Vera CPU is all about?

Answer: In the early 2010s, data center design focused on maximizing the performance of individual processor cores rather than the sheer quantity of cores. During this period, market leaders like Intel focused on architectures (such as the Haswell generation) that featured fewer cores running at higher clock frequencies.

This era sat in the top-left quadrant of CPU design, characterized by high single-threaded performance but low overall throughput.

"That's a time period when, when Intel was leading in data center market segment share... you have high single threaded perf but low throughput because you don't have as many cores."

As massive cloud hyperscalers grew, the industry pivoted toward scale economics. The demand shifted toward packing as many cores into a single server as possible to maximize monetization per server unit. To achieve this density and control costs, x86 vendors transitioned away from single-chip designs to multi-chiplet layouts. However, this massive expansion in core count came at the direct expense of single-core optimization and Instructions Per Cycle (IPC).

This era occupied the bottom-right quadrant—offering massive throughput but low single-threaded performance.

"...what suffered over that time in terms of CPU design was really single-threaded perf and IPC because when you fill, when you use chiplets, you just, you literally don't have the space on the, on the chip to design for per core perf."

The rise of Agentic AI has placed the CPU back at the center of the hardware design path. AI agents operate in a continuous, rapid loop between the GPU and the CPU. Because GPUs are the most expensive and valuable assets in an AI factory, the CPU must complete its sequential tasks instantly to prevent the GPU from sitting idle. This paradigm requires an unprecedented combination of speed, memory access, and minimal lag.

"...you want the agent to complete whatever sequential work on the CPU it needs, but then go back to the GPU for the next reasoning step because your GPUs are still among your most valuable assets in your AI factory, and so you don't, you want to keep those assets, expensive assets highly utilised..."

NVIDIA’s Vera CPU solves the agentic bottleneck by breaking the historical trade-offs of the cloud era, positioning itself in the elusive top-right quadrant - high single-threaded performance coupled with high throughput. It accomplishes this through three specific design choices: high per-core speed, a custom LPDDR5X memory subsystem for massive bandwidth, and a single monolithic compute die powered by the second-gen NVIDIA Scalable Coherent Fabric to eliminate latency.

"...so it's like a single monolithic die, all the cores on that, single monolithic compute die, all the cores on that die can talk to each other without crossing any IO boundaries and incurring any latency because of that, so that's really the sort of the setup for what Vera is, why we think it's differentiated, it's really kind of sets this new category, like if we come back to the quadrant I was describing, it's neither the pre-cloud CPU nor is it the cloud CPU, it's like the top right box of the quadrant, which is like high single-threaded perf as well as high throughput."

TL:DR: Vera is NVIDIA’s first purpose-built CPU for agentic AI, designed to address the specific bottlenecks of the modern AI era. While the cloud era prioritized throughput with dense, low-performance chiplet cores, agentic AI requires a fast loop between the GPU and CPU. To keep expensive GPUs highly utilized, agents demand high single-threaded performance, high memory bandwidth, and low latency. Vera achieves this through a monolithic compute die, a custom LPDDR5X memory subsystem, and a second-generation Scalable Coherent Fabric, placing it in a new category that combines high single-threaded performance with high throughput.

Disaggregation in CPU function, Olympus cores, and a performance tax from spatial multithreading

Question: With SK Hynix announcing HTPU (Hynix Transaction Processing) - where you basically have about 100 tiny processors inside an SSD handling repeated low-level tasks, leaving the general processor to focus on high-level tasks - is NVIDIA interested in or pursuing a somewhat similar disaggregation in CPU functions?

Answer: The AI infrastructure landscape is shifting away from uniform, all-in-one server nodes toward highly specialized, heterogeneous setups. A prime example of this trend is seen in modern AI inference workloads, where the distinct phases of processing - the initial reading of data (pre-fill) and the sequential generation of tokens (decode) - are now routinely split across entirely separate hardware clusters to optimize efficiency.

"I mean, there's a lot of, I would say, high-level, there's a lot of disaggregation happening in the AI infrastructure space in general, like if you just take inference and how you separate pre-fill and decode on separate infrastructure, now that's largely a trend in heterogeneous infrastructure, that's very much a trend in general."

NVIDIA's original CPU strategy [seen with the Grace processor] focused on creating the ultimate "host node" - a processor designed purely to feed data to a cluster of GPUs within a server rack. However, the unique demands of agentic workflows and reinforcement learning have forced a pivot, especially as NVIDIA realized that there was a use case for pure CPU racks to handle agent tool calls and reinforcement learning tasks.

"...we were focused on CPU with grace as a host node, as the 1-8 or 1-4 in a rack, we wanted the best host node possible. Then we realized, we saw that actually there is a point where you would actually want a lot of standalone CPUs, like pure racks of just CPUs because for all those agent tool calls for reinforcement learning..."

Because of this pivot, the Vera CPU is no longer viewed strictly as an integrated component trapped inside a combined GPU-CPU architecture (like the Vera-Rubin stack). Instead, Vera is being commercialized as an independent, standalone CPU asset, replete with "pure racks" of Vera processors solely to crunch through massive volumes of agent tool calls and reasoning loops.

"...that pivot is a lot like disaggregation in a way, because you're saying instead of it just being part of the Vera-Rubin stack, we think that there's an importance of having Vera standalone CPU.

So I guess in a way to answer your question, that pivot with Vera in and of itself is a step in that direction. But I don't know, like in the particular use case you described, if we have plans along doing something very similar."

TL; DR: While unable to comment on the specific SK Hynix use case, the broader trend in AI infrastructure is heavily moving toward disaggregation. The pivot with Vera itself represents a step in this direction, as NVIDIA realized the market needs massive standalone CPU racks for agent tool calls and reinforcement learning, rather than strictly using CPUs as host nodes within a unified stack.

Question: So NVIDIA mentions that they have around a 50 percent higher IPC than Grace, right? And the CPU also has a massive 10-wide decode width. So I was just wondering if a misprediction occurs, what's the latency like? How many cycles does it take for Vera to recover versus Grace? Can you give me a very rough outline of this?

Answer: The Vera CPU achieves a massive leap in processing power, delivering 1.5x faster performance than its predecessor, Grace. When measured against the industry's leading x86 processors specifically on agentic workloads - which rely heavily on sequential tasks like Python scripting and code compilation - Vera scales up to 2x faster, with a 1.6x advantage on a geometric mean basis.

"...we're 1.5x faster than Grace, but we're also 2x faster on agentic workloads versus x86. So not versus Grace, but versus the leading x86 part. We are 2x faster for agentic workloads."

Rather than relying on traditional Simultaneous Multithreading methods, NVIDIA implements a CPU-native capability called Spatial Multithreading. This proprietary feature allows the chip to rigidly provision dedicated "worker" and "I/O" threads. This division guarantees that even under heavy, data-center-wide compute loads, a core will never starve its primary processing threads of cycles to handle background operations.

"...NVIDIA employs their own proprietary spatial multithreading... It allows you to provision, like, one thread as your worker thread and one thread as your I/O thread. And it's a way to have more optimal resource allocation.

And that underloads, oftentimes, in other competing architectures. You might rob your workerthread of cycles in order to feed I/O or other cores because of the load. And so our spatialmultithreading actually allows you to maintain that type of resource balance."

Addressing industry concerns regarding whether this complex thread scheduling introduces architectural latency or overhead, independent testing has proven there is no performance tax. A recent deep-dive review by the third-party hardware site Phoronix verified that NVIDIA's spatial multithreading yields a notable efficiency gain.

"...we had – I think it was Phoronix who published a piece just a week ago... they confirmed that the NVIDIA SMT on Vera actually delivers about a 20 percent performance uplift. So not a tax, but in fact quite the opposite, the inverse."

TL; DR: Vera is 1.5x faster than Grace overall, and notably 2x faster than leading x86 competitors specifically on agentic workloads like Python scripting and LLVM compilation. NVIDIA’s proprietary spatial multithreading does not incur a tax; rather, it provisions worker and I/O threads to maintain optimal resource balance even under heavy loads. A third-party analysis by Phoronix confirmed that this multithreading actually delivers about a 20 percent performance uplift.

NVIDIA's view on where the CPU-to-GPU ratio might eventually land

Question: When Meta launched its Muse agent, some tidbits suggested the CPU-to-GPU ratio in AI environments could eventually hit 80:1. Is this realistic, and how does it impact Vera's demand?

Answer: While NVIDIA does not officially endorse or project the 80:1 ratio, the industry volume of agent tool calls is massive, with some sandbox providers hosting up to 400,000 sandboxes for a single customer. The infrastructure ratio is actively shifting heavily in the direction of CPUs to support potentially trillions of executing sandboxes, which is driving NVIDIA's push for standalone Vera CPU racks.

"We are generally very, I mean, to be honest, I know that folks will ask this question. We don'tfrankly, quite honestly, truthfully, we do not have a point of view on the ratio question. People ask often and we don't.

And, you can imagine why we wouldn't want to say, we wouldn't want to put anything out there because that would immediately signal something about the direction of the business.

And I'll just say that we in the CPU team really don't even, we don't have an idea of what the number is going to be, but we do believe that it's a shifting number, and it's shifting in the direction of CPUs, and hence why we are putting all this effort into making sure that the market understands what Vera is good for, that Vera is the agentic CPU. If you want to deploy standalone CPU racks for agentic or reinforcement learning, we hope that you're thinking about Vera for that."

NVIDIA Vera's monolithic die

Question: Why did NVIDIA choose a monolithic die for Vera instead of the chiplet architecture used by competitors?

Answer: Unlike legacy x86 manufacturers who must design complex, flexible architectures to serve both everyday consumers (client PCs) and high-end data centers, NVIDIA entered the standalone CPU space completely unencumbered by historical segments. This unique positioning gave them the luxury of designing Vera exclusively for a single target: the ultra-high-performance AI data center.

"...truly Vera is the first one that we're positioning after, is that we are unencumbered by segments past... AMD and Intel, they have to serve not only data centre, but they also have to serve client. They have different segments that have to serve with a single architecture... We really had the luxury of designing for the data centre market."

To satisfy the intense memory-fetching loops of AI agents and data analytics, NVIDIA deliberately rejected the multi-chiplet trend of its competitors in favor of a single monolithic die. Basically, by keeping all cores on one continuous piece of silicon, NVIDIA was able to completely eliminate the internal performance "tax" that occurs when data is forced to cross-communicate boundaries between separate chiplets.

"...having a monolithic die, where the core in the top left corner of the chip can talk to the core at the bottom right corner of the chip without crossing I/O boundaries and incurring attacks, that was a valuable design point for us... it was important to our design point for like maximising data movements for not only agentic AI, but also just straight up data processing and data analytics."

This monolithic engineering translates into a massive performance advantage for collaborative computing. The Vera CPU achieves an unprecedented 3.4 terabytes per second of bisection bandwidth, allowing individual processor cores to collaborate and share contextual information 3x faster than the leading x86 chips.

"...we can deliver 3.4 terabytes per second of bisection bandwidth, which is 3x higher than the leading x86 chip. So that means like, you know, sometimes cores have to collaborate on shared context. And so they don't, you know, it's three times faster for them to collaborate on shared context than it is for an x86 chip."

While the monolithic approach was mathematically required to achieve Vera's specific performance targets, NVIDIA maintains a highly pragmatic, agnostic stance on chiplet technology for the future. The company will pivot to chiplet designs if a future market opportunity dictates it, choosing engineering utility over rigid design philosophies.

"...I think we're just going to be very balanced about like, what are the design points we want to meet for what market opportunity?... But I would say it's sort of like, we're not, I guess I wouldn't say we don't have a religion around it. It's more like what is right for what we're trying to achieve."

TL; DR: Because NVIDIA is unencumbered by legacy segment demands [like balancing data center and client needs], it has the luxury of designing its chips strictly for data center and agentic AI. A monolithic die maximizes data movement by allowing cores across the chip to communicate without crossing I/O boundaries, yielding 3.4 terabytes per second of bisection bandwidth - 3x higher than leading x86 chips. Even so, NVIDIA remains open to chiplet architectures in the future if market demands necessitate a different architecture.

Follow Wccftech on Google to get more of our news coverage in your feeds.

Deal of the Day

Read all comments on Exclusive: NVIDIA’s Vera Team Says Chiplet Designs Sacrificed Single-Threaded Performance For Core Counts, And Agentic AI Is Now Forcing A Reversal

Trending Stories

• Newegg’s Ryzen 7 9800X3D CPU Bundle Is $189 Off And Includes A 32GB DDR5 RAM kit And ASUS Motherboard, But You’ll Love These Freebies Worth $185

  51 Active Readers

• Square Enix Vows Final Fantasy VII Revelation Is Series’ Most Challenging Game, As 3 Major Shifts Redefine Hard Mode

  31 Active Readers

• RTX 3090’s Token Generation Speed Gets Up To A 4.6x Boost Running A 27B LLM Through Just Inference Optimizations, Opening New Doors For Older GPUs

  27 Active Readers

• Fallout: New Vegas RTX Modder Kim2091 Spent Months Reworking NVIDIA’s RTX Remix, and the RPG Masterpiece Is Now Path Traced

  23 Active Readers

• Intel Core i5 4670K Pushed To 6.3 GHz, Setting A New Cinebench R11.5 World Record After 12 Years

  16 Active Readers

Popular Discussions

• NVIDIA DLSS 5 Runs 30% Faster Now, Reduces VRAM Use By 20% & Coming To Hogwarts Legacy & Where Winds Meet In November

  712 Comments

• NVIDIA DLSS Ray Reconstruction Now Works On AMD Radeon GPUs Through New Mod, Alongside DLSS SR 4.5, MFG & DLSS 5

  443 Comments

• NVIDIA’s Faster DLSS 5 Neural Rendering Model Already Modded Into Unsupported Games, Delivers Up To 40% Higher FPS

  420 Comments

• NVIDIA RTX 5060 Ti 16 GB Surges Up To 144% In Japan, Hitting $1,292 As RTX 5080 Climbs Past $2,400

  396 Comments

• NVIDIA’s RTX 5090 Disappears From Its Marketplace, Adding Fuel To Discontinuation Rumors

  329 Comments