Agentic AI infrastructure must support the work between a model’s decisions as agents move from answering questions to executing tasks.
Central processing units handle much of that execution, while graphics processing units power model reasoning. Those complementary roles shape Nvidia Corp.’s planned Vera CPU deployment with CoreWeave Inc., according to Hannah Coutand (pictured, right), director of product marketing, Vera CPU, at Nvidia.
“The reason why Vera is so important is because we recognize that extreme co-design means that we look across the AI factory and we want to make sure we solve for any inefficient bottlenecks,” she said. “We recognized the CPU was becoming one of them. We didn’t set out to say, ‘Let’s go and build CPUs.’ We set out to solve this bottleneck.”
Coutand and Harsh Banwait (left), senior director of product at CoreWeave, spoke with theCUBE Research’s Dave Vellante and John Furrier at the Fully Connected event, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed CPU requirements, sandbox performance and infrastructure security. (* Disclosure below.)
Agentic AI infrastructure puts CPU performance under pressure
As training and inference become increasingly intertwined, infrastructure must accommodate both model computation and the environments where agents act. Tool calls, application programming interface calls and SQL queries increase the execution work handled by CPUs, Coutand explained.
“I think the volume certainly plays a role,” she said. “That’s why having a CPU that handles those types of calls extremely well, with fast, beefy cores, high memory bandwidth and low latency, it handles both operating in this new world … and serves as a great foundation for simply agentic use cases [and] reinforcement learning.”
CoreWeave announced plans to offer standalone Vera CPU capacity alongside its existing infrastructure. Its Sandboxes offering provides isolated environments where agents can execute tasks during reinforcement learning and inference, Banwait explained.
“Quite recently, we also tested that with Vera CPUs, and we were proud to share that we saw about a 3x improvement in performance in terms of Sandbox startup times,” he said. “For us, it’s a very important combination of getting the performance that we need from the silicon and getting our customers the experience that they expect from a product.”
Coordinating security and performance across the AI factory
Security also shapes agentic AI infrastructure. Nvidia’s Open Agent Safety Platform combines OpenShell runtime controls with Nvidia Sentry running on BlueField-4 data processing units to provide an independent layer of monitoring and enforcement.
“They’re all in a single Vera Rubin tray, which includes Vera CPU as well as the DPU; that’s all included in the same hardware infrastructure,” Coutand said. “So, Open Agent Safety Platform, the secure runtime, which is OpenShell, runs on the Vera CPU in that tray, and DOCA Sentry runs on the DPU part.”
Operating that infrastructure requires attention to power, cooling and automation. Networking, CPUs and storage also figure in CoreWeave’s engineering priorities, Banwait explained.
“All of that needs to be able to keep up,” he said. “We’re going to continue to focus on wherever the bottleneck is so that when the entire system is kind of advancing, it’s doing that in one cohesive way. Otherwise, the weakest link in the chain kind of holds it all back.”
Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of the Fully Connected event:
(* Disclosure: TheCUBE is a paid media partner for the Fully Connected event. Neither CoreWeave, the sponsor of theCUBE’s coverage, nor other sponsors have editorial control over the content on theCUBE or SiliconANGLE.)