Nvidia Corp. is extending the data processing unit from infrastructure offload to a broader security role across the artificial intelligence factory. The opportunity is to make agentic AI safer to operate at scale.
That’s according to Gilad Shainer, Nvidia’s senior vice president of networking, who sat down with theCUBE last week at our NYSE studios. The strategic implication is an even larger role for Nvidia in enterprise infrastructure. With the current climate focused intensely on AI safety, this accelerates trust. It also deepens Nvidia’s foothold on the ecosystem.
What’s new?
Nvidia is at it again. Having built out its scale-up, scale-out and scale-across network architecture, the company is introducing another category it is calling scale-in. Its purpose is to connect more of the AI factory’s resources while extending security controls across the infrastructure. In Nvidia positioning, the DPU extends beyond a device that secures access to a server and becomes a separate place to monitor and control activity across the factory.
The reason for this move is an AI system that simply answers a question presents one set of infrastructure requirements. A system in which agents make repeated model calls, retrieve information, access applications and then act, presents a more pressing challenge. Fast communication between graphics processing units remains fundamental. But operators must also control what agents can access, what actions they can take and how their activity affects other workloads.
Our premise is that agentic AI raises the value of infrastructure that can enforce boundaries independently of the agents themselves. Nvidia is using that requirement to expand the role of BlueField, its data processing unit, alongside DOCA software and the recently introduced OpenShell agent runtime.
Nvidia’s OpenShell 0.1.0 is an open-source runtime for defining and enforcing which systems and data an agent can access. According to Nvidia, it combines sandboxed execution, controlled service access, credential management and formal policy analysis. Teams can grant agents the capabilities a task requires while OpenShell enforces those permissions outside the workload.
OpenShell supports Codex, Claude Code, Pi, Hermes, and future frameworks across enterprise applications, frontier research and physical AI. The potential customer benefit is stronger control without putting all the associated burden on the systems running applications (core GPUs and central processing units such as Vera). The strategic benefit for Nvidia is a larger role in defining how an AI factory operates.
In this Special Breaking Analysis, we draw on our conversation with Shainer for theCUBE’s NYSE Wired AI Factories series. We explain what scale-in means, why agentic workloads strengthen its rationale, how Nvidia is approaching the problem and what this could mean for operators and partners. We also examine a secondary but useful investor point of why a chip version can remain valuable even after a newer generation arrives.
What scale-in actually means
The easiest way to understand scale-in is to place it alongside Nvidia other infrastructure categories.
| Infrastructure | Primary purpose in Nvidia’s positioning |
|---|---|
| Scale-up | Use NVLink to combine GPUs into a larger unit of compute. |
| Scale-out | Use InfiniBand or Spectrum-X Ethernet to connect those units into a larger GPU cluster. |
| Scale-across | Connect multiple AI factories so their compute resources can contribute to larger jobs. |
| Scale-in | Bring users, models, agents, storage and compute resources into the AI factory with security controls that extend across its infrastructure. |
Shainer describes these as complementary infrastructures serving different requirements. Scale-in adds to the architecture rather than replacing the networks that connect GPUs.
In his account of earlier AI systems, the distinction was pretty clear. The back-end network supported communication among compute resources. The front-end network provided access for clients. Scale-in expands the role of that access infrastructure so it reaches further into the AI factory.
Using two concepts originally presented on theCUBE back in 2012 by Arista Networks Inc. Chief Executive Jayshree Ullal, helps explain further: North-south traffic describes access into and out of the environment. East-west traffic describes communication within it. Shainer’s point is that securing access at the entrance is insufficient for the agentic environment Nvidia sees coming. Controls must also extend across activity inside the factory, including its compute, memory and storage resources.
BlueField-4 is central to this expansion. Shainer describes it as connected to the other infrastructures and to ConnectX network interfaces, with responsibility for security across a much broader set of traffic. He compared a previous 400-gigabit secure-access path with 7.2 terabits of secure traffic coverage in the BlueField-4 architecture.
The opening of our discussion puts BlueField-4, DOCA and Spectrum-X Ethernet together in this accelerated infrastructure-services layer. Shainer’s explanation then focused on what that layer can reach and control. The important change of note is more significant than a faster network connection in our view. It is the DPU’s expanded responsibility across the AI factory.
The key to understand is that Nvidia approach makes the AI factory’s internal resources accessible under controls that operate separately from the agents using them. The monitoring is done out-of-band and as such Nvidia has mitigated performance degradation concerns.
Agentic AI changes both the workload and the trust problem
Shainer emphasized a nuance regarding inference that can sometimes get lost in the noise.
When we look at an agent, agent is not just one inference call. There are many inference calls.
He described a flow of operations involving compute, storage, memory and networking. The key concept is that the infrastructure must support that flow, rather than treating the workload as a single request sent to a model.
Consider a hypothetical purchasing agent. It might retrieve a supplier contract, check inventory, ask a model to interpret a requirement, access a purchasing application and prepare an order. Depending on its authority, it might also submit that order.
Each step described above introduces another series of risk, accessing resources, granting permissions or creating potential dependencies. A low-latency response from a large language model does not address whether the agent should be allowed to read that contract or commit the transaction.
This is why the security question is integral to the infrastructure. The agent needs trusted access to complete its task. The operator needs confidence that this access will remain within defined edicts of the organization throughout the sequence of events.
Shainer contrasted an infrastructure organized around a coordinated training job with one supporting many agents that must remain contained within their sandboxes. Nvidia’s perspective is the latter requires security to extend across access, scale-out communication, memory and storage.
We would read this as a change in risk profile, not as a claim that training environments never needed security. The important distinction is between optimizing a large compute job to run efficiently with high GPU utilization versus controlling many independent agentic sequences across shared resources.
There is also a related data problem. Shainer said that escalating context requirements can exceed the memory available within a compute server. He identifies CMX and STX as architectural elements that use BlueField as a storage controller for context infrastructure. This is adjacent to scale-in, rather than the same thing, but it reinforces the wider point that agentic infrastructure must coordinate more than GPU-to-GPU communication.
Our takeaway is the following: The operational unit of interest must expand from the model call to the completed task. Operators should care about the speed and cost of the full sequence, including access controls and containment.
A factory that produces tokens quickly but cannot control the actions built around those tokens has not solved the enterprise deployment problem.
BlueField, DOCA and OpenShell divide the work
Nvidia’s approach combines hardware separation, infrastructure software and an agent runtime. These components each have different jobs.
BlueField-4 provides a separate place to run infrastructure controls
In our discussion, Shainer described BlueField-4 as an integration of a CX-9 SuperNIC and a Grace CPU. The network component supplies connectivity and access to infrastructure resources. The CPU supplies compute capacity to process information and take control actions.
He also discussed in-silicon security engines, telemetry collection from compute, memory and storage, and control over functions such as encryption and keys associated with ConnectX interfaces.
The distinction that is most important is separation. The infrastructure controls operate on a device outside the environment in which the agent itself is running.
That device is connected to the resources it monitors. “Separate” does not mean disconnected from the factory. It means the agent and the infrastructure controls do not occupy the same operating domain.
This also addresses our question about offload. We asked, Is BlueField reducing work from the CPU, the GPU or both?
Shainer emphasized that some of the required security work is new because the workload has changed. He also traces the DPU’s role to offloading elements of the hypervisor and isolating the application domain from the infrastructure domain. While solving a different problem, there are similarities with respect to resource isolation which we’ve seen with AWS Nitro.
The conclusion is that scale-in combines offload with additional infrastructure capability. It should not be described simply as moving an existing block of GPU work onto an older chip.
There is still CPU processing involved. In this case, some of it runs on the Grace CPU inside BlueField, in service of infrastructure control rather than the agent’s application workload.
DOCA exposes the hardware capabilities to software
Shainer explained DOCA by comparing its role for DPUs with CUDA’s role for GPUs. The hardware contains capabilities, but software needs a way to access them.
DOCA is Data Center Infrastructure-on-a-Chip. It is a unified software development framework and platform designed to program, accelerate, and secure data center infrastructure using BlueField DPUs and ConnectX SuperNICs. DOCA supplies the frameworks, software and application programming interfaces that expose BlueField’s security engines, storage acceleration and other functions.
This is another example where Nvidia’s architectural advancements have implications for the ecosystem, meaning DOCA this is the integration point. The mandate extends beyond installing a DPU in a server. Infrastructure developers and partners need to connect their software to the capabilities that the DPU provides. Otherwise, their offerings will be less integrated and less valuable to customers.
We believe this is strategically important because it makes software integration part of the value proposition. The silicon supplies the capacity. The software determines how that capacity becomes an operational service.
OpenShell supplies the runtime beneath the agents
Nvidia’s big announcement this week was around around AI safety and control. Jensen is fighting an idealogical battle with those that want to slow down AI development. His philosophy is we need to speed up AI security, safety and controls and that will serve to accelerate AI responsibly. Shainer describes OpenShell as an open-source runtime that sits beneath agents and helps enforce their permissions. He pairs it with BlueField’s separate monitoring and enforcement capabilities.
His explanation follows this rationale:
You cannot count on an agent to contain itself.
The intended division of responsibility is that the agent performs the task. The runtime constrains its activity. The DPU adds a separate infrastructure domain from which to observe and enforce controls.
Shainer uses browsers, virtualization and hypervisors as analogies for this progression. Each enabled broader use of computing by placing controls around software or users whose behavior could not simply be trusted. Nvidia is applying that same containment principle to agents.
In our view, this is the core of the scale-in story. Specifically, an agent’s instructions should not be the only mechanism limiting its authority.
Why separation matters, and what it does not prove
The architectural argument is surely compelling and we acknowledge Nvidia’s contribution in this regard. A control system that runs separately from the agent can provide an additional boundary between the agent and the resources it is permitted to use.
But several claims require more time to prove out in the field.
The first is performance. Shainer states:
There’s no performance degradation because it works out of band.
It makes sense that telemetry collection out of band does not affect GPU or storage work, but we’d still like to understand if there are any tradeoffs or out of scope items that operators need to consider. Separation provides a rationale for reducing interference with application work. Operators still need to measure the complete system with the relevant controls enabled. Our interview does not show how task completion times, resource use or enforcement behavior change under representative load.
The second issue is the difference between infrastructure visibility and business understanding.
Shainer describes BlueField as helping understand what agents are doing and enforce what they are allowed to do. The discussion does not explain the full process by which an enterprise’s business policies become those permissions, or how permissions remain associated with an agent across its sequence of actions.
We emphasize the four following points to put this issue in context:
- Business context. This remains a point of contention for us. In our purchasing example, an agent could have legitimate access to the purchasing system and still prepare the wrong order. Enforcing access to a system does not, by itself, establish that every action that is allowed reflects the business owner’s intent. Indeed, as we’ve often pointed out, Jensen’s five-layer cake doesn’t include a data layer where the system of intelligence would ostensibly harmonize data and provide the correct business context.
- Accountability. Our assessment is that containment provides a foundation for responsible operation. Enterprises still need to define authority, identify actions that require approval, retain useful evidence and assign responsibility when something goes wrong. Today the entire AI ecosystem is talking about shared responsibility. But we’ve argued that framework must evolve into shared accountability in this AI era.
- AI sovereignty. The third issue is the security of the control infrastructure itself. Moving control to a separate device creates a boundary, but it also creates a dependency. That device, its software and its administrative access need their own protection, update process and recovery procedures. Customers must be cognizant of lock-in, switching costs and ensuring the business case fits their sovereignty requirements.
- Operational model. Finally, the workload argument does not establish that every enterprise needs the same implementation. A tightly restricted assistant and a shared environment hosting agents with access to consequential business systems present different operating requirements.
We believe agentic AI strengthens the case for independently enforced controls. Whether BlueField-4 and Nvidia’s surrounding stack are the right implementation depends on the workload, existing infrastructure and evidence from deployment.
Grace offers a useful lesson about silicon useful life and monetization
There is a secondary investor point in this discussion that warrants attention without taking over the point of this story.
Shainer confirms that BlueField-4 uses Grace. During the interview, we raised the observation that Grace can remain useful in this role even as Nvidia discusses Vera elsewhere in its AI factory architecture.
This supports a narrower argument than the claim that every new chip generation makes the previous generation financially obsolete. Different functions have different requirements. A CPU design can remain useful for infrastructure services even when another design becomes the focus of a newer compute platform.
However, the following three points are worth clarifying:
- Using an established CPU design in a new product does not suggest that previously deployed physical chips are being recovered and reused. The point for investors is Nvidia can continue to add value in new use cases with older chips.
- It also does not provide a Grace-vs.-Vera cost comparison. If Vera were less supply constrained would the economics favor using it in the design?
- Most important, the useful life of a CPU design inside a DPU is different from the depreciation of an installed GPU estate.
Nonetheless, the Grace example challenges a simplistic view that older designs immediately lose relevance.
Our key point is that silicon should be evaluated by the work it can perform economically, not simply by whether a newer product exists. BlueField-4 illustrates the diversity of roles within the AI factory– we’re not claiming it resolves the broader GPU-depreciation debate.
The industry impact could extend well beyond networking
We see several likely architectural consequences if Nvidia’s approach gains broad adoption:
Nvidia’s role expands in each AI factory
Scale-in gives Nvidia another way to participate in the infrastructure surrounding its GPUs.
The opportunity full kit includes DPU hardware, network integration, software interfaces and reference designs. More important, it expands Nvidia’s influence over the operating architecture – i.e. how resources connect, where controls run and how infrastructure services are implemented.
This could make the overall system easier for customers to assemble and operate. It could also increase the switching costs and effort required to move away from Nvidia-specific integrations.
This is the central strategic tension when AI sovereignty comes into play. A more complete platform can reduce integration work while making the platform supplier more important to future decisions. It is up to the operator to control their sovereignty, with awareness of the risks and compensating controls necessary to mitigate those risks.
Additionally, don’t assume that every new infrastructure element produces an entirely new budget. Some spending could move from existing host-based functions or other infrastructure products. The key issue is how much value Nvidia and its partners capture from the total AI factory, not simply how many new categories come to market as SKUs.
Networking suppliers face a broader basis of competition
Raw connectivity remains fundamental. Scale-in raises the importance of additional capabilities – e.g. access to telemetry, programmable enforcement, integration with storage and compute, and the software needed to operate those functions.
We expect that to broaden the competitive discussion from network capacity to the combination of performance, control and manageability.
The real networking competition centers on Nvidia, Cisco Systems Inc., Arista and Broadcom Inc. Advanced Micro Devices Inc. is knocking at the door with its Pensando acquisition but is still not a major player in networking. The competitive dynamic is less a head-to-head fight and more a tiered architecture where different vendors dominate different layers as follows:
- Tier 1 — GPU fabric/intra-cluster (Nvidia’s home turf)
- Tier 2 — Spine/leaf and east-west fabric (Arista, Cisco, Juniper Networks/HPE)
- Tier 3 — Merchant silicon (Broadcom, Marvell, etc.)
Hyperscalers are an orthogonal cut through all three tiers.
We queried the Qualitate platform to capture first party customer input from experts using these networking solutions. The results are depicted in the following table, sourced from 26 in-depth conversations with experts that are using or evaluating these products:
Here’s an example of the sentiment on networking competition from an operator from the Qualitate data:
We’re primarily increasing [spend] with our existing vendors, the Arista, Juniper, Cisco and particularly Nvidia Mellanox. They are the big ones. And again, if we’re using a bunch of Nvidia servers and they’re all, say, B300s or going back a generation, H100s, H200s, they’re almost always directly plugged in to Mellanox switches. After that you can get a little more creative, like high-performance compute if you’re not GPU-heavy. You could use whatever you wanted in there, but, even there, you’re getting minimum 100-gigabit-per-second switches, if not 400 or 800, depending upon your east-west traffic demand. And that goes for storage as well. You’ve got racks and racks and racks of NVMe storage. They’re all also plugged into these switches because again, the data’s got to transfer, it’s got to transfer fast. So pretty much everything is just going warp speed inside AI data centers, whether it’s actual hard drives, NVMe, or whether it’s spinner disks, although those are a little more antiquated. Those are more for inference and long-term storage when it comes to AI anyways. It’s all definitely just massive. And those are the biggest vendors by far. Mellanox in particular, they’re owned by Nvidia. So you have a complete design advantage. – lead data acientist and AI/machine learning engineer, financial services and insurance
This does not imply that existing network suppliers disappear. It suggests that their position will depend partly on how effectively they participate in the control and services architecture around AI workloads.
Suppliers that can demonstrate interoperability, reliable operations and useful visibility across mixed environments could have an important role. Those capabilities are especially important where customers cannot or will not standardize on one supplier’s complete design.
Security and storage partners gain an integration opportunity; and a mandate
For security partners, the opportunity is to use the separate enforcement and telemetry capabilities described by Shainer while contributing policy, operational context and integration with enterprise processes.
For storage partners, the opportunity is tied to both data access and the growing context requirements Shainer identified.
The strategic risk is that some functions once sold as separate infrastructure capabilities become deeply embedded as part of a common platform foundation. Partners therefore need to be clear about the value they add above that foundation.
We believe the best opportunity lies in making the architecture work for specific customer requirements such as translating policies into effective controls or surfacing evidence to operating processes and proving agent behavior under real-world loads.
Shainer points partners toward DOCA, Nvidia’s reference architectures and the DSX AI Factory effort, which he describes as including a full reference design and digital twin. These are the ecosystem entry points he identifies in the conversation.
We do not have a complete account of deployment prerequisites, pricing or operational nuances. Partners will need to rely on their Nvidia relationships to turn the architectural story into a customer offering they can monetize beyond a channel resale.
AI sovereignty: Openness and portability will require separate evaluation
OpenShell is open source. Shainer discussed its use on platforms based on Nvidia technology. Customers in our view should distinguish visibility, access to APIs and the ability to move an implementation across suppliers. These are different levels of openness.
An open-source runtime can be highly valuable without establishing that every component of the surrounding system is open or that the complete design is portable.
We believe this will become an important point of negotiation. Customers need to understand which policies, integrations and operational data they can carry into another environment, and which capabilities depend exclusively on Nvidia-specific hardware or software.
For partners, supporting Nvidia’s architecture and preserving customer optionality don’t have to be in conflict. In fact, demonstrating both could become a source of differentiation.
The DPU is becoming more central to the AI factory
The most significant takeaway from this interview is the expanded role of a separate infrastructure domain.
Shainer’s argument is that agentic AI requires controls that extend across more resources and operate independently of the agents themselves. BlueField-4 supplies the separate hardware domain. DOCA exposes its capabilities. OpenShell supplies a runtime beneath the agents.
Our view is that Nvidia is trying to elevate the DPU from a useful offload component to a more central part of the AI factory’s operating architecture. The OpenShell announcement is timely and connects to the zeitgeist around AI safety.
These announcements can improve the economics of controlled agent execution by moving infrastructure work away from application resources and providing common capabilities across the AI factory. It could expands Nvidia’s influence over security, storage integration and infrastructure operations.
The remaining question is how well the complete design works outside the vendor’s explanation. Nvidia’s track record in this regard is impressive. Nonetheless, operators need evidence of effective containment, performance and manageable operating complexity. Partners need to show where their contribution improves those outcomes and where they can add value.
The opportunity is substantial. So is the need to distinguish an alluring architecture from a proven deployment.
Action item: Make proof of control the condition for greater autonomy
For operators and partners, make one representative agent workflow pass a live containment test before expanding its authority. Use a workflow with meaningful business value, run it under realistic conditions and deliberately attempt an action outside the agent’s permissions. Require the system to block that action and produce evidence that an operator can understand and explain to a compliance officer and a board of directors. Evaluate the performance of the complete workflow with those controls active. The decision should be based on demonstrated behavior.
For partners, build your customer offering around passing that same test. DOCA integration and Nvidia reference designs are useful starting points. The solution we envision is an operating system of controls that the customer can verify and support. Scale agent authority only as fast as you can prove independent control.
Watch the full conversation: Gilad Shainer, Nvidia | theCUBE + NYSE Wired: AI Factories — Data Centers of the Future