Nvidia Corp.’s first AI infrastructure boom concentrated enormous amounts of compute. The next opportunity will be distributing it through high-speed network fabrics.
Everyone knows that I love stories that connect Silicon Valley to Wall Street. Well, two important events this week did just that, the just-concluded Hot Chips conference at Stanford University and today’s monster blowout earnings from Nvidia.
The narrative surrounding AI infrastructure has reached a fever pitch. Nvidia’s latest earnings prove that the world’s appetite for AI compute remains insatiable. Today Nvidia posted $96.2 billion in revenue for the quarter ended July 26, and guided for $108 billion in the current period.
With data center revenue surging and hyperscalers dropping massive capital expenditure into monolithic, liquid-cooled mega-clusters, Chief Executive Jensen Huang made it clear that the AI infrastructure buildout is fully underway as Nvidia’s flagship Vera Rubin AI chips are in full production.
“AI has reached its inflection point,” he said. “It’s doing useful work. Its tokens are productive and profitable. Now, compute is revenue.”
Yet underneath the headline numbers lies a stark physical reality. Modern NVLink-scale architectures require up to 140 kilowatts (or more) of power density inside a single physical rack. For massive centralized facilities built from the ground up, that is a manageable engineering task. But as AI expands outward from centralized training to distributed inference, autonomous systems and enterprise operations, it hits a hard physical wall: The existing edge data centers cannot handle the physical weight, cooling or power density of a modern AI rack.
How does Nvidia keep the revenue and profit machine cranking to create the next billions of dollars? The answer is the distributed edge or AI at the Edge.
This next multibillion-dollar market isn’t just about building larger AI factories in the desert. It is about taking the AI scale-up domain and extending it into places where conventional rack-scale systems physically cannot fit.
The edge paradox: 30 gigawatts stuck in 30-kilowatt racks
Across telecom central offices, industrial campuses, regional facilities and enterprise datacenters sits a vast, installed infrastructure footprint representing roughly 30 gigawatts of aggregated power capacity (my estimate from the past year conversations and data gathering).
The problem? It is fragmented.
A typical telco site or enterprise facility is thermally and electrically capped at roughly 30 to 50 kilowatts per rack. Attempting to drop a 140-kilowatt monolithic AI factory unit into these environments breaks power delivery, overloads liquid cooling capabilities and violates physical building limits.
This creates a structural impasse. The industry has plenty of power at the edge in the aggregate, but it cannot absorb the monolithic density required by top-tier scale-up fabrics.
The paradigm shift: Breaking the physical chassis
To solve this density mismatch, forward-thinking infrastructure architects are turning to a simple yet radical proposition: Stop treating the physical rack as the computer. This is a stark change from today’s rack scale system need for big AI clouds, or neoclouds.
Instead of trying to jam a 140-kilowatt footprint into a single cabinet, architects are taking that exact compute envelope, disaggregating it across four or five 30-kilowatt physical racks, and interconnecting them with a high-speed networking scale-up fabric.
Because optical interconnects provide massive bandwidth without the strict distance-and-power penalties of copper links, these geographically adjacent physical nodes behave logically as one unified, low-latency AI system.This transforms the entire deployment paradigm. You do not ask the customer to tear down a facility or spend millions retrofitting a building’s electrical system. You adapt the compute topology to fit the facility.
Disruption via market creation
In classic disruption theory, disruptive architectures rarely win by attacking the incumbent head-on in high-end environments. They enter where the incumbent physically cannot go, establish a beachhead and scale upward.
High-speed networking disaggregation creates a market expansion strategy:
By disaggregating physical hardware over optical or superfast ethernet switching layers, operators do not need to displace existing hyperscale infrastructure. They enable net-new enterprise and distributed deployments that would otherwise be impossible.
The strategic verticals: Monetizing power, telcos and enterprise ‘AI outposts’
This architectural breakthrough unlocks three massive market shifts:
1. The telco pivot: From transport to intelligent services
Telecommunications companies have spent a decade attempting to monetize 5G investments beyond simple data caps and bandwidth plans. By injecting disaggregated AI compute into regional central offices, operators can aggregate fragmented local capacity into a distributed AI factory. The edge transforms from a passive transport pipe into an active, programmable inference platform handling real-time model routing, security and context processing close to the end user.2. Workload orchestration: Compute moves to power
Historically, data centers brought power to where the compute was racked. In a distributed scale across (optical/photonic) topology, intelligent orchestration software shifts AI workloads across geographically separated nodes based on real-time variables: power availability, cooling efficiency, local energy pricing, latency and data proximity. The resource being virtualized is no longer just compute — it is compute, network, power, cooling and geography managed as a unified pool. This drives the research around Systems of Intelligence and Systems of Execution we’ve recently published.
3. Enterprise ‘AI outposts’
Amazon Web Services Inc.’s Outposts for general cloud compute were the conceptual pioneer that brought cloud to the enterprise. The enterprise AI model is adopting an outpost design optimized for real-time inference and scale-up which bring intelligence to the physical world.
Enterprises adopt this AI Outpost model for three critical reasons:
- Ultra-low latency: Millisecond-level responses required for autonomous agents, medical imaging and computer vision.
- Data governance and sovereignty: Sensitive corporate or patient data remains within single-tenant local token economics: Shifting continuous local inference from pay-per-token public APIs to fixed local capital infrastructure.
Robotics: The leading indicator for physical AI and edge AI
If you want to know when edge AI has truly arrived at scale, watch the robotics and autonomous systems sector or as many call the physical AI sector.
Robots, automated factory floors, smart warehouses and autonomous vehicles generate continuous streams of sensor data that demand real-time inference. They cannot tolerate a 50- to 100-millisecond round-trip to a distant centralized cloud.
However, a manufacturing facility does not need a $3 million hyperscale rack sitting next to an assembly line. It needs small, modular, distributed AI nodes networked across the campus — looking far more like campus networking infrastructure than a miniature data center.
The evolution: AI infrastructure moves to the swarm
Just as enterprise networking evolved from monolithic mainframe connections into distributed campus switches, AI compute is following the exact same evolutionary path.
The future of the next billions of dollars for Nvidia in AI infrastructure is not bound by the just the big AI factories or the small AI factories requirements.
These new smaller footprints of power, land and shell as it is called are bound to the physical dimensions of a 19-inch server rack or a 140-kilowatt power delivery system.
Where’s the money?
By decoupling the physical cabinet from the logical compute domain via high speed networking scale-up fabrics, the industry is entering an era where the rack stops being a physical chassis and becomes a logical scope. Compute can finally go wherever power, cooling and latency dictate — turning every node at the edge into part of one massive, distributed AI computer.
AI infrastructure is not a bubble and the above is what we’ve been seeing and documenting at theCUBE and NYSE Wired.
Nvidia’s first AI infrastructure boom was about concentrating enormous amounts of compute. The next one may be about distributing it.
Tell me where I’m wrong.