Enterprises moving artificial intelligence from pilot projects into production are discovering that the hard part is no longer the model. It’s the infrastructure beneath it. Cost, tokenomics, data privacy and the manual labor of stitching together graphics processing units, servers, networking and software stacks have become gating factors for deployment at scale.
Those pressures are pushing more production AI workloads back into the data center, where organizations can keep models close to their data. Broadcom Inc. is betting that turnkey automation can simplify that shift, according to Prashanth Shenoy (pictured, right), chief marketing officer and vice president of the VMware Cloud Foundation division at Broadcom.
“A lot of our customers are looking at private cloud in an on-premises environment to deploy their production AI workloads at scale. But as they’ve been trying to do this, it’s been a very complex process from what we call the metal to model,” said Shenoy. “Setting up GPUs, servers, networking, Kubernetes, containers, AI software stack, testing, validating which models to use. It’s an extremely manual and complex process.”
Shenoy and Raghu Nambiar (left), corporate vice president of software and solutions at Advanced Micro Devices Inc., spoke with theCUBE Research’s Christophe Bertrand and co-host Alison Kosik at VMware Explore, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed how validated AMD-accelerated VMware Cloud Foundation infrastructure could provide a more automated and flexible path for deploying AI models alongside existing applications. (*Disclosure below.)
Hardware choice underpins the AI factory model
Operational automation is one half of Broadcom’s AI factory pitch; hardware flexibility is the other. That matters because most enterprises already run traditional and container workloads on the same platform and want AI to arrive without a parallel infrastructure to manage alongside it, Shenoy noted.
“They already have the AI factory built in with VCF,” he said. “We have automated this to do a lot simpler way of deploying.”
AMD’s contribution spans both compute tiers, with the recently launched MI350P PCIe accelerator aimed at enterprises starting out. The company has more than 1,600 vSAN ReadyNodes in market across major server suppliers, Nambiar noted, and sizing guidance now follows model scale.
“If your problem size is 10 billion parameters, CPU is the answer,” he said. “But if you’re looking at the 100 billion parameters range, then MI350P is the answer. If you have a larger model, 1 trillion plus, then MI355X is the answer.”
Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of VMware Explore:
(* Disclosure: TheCUBE is a paid media partner for VMware Explore event. Neither Broadcom, the sponsor of theCUBE’s event coverage, nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)
