AI factory deployment is becoming a workload-sizing exercise rather than a custom infrastructure project. Enterprises need infrastructure spanning edge inference and large-model training.
Broadcom Inc. and Cisco Systems Inc. are addressing that challenge with a validated design combining VMware Cloud Foundation and Cisco Unified Computing System infrastructure. It maps workloads to hardware, from processors for smaller models to graphics processing unit-dense systems, according to Sabina Anja (pictured, right), chief technologist, VMware Cloud Foundation Division, Broadcom.
“When we came up with the VMware Private AI Cloud, which has VMware AI Factory under that, the whole idea was: How can we give customers a solution to this massive systems problem?” Anja said. “How can you get something that’s bespoke to your outcome and integrates the best in the market for it?”
Anja and Jeff Nichols (left), technical leader at Cisco, spoke with theCUBE Research’s Christophe Bertrand and co-host Alison Kosik at VMware Explore, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed reducing deployment complexity while preserving hardware and location choices. (* Disclosure below.)
AI factory deployment shifts IT from builder to consumer
Sizing begins with requirements for latency, response time and concurrent users. Cisco’s portfolio spans Unified Edge for local inference, AI POD configurations and the eight-GPU UCS C885A M8 for demanding data-center workloads, according to Nichols.
“Typically, what we do with AI workloads is we start with the workload itself, not the system,” he said. “So what are the requirements? Is the end user going to be doing inferencing? Are they going to be doing retrieval-augmented generation, fine-tuning [or] training?”
VMware Cloud Foundation virtualizes hardware so customers can allocate processors or accelerators according to demand. With validated Cisco systems, AI factory deployment can arrive fully configured rather than becoming a continuing integration assignment.
“Really, the philosophy behind the AI factory is that the end user doesn’t have to be in there, and the IT department doesn’t have to be a perpetual AI infrastructure architect or builder,” Nichols said. “They become AI consumers. The AI factories are shipped on-site, fully built, fully configured and, with VCF on top, they’re ready to go on day one and two.”
Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of VMware Explore:
(* Disclosure: TheCUBE is a paid media partner for VMware Explore event. Neither Broadcom, the sponsor of theCUBE’s event coverage, nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)