AI factory deployment has long been a race to secure graphics processing units, but the builders of those systems are finding that compute is only half the problem. Data pipelines, storage costs and the clock on getting a cluster running now weigh as heavily as raw horsepower.
The companies supplying that infrastructure are rethinking where the planning should begin. Customers that once focused only on standing up the biggest GPU clusters are now puzzling over how to feed them with data, according to Ben Lee (pictured), director of solution management at Super Micro Computer Inc.
“In the past, we really, really focused on the computational resources, right?” Lee said. “But recently we figure out that people need to start looking from the data perspective.”
Lee spoke with theCUBE Research’s John Furrier at the AI Data Pipeline Forum, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed how data pipelines, full-stack validation and dedicated deployment teams help customers bring AI factories online faster. (* Disclosure below.)
AI factory deployment starts with the data pipeline
Supermicro’s customers often discover the data problem after the GPU build is done. This year, soaring solid-state drive and memory prices have made them hesitant to size storage clusters, yet the money already sunk into compute leaves little choice, Lee explained. Supermicro now plans storage alongside compute with partners such as Vast Data Inc.
“We still talk about the GPU workload, size the GPU cluster. We know this is very important. But now we partner with a strong partner like VAST,” Lee said.
Supermicro treats the AI factory as one big computer and pitches single accountability for it, an approach that extends to its AI factory management work with Broadcom Inc. That spans servers, storage built with partners such as ScaleFlux and its Data Center Building Block Solutions, which the company says scale from 5 megawatts to 1 gigawatt. As deployments grow, the fundamentals matter most, Lee noted.
“Fully validate, because every small things will just create the big issues for the whole big clusters,” Lee said. “So you have to really go into every single detail for the validation, down to the component level, down to a system level, and to the rack level and the cluster level.”
That discipline now sets the pace of AI factory deployment. Supermicro brought xAI’s liquid-cooled Colossus-1 cluster online in 120 days, with a tiger team coordinating supply chain, rack and field engineering work, according to Lee.
“Then we talk about a time to deploy, and more important, time to be operational,” Lee said. “This is really important because when you start to put the money and you have to pay Supermicro in certain way, you want to see your whole system or whole cluster online in very, very short period.”
Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of the AI Data Pipeline Forum:
(* Disclosure: ScaleFlux Inc. sponsored this segment of theCUBE. Neither ScaleFlux Inc. nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)