Voice AI is quickly becoming core infrastructure as people swap screens and clicks for conversations with their devices. That shift puts new pressure on the compute, storage and partnerships behind the models doing the listening and talking.
Deepgram Inc., which expanded its Flux model to 10 languages earlier this year, needs infrastructure for both its in-house research team and tens of thousands of external customers, according to Nick Mann (pictured), staff technical program manager at Deepgram.
“We have 45 of the smartest researchers you’ve ever seen who can go prospect and find amazing things,” Mann said. “You know what they like to eat? Compute. Just tons of compute, tons of data.”
Mann spoke with theCUBE Research’s John Furrier at the Dell “AI Leadership Symposium,” during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed voice AI infrastructure, how Deepgram keeps improving its models and the push to run voice AI on personal devices. (* Disclosure below.)
Scaling voice AI from the lab to the laptop
Deepgram’s 13 years in the market have given it the know-how to tune models for noisy settings such as air traffic control and emergency medical services, according to Mann. Still, the company assumes any model it ships will soon be surpassed and treats each release as a candidate for retraining.
“Let’s say it was trained 10 times to get as good as it was when we released it,” Mann said. “What’s stopping us from retraining it on better data?”
Those models also run outside Deepgram’s own cloud. A large percentage of customers host the models themselves, and a recent partnership with Fortanix Inc. brings private, on-premises voice AI to regulated industries, according to Mann. Dell Technologies Inc. supplies Deepgram’s infrastructure while also running its models in Dell’s Customer Solution Center Innovation Lab.
“The great thing about a partner like Dell … is they also have a huge amount of customers themselves that range from large [to] small,” Mann said. “There’s enterprise, there’s small companies.”
That reach runs from a small business handling interactive voice response on a compact Nvidia Corp. GB10 system to far larger deployments, Mann noted. But a larger problem he’s chasing sits outside the data center: the AI-enabled laptops students now carry to school.
“I want our models on those, on the hardware, to run natively,” he said. “Air-gapped. That’s where we want to get to. That’s where AI as a whole needs to start meeting people where they are and how they’re going to use it.”
Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of Dell’s “AI Leadership Symposium“:
(* Disclosure: TheCUBE is a paid media partner for Dell’s “AI Leadership Symposium.” Neither Dell, the sponsor of theCUBE’s event coverage, nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)