Enterprise AI scalability is running into a hard budget wall for some customers. As agentic workloads multiply, cloud token bills are consuming allocations faster than planned, prompting companies to rethink where inference should run.
That cost pressure is reshaping enterprise AI deployment strategy, with on-premises infrastructure and open-weight models moving back into the conversation alongside frontier models in the cloud. Some customers are quickly exhausting token budgets, according to Helen O’Sullivan (pictured, left), AI business development manager and solutions specialist at Dell Technologies Inc.
“A lot of our customers I support in the Bay Area, they’re dealing with this real immediate struggle of running through their token budget in a matter of weeks that they had allotted for months,” she said. “You still use those frontier models, but you use the open source and infrastructure on-prem to at least eliminate some of the token stress and token maxing going on right now.”
O’Sullivan and Tim Wood (right), sales director for the Northwest region at Intel Corp., spoke with theCUBE Research’s John Furrier at the Dell “AI Leadership Symposium,” during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed enterprise AI scalability, rapid cloud token budget exhaustion and leveraging existing central processing unit infrastructure for agentic workloads. (* Disclosure below.)
Balancing AI scalability and cost
Agentic AI is changing the balance of processor demand. Many buyers remain focused on graphics processing units even as their workloads evolve, O’Sullivan noted.
“You can no longer do AI just for the sake of doing AI. You need to have a goal in mind that is in line with the corporation’s goals,” he said.
Agentic AI is also changing the hardware mix. Intel’s agentic AI architecture spans orchestration, inference and edge computing. O’Sullivan’s team anticipated the rebalance months ago, though many buyers remain fixated on graphics processing units, she noted.
“We saw the rebalance and the heavier utilization on the CPU when it comes to agents. Customers are getting there,” she said. “They’re not quite there yet from the scope I’m seeing.”
Enterprises can evaluate deployment options before committing to production. Testing open-weight models in sandbox environments can help companies assess what their workloads require, Wood explained.
“A lot of enterprises already have CPUs in their data center. There may be existing resources and headroom in the infrastructure they already have that’s capable of doing some of these next-generation apps and workloads,” he said.
Both companies are taking a consultative approach, weighing existing infrastructure and capital versus operating expense models before recommending a plan. Enterprise AI scalability requires assessing the whole stack, O’Sullivan noted.
“A lot of [customers] just see the [graphics processing unit] and [ask], ‘How fast can I get the GPU?’ or Vera Rubin or what have you,” she said. “But it’s about the data.”
Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of Dell’s “AI Leadership Symposium”:
(* Disclosure: TheCUBE is a paid media partner for Dell’s “AI Leadership Symposium.” Neither Dell, the sponsor of theCUBE’s event coverage, nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)