As AI workloads move from training to real-world inference, Arrcus CEO says, network fabrics must evolve to keep up with the demands
At Mobile World Congress Barcelona, Arrcus outlined how the rise of AI inference at the edge is driving demand for intelligent, policy-aware network fabrics.
The shift from large-scale model training in data centers to real-world inference at the edge has opened networks to a gamut of new use cases — autonomous driving, oil drilling, retail points of sale, precision-guided farming — requiring extreme throughput, sub-second latency, and policy awareness.
Side by side, demand for low-cost compute, data sovereignty, security, and power supply too have shot up.
Supporting this new class of inference workloads requires reimagining the network architecture, says Shekar Ayyar, CEO of Arrcus.
“The old world ideas around things like load balancing and caching, which applied to websites, are no longer just easily translatable into the world of AI,” Ayyar said in an interview with Sean Kinney of RCR Wireless News.
He argued that a new type of distributed architecture that connects edge nodes to training nodes and routes traffic intuitively between them best serves inference workloads.
“Furthermore you need to have the policy richness to impose that on top of all of these steering points or routing points,” he added.
The Arrcus Inference Network Fabric (AINF) is purpose-built for this, he said, adding that it is essentially “policy-aware AI”.
AINF leverages a distributed architecture designed to connect training clusters to edge nodes where inference workloads run. It offers a set of granular policy controls on top that allows operators to define how traffic should flow based on the needs of the workloads.
“You can take those policies…latency reduction, throughput improvement, power optimization or sovereignty for localization of data, and use them to create the most efficient inferencing architecture,” Ayyar said.
New partnerships
At MWC, Arrcus announced partnerships with multiple vendors, building on the work to expand inference infrastructure at the edge.
One of the collaborations announced is with Fujitsu whose AI inference chip, MONAKA, an Arm-based processor, and optical interconnects will now power AINF to build what would be a “completely efficient AI infrastructure for inferencing for customers,” Ayyar said. The company also has ongoing partnerships with Nvidia, Broadcom and others for silicon integration.
It also announced a partnership with Lightstorm — a cloud connectivity provider in Asia — integrating its network-as-a-service (NaaS) solution, Polarin, with AINF.
“The two together are now going to be able to support large customers, hyperscalers, as well as other enterprise customers as they enter markets in the Asian geography,” Ayyar said.
The company is also teaming up with UfiSpace and Lanner to deliver AI-optimized hardware solutions for data centers and AI companies.
Taken together, the announcements suggest that the age of AI inference is upon us. As Ayyar put it, “Training is about taking really good big large models, adding more and more intelligence to it…Now it’s time to figure out how to use those models effectively for results.”
