UK-headquartered chip design company Arm has extended its version-nine (Armv9) architecture to the far edge to handle AI workloads directly on IoT devices of various types, including vehicles, cameras, machines, and sundry cellular IoT sensors. It has introduced a new Armv9 edge AI platform, which it says is “optimised for IoT”. It called it a world-first, and commended its integration with its brand new Cortex-A320 and Ethos-U85 central (CPU) and neural (NPU) processing units.
“The AI revolution is no longer confined to the cloud,” it declared in a blog post. Combination with the Cortex-A320 and Ethos-U85, for processing and acceleration, supports “AI models of over one billion parameters to run on-device”, it said. The Cortex-A320 delivers a 10-times (1,000 percent) uplift in ML performance and a 30 percent uplift in scalar performance compared to its Cortex-A35 predecessor, apparently; it brings advanced AI capabilities and developer benefits to IoT, it said.
Its introduction means Softbank-owned Arm has the IoT space covered for Armv9 CPUs – with products “ranging from ultimate performance, to cost and energy-constrained devices”. Its v9 (Armv9.2) architecture brings support for SVE2 for ML performance and security features like Pointer Authentication (PAC), Branch Target Identification (BTI) and Memory Tagging Extension (MTE) to the smallest Cortex-A IoT devices, often handling sensitive data in business/mission critical edge environments.
As well, Arm has extended its Kleidi software for AI inference workloads to edge IoT devices. Kleidi comprises a set of compute libraries to optimise AI and ML on Arm-based CPUs without “additional developer work”. KleidiAI is already integrated into IoT AI frameworks (such as Llama.cpp and ExecuTorch or LiteRT) via the XNNPACK library (of floating-point neural network inference operators) to accelerate performance of key models (such as Meta Llama 3 and Phi-3).
The blog announcement suggested Kleidi AI brings “up to 70 percent more performance” to the Cortex-A320 CPU (“when running Microsoft’s Tiny Stories dataset on Llama.cpp”, for example). Paul Williamson, senior vice president and general manager of Arm’s IoT business, explained: “This matters because… time-to-market can make or break… a product.” The new platform also maintains software compatibility with higher-performance Cortex-A units.
Williamson said: “AI will shift towards the edge and this platform will be a catalyst for the next wave of IoT innovation. The platform’s ability to run tuned large and small language models (LLMs and SLMs) for agent-based AI apps opens up new categories of edge use cases. We’re moving toward a future where intelligent decision-making happens closer to the point of data collection, reducing latency and improving privacy.”
He added: “This isn’t just another incremental step forward – it represents a fundamental shift in how we approach edge computing and AI processing. For the first time, we’re seeing an Armv9 CPU specifically optimized for IoT applications, bringing together ultra-efficiency and advanced AI capabilities in a way that hasn’t been possible until now.”
Advantech, AWS, Eurotech, Renesas, and Siemens lent their support in the press note.
Miller Chang, president of embedded sector at Advantech, said: “The evolution of edge AI is accelerating, and advancements in Arm’s IoT computing architecture will bring new possibilities for intelligence at the edge… [This is] a significant step for the broader Arm ecosystem, enabling smarter, more efficient, and secure AI-driven applications across industries. This innovation will drive industry growth and technological breakthroughs in the edge computing market.”
Yasser Alsaied, vice president of loT at AWS, said: “The new Arm edge AI platform will enable our customers to run nucleus lite, a lightweight device runtime of AWS IoT Greengrass for constrained edge devices with minimal memory needs, on Armv9 technology. This seamless integration between the two technologies provides an optimized solution for developers to build modern edge AI applications like anomaly detection in precision agriculture, smart manufacturing, and autonomous vehicles.”
Marco Carrer, chief technology officer at Eurotech, said: “Arm’s new edge AI platform provides us with the foundation to build the next generation of rich IoT devices, with Armv9 giving us access to new levels of secure performance, energy-efficiency and software flexibility.”
Daryl Khoo, vice president of embedded processing products at Renesas, said: “We are excited about the latest Armv9 Cortex-A320 CPU, which delivers high AI/ML performance and enhanced security with power and area efficiency. It will allow us to innovate at pace and implement efficiency with scalability.”
Herbert Taucher, vice president of research and pre-development for IC and electronics at Siemens, said: “The new Armv9 based edge AI platform will help to extend our portfolio of highly secure, performant and energy efficient AI innovation to all of our customers, across a range of industrial, smart infrastructure and mobility applications.”
