The compute rethink: Scaling AI where data lives, at the edge
Presented by Arm
AI is no longer limited to the cloud or data centers. Increasingly, it is performed directly where data is created: on devices, sensors and networks at the edge. This shift to on-device intelligence is driven by latency, privacy, and cost issues that companies face as they continue their investments in AI.
For leadership teams, the opportunity is clear, says Chris Bergey, SVP and GM of Arm’s Client Business: Invest in AI-first platforms that complement cloud usage, provide real-time responsiveness and protect sensitive data.
“With the explosion of connected devices and the rise of IoT, edge AI offers organizations a significant opportunity to gain a competitive advantage through faster, more efficient AI,” Bergey explains. “Those who take the first step will not only improve efficiency, they will also redefine what customers expect. AI will become a differentiator in terms of trust, responsiveness and innovation. The sooner a company puts AI at the center of its workflows, the faster it will compound that advantage.”
Use cases: Deploy AI where data exists
Enterprises are discovering that edge AI is not just a performance improvement, but a new operating model. Processing locally means less dependency on the cloud and faster, more secure decision-making in real time.
For example, a factory floor can instantly analyze equipment data to avoid downtime, while a hospital can safely run diagnostic modeling on site. Retailers are deploying in-store analytics using vision systems, while logistics companies are using AI on devices to optimize fleet operations.
Instead of sending massive volumes of data to the cloud, organizations can analyze and act on insights as they occur. The result is a more responsive, privacy-preserving, and cost-effective AI architecture.
Consumer expectations: immediacy and trust
Working with Alibaba’s Taobao team, the largest Chinese e-commerce platform, Arm (Nasdaq:Arm) enabled on-device product recommendations that are updated instantly without relying on the cloud. This allowed online shoppers to find what they needed faster, while keeping browsing data private.
Another example comes from consumer technology: Meta’s Ray-Ban smart glasses, which combine cloud and on-device AI. The glasses process quick commands locally for faster responses, while more demanding tasks such as translation and visual recognition are processed in the cloud.
“Every major technology shift has created new ways to engage and generate revenue,” says Bergey. “As AI capabilities and user expectations grow, more intelligence will need to move closer to the edge to deliver the kind of immediacy and trust people now expect.”
This shift is also happening with the tools people use every day. Assistants like Microsoft Copilot and Google Gemini combine cloud and on-device intelligence to bring generative AI closer to the user, delivering faster, more secure, and more context-aware experiences. The same principle applies across industries: the more intelligence you move securely and efficiently to the edge, the more responsive, private and valuable your operations become.
Build smarter for scale
The explosion of AI at the edge requires not only smarter chips, but also smarter infrastructure. By matching computing power to workload demands, enterprises can reduce energy consumption while maintaining high performance. This balance between sustainability and scale is quickly becoming a competitive differentiator.
“Computing needs, both in the cloud and on-premises, will continue to rise sharply. The question becomes: How do you maximize the value of that computing power?” he said. “You can only do this by investing in computing platforms and software that scale with your AI ambitions. The true measure of progress is enterprise value creation, not raw efficiency metrics.”
The intelligent basis
The rapid evolution of AI models, especially those powering edge inferencing, multimodal applications, and low-latency responses, requires not only smarter algorithms, but also a foundation of high-performance, energy-efficient hardware. As workloads become more diverse and distributed, legacy architectures designed for traditional workloads are no longer sufficient.
The role of CPUs is evolving and they are now central to increasingly heterogeneous systems that deliver advanced AI experiences on devices. Thanks to their flexibility, efficiency, and mature software support, modern CPUs can run everything from classic machine learning to complex generative AI workloads. When combined with accelerators such as NPUs or GPUs, they intelligently coordinate computing power across the system so that the right workloads are running on the right engines for maximum performance and efficiency. The CPU remains the foundation that enables scalable, efficient AI everywhere.
Technologies such as Arm’s Scalable Matrix Extension 2 (SME2) bring advanced matrix acceleration to Armv9 CPUs. Meanwhile, Arm KleidiAI, the intelligent software layer, is extensively integrated into leading frameworks to automatically improve performance for a wide range of AI workloads, from language models to speech recognition to computer vision, running on Arm-based edge devices – without developers having to rewrite their code.
“These technologies ensure that AI frameworks can leverage the full performance of Arm-based systems without additional developer effort,” he says. “This is how we make AI both scalable and sustainable: by anchoring intelligence in the foundation of modern computers, so that innovation takes place at the speed of software, and not hardware cycles.”
That democratization of computing power will also facilitate the next wave of intelligent, real-time experiences across the enterprise, not just in flagship products, but across entire device portfolios.
The evolution of edge AI
As AI evolves from isolated pilots to full deployment, the companies that succeed will be those that connect intelligence across every layer of infrastructure. Agentic AI systems will depend on this seamless integration, enabling autonomous processes that can instantly reason, coordinate, and deliver value.
“The pattern is familiar, because with each disruptive wave, slow-moving incumbents risk being overtaken by new entrants,” he says. “The companies that do well will be the ones that wake up every morning asking how they can make their organizations AI-first. As with the rise of the internet and cloud computing, those who commit and become truly AI-enabled will shape the next decade.”
Sponsored articles are content produced by a company that pays for the post or has a business relationship with VentureBeat, and is always clearly marked. For more information please contact sales@venturebeat.com.
