The application scenarios for agents are becoming increasingly specialized. From AI programming tools like Claude Code to systems capable of processing documents, managing tasks...
The application scenarios for agents are becoming increasingly specialized.
From AI programming tools like Claude Code to office assistants capable of handling documents and operating software, and even personal assistants that help users plan trips and manage their schedules, agents are increasingly taking on a growing array of concrete tasks.
The continuous expansion of application scenarios is also placing new demands on underlying computing power.
Against this backdrop,$Arm Holdings (ARM.US)$Recently, a CPU performance evaluation methodology tailored for agents has been introduced, aiming to help enterprises assess the performance of various CPUs in agent‑related workloads through tests that more closely mirror real‑world use cases.
This is not an isolated case. AMD previously introduced a CPU workload benchmarking framework for Agent AI, incorporating stages such as tool execution, data retrieval, and task orchestration into its testing; Intel has also conducted related tests on its Xeon 6 processors, focusing on the throughput and concurrent execution efficiency of Agent tasks.
Several chip manufacturers have successively explored new evaluation methodologies, driven by the fact that agents are reshaping CPU workloads.
Over the past few years, competition in large‑model computing power has largely revolved around GPUs, with metrics such as token‑generation speed and throughput serving as key benchmarks for AI inference performance.
As the agent begins executing its actual tasks, in addition to invoking large language models for reasoning, it must also access databases, execute code, and call external tools—each of these steps can consume substantial CPU resources.
Ravi Malhotra, Director of Cloud AI Solutions Architecture at Arm, used the software development agent as an example to point out that the model first performs inference and generates code, after which the CPU must still handle code building, compilation, and testing—tasks that can be extremely time-consuming.
"In these kinds of use cases, it's crucial how quickly the CPU can finish its portion of the work, so that control can be handed back as soon as possible to the more expensive GPU or accelerator, enabling the next step in the pipeline," Ravi Malhotra told media outlets including All Weather Tech. "If the CPU isn't fast enough, the pricier GPU will sit idle."
To make matters more complicated, the performance bottlenecks vary across different agents. Some tasks are primarily constrained by GPU inference speed, while others are limited by CPU sandbox execution capabilities or data retrieval efficiency. Even for agents of the same type, their performance bottlenecks can shift depending on the model, deployment environment, and specific task at hand.
Take software development as an example: code compilation and testing often require sustained CPU utilization, making the execution performance of individual agents a key consideration. By contrast, in scenarios such as enterprise customer support or personal assistants, individual interactions are typically brief but may need to handle a large volume of requests concurrently, placing greater demands on the CPU's concurrency capabilities.
These differences have also led CPU performance evaluation to focus on the execution efficiency of complete agent tasks and on performance across various workloads.
This time, Arm has proposed integrating component-level testing with end-to-end agent benchmarking.
Specifically, component-level testing covers stages such as the web server, database, model services, and sandbox execution, while end-to-end testing leverages agent-based benchmarks like SWE-bench and Terminal-Bench to evaluate performance across the entire workflow.
In addition to traditional metrics such as single-core performance and multi-core throughput, Arm also places emphasis on the number of concurrent agents, task execution efficiency, and per-rack performance.
Ravi Malhotra emphasized that performance evaluation must strike a balance between throughput and SLA (Service Level Agreement) requirements, and should not simply aim to run more agents concurrently.
Given the varying requirements across different scenarios, Arm does not recommend a one-size-fits-all approach to assigning weights to various agent use cases; instead, it advocates that enterprises select appropriate evaluation metrics based on their specific applications.
As agents increasingly integrate into enterprises' real-world business processes, these performance metrics have become key considerations in selecting and upgrading CPU infrastructure.
Ravi Malhotra believes that scenarios such as software development, customer support, supply chain management, and personal assistance will serve as key drivers for enterprises to upgrade their CPU infrastructure.
"For enterprises, in scenarios such as customer support, supply chain management, and personal assistants, the focus is on per‑rack performance, and the scalability and energy‑efficiency advantages of Arm's AGI CPU are particularly pronounced in these use cases," said Ravi Malhotra.
Ravi Malhotra further noted that whether it involves upgrading existing infrastructure for agent applications or designing and developing entirely new infrastructure, these scenarios will generate clear upgrade demands and are expected to serve as the primary drivers behind the growth and widespread adoption of Arm's AGI CPUs.
For enterprises, the value derived from improved CPU performance ultimately hinges on its impact on operational efficiency and cost.
The workloads of different agents vary significantly, and whether the performance gains observed in benchmark tests can indeed translate into improved task‑execution efficiency and reduced deployment costs remains to be confirmed through further real‑world applications.