Building an Enterprise Work Environment for AI Agents
A new report published by Intel presents insights and practical lessons from thousands of experiments conducted on agentic AI workloads in enterprise environments. According to the company's findings, the promise of AI agents in large enterprises goes significantly beyond the capabilities of enhanced chatbots. These are autonomous software programs that execute end-to-end business tasks by integrating people, business workflows, data, and various systems. To run these agents optimally, organizations require a platform built with appropriate CPU capacity, resilient data access, policy-aware tool use, observability, memory management, and the ability to predictably plan and scale. Following its thousands of experiments, Intel presents five practical lessons for enterprise leaders, explaining how to build the most suitable infrastructure to run these agents efficiently and cost-effectively.
Beyond Inference: Agents as Workflow Automation
Intel emphasizes that agentic AI is far more than large language model (LLM) inference. The enterprise value of these agents depends on the entire system: task orchestration, data access, tool execution, latency management, corporate governance, and scalable infrastructure. An agent is defined as an automated enterprise workflow process guided by fixed goals—it plans a multi-step task, invokes external tools, reads the results, and performs retries when a specific step fails. For this reason, AI agents do not merely represent an inference problem, but a much broader and more complex systems challenge. Most of the testing tools available today are limited and do not measure overall system performance, highlighting the need for a more comprehensive approach to measuring success.
Defining Success Metrics in the Enterprise
Most existing metrics for agentic AI focus on evaluating the specific large language model (LLM) being used. However, enterprise platform teams require much broader information: how long tasks take, how many agents the server fleet can support simultaneously, what end users experience at the end of the execution process, and how costs change when multiple agents operate concurrently. To answer these questions, Intel proposes looking at six key metrics for enterprise performance evaluation: task success rate, cost per task, time per task, task throughput, agent density (measured as agents per virtual CPU, or agents per vCPU), and latency. These metrics allow enterprise AI operators to determine whether the system is performing as expected, how many agents it can sustainably support, and how to scale it to accommodate additional agents.
Extending the Terminal-Bench Platform for Accurate Measurement
To deeply understand the performance of agentic AI workloads, Intel extended the open-source benchmarking tool Terminal-Bench, which is used to evaluate AI agents, adding profiling, telemetry, and run-replay capabilities. This extension allowed the company to understand exactly where agents spend their time beyond the LLM inference process itself. To neutralize the impact of internal variations in LLM responses and isolate the performance of the agents themselves, the benchmark extension used deterministic recording and replaying of LLM responses. The language model responses were recorded once and replayed identically across different runs, which reduced run-to-run variance and created a much more reliable and accurate comparison baseline. The task mix used in Terminal-Bench was intentionally broad to simulate real enterprise environments, including code compilation, testing, database operations, Boolean logic, interpretation, ray tracing, data compression, linear algebra, video transcoding, and machine learning training.
Capacity Planning by Agent Density, Not Agent Count
The first dimension of deploying AI agents is capacity planning. Intel recommends planning computing resources based on agent density (measured as the number of agents per vCPU) rather than the total number of agents. Agent density serves as the leading signal for identifying system saturation. For example, 10 agents running on a system with 8 vCPUs and 20 agents running on a system with 16 vCPUs will behave very similarly because their density is identical. This approach provides system architects with a portable and simple way to compare capacity across different instance sizes and processor generations. The correct density depends on the organization's business objective. Interactive copilots and user-facing assistants should operate at a lower density because response time for end users is critical. Conversely, batch workloads, such as IT workflows, can often run at a higher density. This distinction allows teams to tune agent fleets according to service-level objectives (SLOs) and total cost of ownership (TCO).
Performance Monitoring and Bursty Work Patterns
The second dimension concerns the need for new observability and monitoring methods. Average CPU utilization is a weak and insufficient signal for monitoring the performance of AI agents. This is because agents tend to display a "bursty" work pattern: they alternate between waiting relatively long periods for a model response and executing short bursts of demanding, compute-intensive work. Due to this pattern, average CPU utilization can appear perfectly normal and acceptable even when those work bursts create long queues and slow down the user experience. Therefore, the leading and more accurate metric for monitoring is the 95th percentile task latency (P95 task latency). This metric shows when workflows begin waiting in a queue before average task duration shows significant degradation. A practical operating model is to set alerts first on P95 latency and then validate the issue by examining the total task duration over time.
Scaling Strategy: Defaulting to Scale-Out
The third dimension is favoring a scale-out (horizontal scaling) approach by default when expanding the system, rather than scale-up (vertical scaling). Scaling out means adding more systems and servers to increase overall agent capacity. In contrast, scaling up means adding CPU cores or memory to a single system to handle heavier computing bursts from specific agents. Intel's testing data showed that scaling out is generally the preferred default. This aligns with the fact that agents are typically semi-independent and present relatively modest computational bursts per individual agent. Scaling out improves overall performance, supports high availability, generally reduces costs, and makes it easier to maintain the target agents-per-vCPU ratio as the platform grows. However, vertical scaling should be performed in specific cases where agents require heavier parallel computing, when a shared state limits partitioning capabilities, when memory locality is crucial, or when licensing constraints exist.
Business Implications and Practical Enterprise Implementation
Where will AI agents create the first business value for an organization? According to the experimental findings, organizations achieving production-grade results are those wrapping an automation layer around workflows that already have codified rules and measurable service levels. Examples include code creation, regression test farms, ticket triaging, market analysis, and security reviews. The ideal enterprise persona to lead AI agent projects is not the experimental user chasing technological novelty for its own sake, but the accountable manager who must improve cycle times and productivity, protect service quality, enforce corporate policies, and scale adoption within budget and cost constraints. The real value of AI agents comes from helping organizations complete actual work across different teams, systems, data, and processes. For enterprises, stakeholders are not merely looking for better model performance, but for the creation of a reliable environment where AI agents can support business workflows, improve productivity, operate under corporate governance requirements, and scale as adoption expands. Practical success in deploying agents depends on building the right foundation to deliver consistent outcomes, manage costs, maintain control, and move confidently from pilot phases to full production.