In an opinion column published in SiliconANGLE, Jason Bloomberg, founder and managing director of the advisory firm Intellyx, examines the use of artificial intelligence agents (AI agents) and the risks associated with their nondeterministic behavior in enterprises. According to Bloomberg, the year 2026 is characterized by the widespread use of AI agents for various purposes, including writing code, communicating with customers, identifying and resolving issues in information technology (IT) environments, and executing complex tasks in enterprise environments. Alongside the success stories, reports have featured incidents where agents exceed their defined operational boundaries, break out of sandboxes, and compromise systems belonging to other companies and government agencies. In light of this, the question arises whether enterprises should rely on these agents to achieve business success.
Bloomberg notes that any decision to deploy agents depends on understanding their nondeterministic behavior. Most agents today rely on large language models (LLMs). These models take human language inputs and produce human language outputs by predicting the next word in a sequence. Because of how their training process works, repeating a given input may produce significantly different results; therefore, agents based on LLMs are inherently nondeterministic. This nondeterminism gives agents the ability to act (agency), defined as the ability to make decisions based on available information in order to take different actions to accomplish the goals set for them. Software lacking this capability is not an agent by definition, despite the phenomenon of labeling deterministic software as agents ("agent-washing").
The article presents additional examples of nondeterministic software that lack the agency provided by LLMs, specifically software based on stochastic policy-based nondeterminism. Common examples include reinforcement learning and Monte Carlo simulations, where decisions are made based on randomization. Aside from these limited examples, the vast majority of software written in the past was deterministic: for a given input, the software always performs the exact same action. Predictable behavior was the motivation for developing software in the first place for tasks such as processing bank transactions, running payroll, and assigning seats on airplanes.
According to Bloomberg, deterministic software is the best choice when there is a single right answer and one seeks to find it, such as in running payroll systems designed to ensure accurate payment for every employee. In contrast, nondeterministic reasoning based on LLMs makes the most sense when seeking the best result across a range of different or changing inputs, even when there is no single right answer. Examples include weather-dependent planning, incident mitigation in complex IT environments, and business forecasts based on multiple inputs. In these situations, there is always a possibility that the agent will provide a poor answer or take an undesirable action.
The article details a series of situations where using AI agents is not recommended:
- When deterministic software meets the needs – agents may misbehave on occasion, and running them can be expensive.
- When every decision or action must be completely accurate – for example, in payroll calculations.
- When agent misbehavior could cause severe harm – such as harming a hospital patient's life, a plane crash, or an oil rig explosion.
- When the cost of running the agents is too high relative to the business benefit – agents may make multiple calls to LLMs, retry actions repeatedly, and invoke expensive tools to complete their goals.
- When human qualities are essential to achieve the goals – LLMs simulate human characteristics such as empathy, creativity, insight, and understanding, but this is merely mimicry. Anthropomorphizing agent behavior can lead to risks such as overdependence on technology, increased suicide risk, and harmful behavior.
To determine whether deploying agents is worthwhile, Bloomberg suggests using the concept of an error budget from site reliability engineering (SRE). The error budget represents the acceptable cost of agent misbehavior as a specific percentage. For example, a 5% error budget reflects a willingness for the agent to make mistakes in 5% of cases, given the cost required to reduce that rate. If the calculated error budget is zero, it means the agent should not be deployed at all, since there is always a chance of misbehavior.
Under the decision-theory model presented in the article, an agent should not be deployed when the product of the error budget (probability of failure) and the total cost of failure is greater than the product of the probability of success and the quantified business benefit of that outcome. In other words, the expected business benefit from proper agent behavior must exceed the total cost of failure. Because the calculation relies on estimates that may be incorrect, the expected business benefit should exceed the cost of failure by a significant margin, depending on the organization's risk appetite.
Bloomberg addresses the market trend of developing governance and security technologies aimed at restricting the unpredictable nature of agents to mitigate risks. In his view, there is a flaw in this reasoning, as attempting to limit agents' unpredictability harms their agency, which is the source of their power, and can thereby make them less useful. Bloomberg's forecast is that once the hype subsides and the technology matures, it will turn out that agents are best used only sparingly, as deterministic software has met needs for decades.