{"id":7817,"date":"2025-10-21T05:03:59","date_gmt":"2025-10-21T05:03:59","guid":{"rendered":"https:\/\/serisec.com\/index.php\/2025\/10\/21\/agentic-ais-ooda-loop-problem-html\/"},"modified":"2025-10-21T05:03:59","modified_gmt":"2025-10-21T05:03:59","slug":"agentic-ais-ooda-loop-problem-html","status":"publish","type":"post","link":"https:\/\/serisec.com\/index.php\/2025\/10\/21\/agentic-ais-ooda-loop-problem-html\/","title":{"rendered":"Agentic AI\u2019s OODA Loop Problem"},"content":{"rendered":"\n
Agentic AI\u2019s OODA Loop Problem<\/div>\n

\t

\n
<\/BR>
\n
\n \t

\n
<\/BR><\/p>\n

\n

The OODA loop\u2014for observe, orient, decide, act\u2014is a framework to understand decision-making in adversarial situations. We apply the same framework to artificial intelligence agents, who have to make their decisions with untrustworthy observations and orientation. To solve this problem, we need new systems of input, processing, and output integrity.<\/b><\/p>\n

Many decades ago, U.S. Air Force Colonel John Boyd introduced the concept of the \u201cOODA loop,\u201d for Observe, Orient, Decide, and Act. These are the four steps of real-time continuous decision-making. Boyd developed it for fighter pilots, but it\u2019s long been applied in artificial intelligence (AI) and robotics. An AI agent, like a pilot, executes the loop over and over, accomplishing its goals iteratively within an ever-changing environment. This is Anthropic\u2019s definition: \u201cAgents are models using tools in a loop.\u201d1<\/a><\/sup><\/p>\n

OODA Loops for Agentic AI<\/h3>\n

Traditional OODA analysis assumes trusted inputs and outputs, in the same way that classical AI assumed trusted sensors, controlled environments, and physical boundaries. This no longer holds true. AI agents don\u2019t just execute OODA loops; they embed untrusted actors within them. Web-enabled large language models (LLMs) can query adversary-controlled sources mid-loop. Systems that allow AI to use large corpora of content, such as retrieval-augmented generation (https:\/\/en.wikipedia.org\/wiki\/Retrieval-augmented_generation<\/a>), can ingest poisoned documents. Tool-calling application programming interfaces can execute untrusted code. Modern AI sensors can encompass the entire Internet; their environments are inherently adversarial. That means that fixing AI hallucination is insufficient because even if the AI accurately interprets its inputs and produces corresponding output, it can be fully corrupt.<\/p>\n

In 2022, Simon Willison identified a new class of attacks against AI systems: \u201cprompt injection.\u201d2<\/a><\/sup> Prompt injection is possible because an AI mixes untrusted inputs with trusted instructions and then confuses one for the other. Willison\u2019s insight was that this isn\u2019t just a filtering problem; it\u2019s architectural. There is no privilege separation, and there is no separation between the data and control paths. The very mechanism that makes modern AI powerful\u2014treating all inputs uniformly\u2014is what makes it vulnerable. The security challenges we face today are structural consequences of using AI for everything.<\/p>\n

    \n
  1. Insecurities can have far-reaching effects. A single poisoned piece of training data can affect millions of downstream applications. In this environment, security debt accrues like technical debt.<\/li>\n
  2. AI security has a temporal asymmetry. The temporal disconnect between training and deployment creates unauditable vulnerabilities. Attackers can poison a model\u2019s training data and then deploy an exploit years later. Integrity violations are frozen in the model. Models aren\u2019t aware of previous compromises since each inference starts fresh and is equally vulnerable.<\/li>\n
  3. AI increasingly maintains state\u2014in the form of chat history and key-value caches. These states accumulate compromises. Every iteration is potentially malicious, and cache poisoning persists across interactions.<\/li>\n
  4. Agents compound the risks. Pretrained OODA loops running in one or a dozen AI agents inherit all of these upstream compromises. Model Context Protocol (MCP) and similar systems that allow AI to use tools create their own vulnerabilities that interact with each other. Each tool has its own OODA loop, which nests, interleaves, and races. Tool descriptions become injection vectors. Models can\u2019t verify tool semantics, only syntax. \u201cSubmit SQL query\u201d might mean \u201cexfiltrate database\u201d because an agent can be corrupted in prompts, training data, or tool definitions to do what the attacker wants. The abstraction layer itself can be adversarial.<\/li>\n<\/ol>\n

    For example, an attacker might want AI agents to leak all the secret keys that the AI knows to the attacker, who might have a collector running in bulletproof hosting in a poorly regulated jurisdiction. They could plant coded instructions in easily scraped web content, waiting for the next AI training set to include it. Once that happens, they can activate the behavior through the front door: tricking AI agents (think a lowly chatbot or an analytics engine or a coding bot or anything in between) that are increasingly taking their own actions, in an OODA loop, using untrustworthy input from a third-party user. This compromise persists in the conversation history and cached responses, spreading to multiple future interactions and even to other AI agents. All this requires us to reconsider risks to the agentic AI OODA loop, from top to bottom.<\/p>\n