OpenAI spills technical details about how its AI coding agent works

OpenAI finally pulled back the curtain on the internal workings of its AI coding agent, Codex CLI, offering an unprecedented look at the "agentic loop" that drives its code generation and problem-solving capabilities. This isn't just academic; it’s a critical blueprint for anyone building with, or relying on, AI agents.

The Agent Loop: It’s All About Cycles

Every AI agent, including OpenAI’s Codex, operates on a core, repeating cycle: the agent loop. It’s simple: input comes in, the model processes it, decides on an action (code, tool call), executes it, and then feeds the results back into the loop. This isn't a one-and-done deal. It’s an iterative process. You give it a task; it tries, runs a test, sees the failure, adjusts, tries again. This continuous feedback loop is how these agents learn and refine their output. Without it, they're just glorified autocomplete.

Bolin’s post confirms what we suspected: this loop is the engine. It takes your vague request, transforms it into a structured prompt, sends it to the model, and then interprets the model's response. If that response is a tool call – say, running a shell command or reading a file – the agent executes it. The output of that tool call then gets appended to the original prompt, and the whole cycle repeats. This continues until the model decides it’s done and provides a final answer. This isn't magic; it's a meticulously engineered state machine, constantly refining its understanding and action based on real-time feedback. You’re not just asking for code; you’re starting a conversation where the agent actively explores and reacts to the environment you've provided.

Prompt Construction: The Unseen Architecture

You don't just throw text at an AI agent and hope for the best; the way a prompt is constructed dictates its effectiveness. OpenAI's Codex doesn't rely on a flat text input; it builds its prompts from distinct components, each with an assigned role that tells the model how to interpret the information. We're talking system, developer, user, and assistant roles. This isn't just formatting; it's a hierarchy of intent.

The initial prompt sent to OpenAI's Responses API—the inference engine—is a composite. It starts with base instructions, either from a config file or bundled with the Codex CLI. Then come the tools definitions: what functions can the model call? Shell commands, planning utilities, web search, even custom tools via Model Context Protocol (MCP) servers. Finally, the input field bundles sandbox permissions, optional developer instructions, environmental context like the current working directory, and your actual query. This structured approach is crucial. It ensures the model understands not just what you want, but how it should interact with its environment to achieve it. A poorly constructed prompt is a wasted token, and worse, a wasted cycle. Give it garbage, get garbage. Give it structure, get a chance.

The Cost of Context: Quadratic Growth and Statelessness

Conversations with an AI agent aren't cheap, especially when every turn means a longer prompt. Bolin’s insight reveals a significant challenge: quadratic prompt growth. Each new interaction in the agentic loop means adding the complete history of previous messages and tool calls to the prompt. This isn't just inefficient; it's a performance killer. The context window, that finite space for the AI model to process text, fills up fast.

OpenAI made a deliberate choice here: their API requests are fully stateless. They don't use a previous_response_id parameter to reference stored conversation states. Every request sends the entire conversation history. Why? It simplifies things for API providers and supports "Zero Data Retention" policies, where OpenAI doesn't store user data. Good for privacy, bad for your token count and latency. This means you're paying for every word, every tool output, every byte of history, every single time. It's a trade-off. Convenience and privacy against raw computational cost. Understand that trade-off, because it’s hitting your bill.

Mitigating the Mess: Caching and Compaction

So, how do you fight quadratic prompt growth when you're sending the whole conversation every time? OpenAI's answer is twofold: prompt caching and conversation compaction. These aren't perfect solutions, but they're essential for keeping the Codex CLI usable.

Prompt caching attempts to mitigate the issue by storing previous prompt segments. If the current prompt has an exact prefix match with a cached one, the model can potentially reuse computation. The catch? "Exact prefix matches." Change a tool definition mid-conversation, switch models, or modify sandbox configurations, and you've blown the cache. That's a cache miss, and you're back to square one, re-processing everything. It means your interactions need to be stable, predictable.

When the token count inevitably exceeds the model’s context window, conversation compaction kicks in. Earlier versions of Codex required a manual slash command to compact. Now, it's automated via a specialized API endpoint. This endpoint compresses context, summarizing what happened while preserving the model's "understanding" through an encrypted item. It’s a lossy compression, sure, but it’s better than hitting the context wall and having the agent forget everything it just did. This is a critical piece of the puzzle for long-running, complex tasks. You can't just keep feeding it more; you have to summarize.

Beyond the Model: The Harness Problem

OpenAI's detailed breakdown focuses on the model and its immediate interaction loop. But the real-world performance of an AI coding agent isn't just about the model; it's about the "harness" it operates within. What's a harness? It's everything surrounding the model: the orchestration layer, the tooling, the environment setup, the feedback mechanisms, and how humans interact with it. The model is the engine; the harness is the car, the road, and the driver.

The "Coding Agent Index 2026" benchmark made it clear: the same model, wrapped in two different harnesses, can generate a bill that differs by 32x for a single task. Think about that. You can optimize the model all you want, but if your harness is inefficient – if it's making unnecessary tool calls, re-fetching files, or poorly managing context – you're burning tokens and time. This is where the rubber meets the road. Model Context Protocol (MCP) servers are part of this, defining how the agent interacts with external tools. But the overall design of the environment, the prompts you feed it, and the feedback you give it are paramount. A brilliant model in a terrible harness is still terrible.

Your Role in the Loop: Precision and Context

AI coding agents are powerful, but they aren't autonomous. They're brittle beyond their training data and demand human supervision for anything production-ready. This isn't a "set it and forget it" tool. It's an intelligent assistant that needs clear, precise instructions and continuous feedback. You're still the driver.

This is where your input quality becomes paramount. Ambiguous instructions lead to wasted cycles, token bloat, and frustrating debugging sessions. The agent can't read your mind. It needs specific UI elements identified, exact file paths, precise error messages. "Fix the button" isn't enough. "Fix the PrimaryButton component in src/components/ui/PrimaryButton.tsx to handle onClick events more robustly, specifically addressing the TypeError: handler is not a function error when the isLoading prop is true" – that's actionable.

To bridge this gap, tools like markagent become indispensable. Instead of typing out vague descriptions, you mark the exact UI element on a webpage, add your note, capture a screenshot, and it extracts all the necessary context: component names, file paths, DOM context, stable CSS selectors, page URL. This isn't just an annotation; it's a structured prompt, ready for your AI agent. It cuts through the ambiguity, providing the agent with pixel-precise, code-level context it needs to act effectively within its agentic loop. You're not just giving it a task; you're giving it a map and a flashlight.

The Future is in the Details

OpenAI spills technical details about how its AI coding agent works because they know the future of AI development isn't just about bigger models. It's about better engineering, smarter orchestration, and more efficient interaction design. Understanding the nuances of prompt construction, the challenges of context management, and the critical role of the surrounding harness transforms you from a casual user into an informed developer. These aren't just features; they're design principles. Ignore them at your peril.

The agent loop is here. Master its mechanics, or get left behind.