Prompts, Loops, and Graphs

Every AI system is one of three shapes, set by where the control flow lives. Read the shape, and the climb.

// Field Guide 03

Prompts, loops, and graphs

Every AI system is one of three shapes. The shape is set by one question: where does the control flow live?

Start

Every vendor says agent. The code says prompt, loop, or graph. This guide reads the code.

Strip the branding off any AI product shipped this decade and you find one of three structures. A prompt: one call to a model, every decision made before runtime. A loop: the model calls tools, reads the results, and picks its own next step until it judges the work done. A graph: multiple calls wired together by your code, which decides what happens between them.

The axis running through this page is control flow. The prompt has none. The loop hands it to the model. The graph takes it back. Anthropic draws the same line in its engineering guidance: workflows run on predefined code paths, agents direct their own processes. Everything below is that distinction, with receipts.

One call. Context in, answer out. Every decision the system will ever make was made by you, before runtime, in the writing of the prompt and the assembly of the context. At runtime, nothing decides anything. The model completes.

fig. 1 · one call, zero runtime decisions. all the engineering is on the left of the box.

system = one model call + the context you assembled

system promptsystem promptStanding instructions the model reads before the user's input. The part of the call you control completely. context windowcontext windowThe model's entire working memory for a call. If it is not in the window, it does not exist for the model. retrieval (RAG)retrieval (RAG)Fetching relevant documents into the context before the call, so the model answers from your material instead of its training. few-shotfew-shot examplesWorked examples placed in the prompt. You show the pattern instead of describing it. structured outputstructured outputForcing the answer into a schema your code can parse. The prompt's handshake with the rest of the software.

hover a term for its definition · tap to pin on touch

Spot it

One request, one response. No tool calls in the trace. The cost of a run is the cost of one call, and latency is the latency of one call. All the engineering effort lives in what gets stuffed into the context, which is why "prompt engineering" quietly became context engineering.

Works when

The task is one decision, everything it needs fits in the window, and a wrong answer is cheap to catch. Most production AI features live here, and should. Anthropic's builder guidance says it plainly: for many applications, optimizing a single call with retrieval and in-context examples is usually enough.

The climb

Watch your own behavior. The moment you are running the same prompt, reading the output, fixing something, and feeding it back in, you have become the loop. That is the tell that the task wants shape two. The machine version just removes you from the middle.

// The honest limit

A prompt cannot check its work against anything. One shot, no ground truth. It answers from the context you gave it, and if the context is thin or wrong, the answer is confidently wrong, and nothing in the system exists to notice. Every verification step you skip at build time ships to the user at runtime.

The model calls a tool, reads the result, and decides what to do next. Repeat until it judges the work done. The control flow belongs to the model. This is the structure the word "agent" actually names, once the pitch deck is closed.

fig. 2 · act, observe, repeat. the loop exits only when the model's reply contains no tool call.

while (the model's reply contains a tool call) { run tool; feed result back }

tool usetool useThe model emits a structured call; your code runs it and returns the result into the context. The loop's only interface to reality. ReActReActThe 2022 research pattern (Yao et al.) that interleaved reasoning steps with actions. The loop's academic ancestor. ground truthground truthA result from the real world the model can check itself against: a test run, a tool output, an error message. What makes a loop more than a chatty prompt. stop conditionstop conditionWhat ends the loop: the model stops calling tools, a turn cap fires, or a budget runs out. The most underdesigned line in most agents. harnessharnessEverything wrapped around the loop: context management, permissions, recovery, compaction. In mature agents this is most of the code. subagentsubagentA fresh loop with its own context window, spawned for a scoped task. The loop's answer to context pollution.

hover a term for its definition · tap to pin on touch

Spot it

Traces with many model turns per user request. Tool-call logs. Nondeterministic paths: the same input takes a different route on the second run. And the bill: agents use roughly four times the tokens of a chat interaction, by Anthropic's own measurement. If the invoice looks like a chat product, it is not a loop.

The canonical receipt is Claude Code. Anthropic's own description of its coding agent is one feedback loop: gather context, take action, verify work, repeat. And its builder guidance says the quiet part in print: agents are typically just LLMs using tools based on environmental feedback in a loop. The capability lives in the model and the tools, not in structure above them.

Works when

The path cannot be written down in advance, the steps vary per task, and there is ground truth the loop can check itself against as it works. Coding is the canonical habitat because the test suite grades every iteration for free. Open-ended research works for the same reason: search results are ground truth of a weaker kind.

The climb

Upward to the graph, and here the ladder inverts. You do not add a graph because the loop got smarter. You add one because you stopped trusting it: the process must be identical every run, auditable, or cheap. The next shape is not more capability. It is less autonomy.

// The honest limit

Tokens and trust. Agents burn about four times the tokens of chat, and errors compound across turns because each step's mistake becomes the next step's context. One bad tool result early can send the whole trajectory somewhere new, which also makes failures hard to reproduce.

And the quiet dependency: without ground truth, the loop is just a prompt that bills you several times over. A loop the system cannot grade is not autonomy. It is a liability with a meter running.

Multiple model calls wired together by your code. The nodes are prompts, or whole loops. The edges are decisions your code makes between them. The control flow comes back to you. This is what Anthropic's guidance calls a workflow: LLMs and tools orchestrated through predefined code paths.

Read the shape plainly. The graph is not the loop's upgrade. It is the loop with the trust removed, one edge at a time.

fig. 3 · the model fills the boxes. the edges, and everything they assume, are yours.

system = nodes (calls or loops) + edges (your code's decisions) + shared state

prompt chainingprompt chainingThe output of one call feeds the next, in a fixed sequence. The simplest graph: a straight line. routerrouterOne call classifies the input; your code sends it down the matching branch. Different inputs, different specialized paths. parallelizationparallelizationIndependent calls run at once and your code merges the results. Buys speed or multiple perspectives. orchestrator-workersorchestrator-workersA lead model breaks the task apart and farms pieces to workers. The subtasks vary; the structure that contains them does not. evaluator-optimizerevaluator-optimizerOne call generates, another grades, repeat on rails until the grade clears. A loop whose shape is fixed by code, which is what makes it a graph. statestateWhat the graph carries between nodes: the plan, intermediate results, the running record. In graph frameworks, state is the first-class object.

hover a term for its definition · tap to pin on touch

Spot it

Framework imports at the top of the file; LangGraph is the shape sold as a product. A flow diagram in the design doc. Deterministic latency and cost per run. And the cleanest tell: the same input takes the same path twice. Loops cannot promise that. Graphs cannot help it.

Works when

The process is the product. Regulated pipelines, high-volume triage, document assembly lines, anywhere "it does the same thing every time" is the feature being bought, and anywhere a wrong answer costs more than model autonomy is worth. If you can draw the steps as a flowchart that holds for every case, the flowchart is the design, and the graph is just the flowchart running.

The climb

Points down the page, and that is the finding. Teams delete edges as models improve, handing control flow back to the loop. Anthropic's standing advice is to find the simplest structure that passes evaluation and add complexity only when it demonstrably earns its keep. The strongest builders climb this chain in reverse.

// The honest limit

Every edge is an assumption with a freeze date. You wrote the graph against this year's model. Next year's model makes half the edges scaffolding debt, and the graph has no way to know. The structure that made the system reliable is the same structure that caps it at your understanding of the problem on the day you drew it.

The multi-model version compounds the bill. Anthropic's multi-agent research system, an orchestrator whose workers are loops, beat single-agent Claude Opus 4 by 90.2 percent on an internal research eval and consumed roughly fifteen times the tokens of a chat. Their own stated boundary: domains where the agents must share the same context, or depend heavily on each other, are not a good fit today. The graph pays off exactly where the work splits clean, and punishes you everywhere it does not.

// What runs underneath

The parts every shape shares

Context

The window is the model's whole world in all three shapes. Prompts front-load it, loops rebuild it every turn, graphs pass it between nodes. Compaction, retrieval, and subagent isolation are all the same job: deciding what the model gets to know right now.

Tools

The loop and the graph act through the same interface: a structured call your code executes. MCP standardizes that layer so tools outlive any one shape. Mature teams spend more effort on tool design than on prompts.

Budgets and stop conditions

Every loop needs a meter and a kill switch: turn caps, token ceilings, spend limits. The multipliers compound when something misbehaves, and an orchestrator that over-spawns workers can turn a question into an invoice.

Evals and ground truth

You cannot ship a shape you cannot grade. Ground truth decides which shapes are even available: no way to verify progress means no loop, whatever the roadmap says.

// Find your shape

Which shape is your system?

Answer for the task in front of you, not the roadmap.

QUESTION 1 / 3

Can everything the task needs fit in one call, with one decision at the end?

QUESTION 2 / 3

Could you draw those steps as a flowchart that holds for every case?

QUESTION 3 / 3

Is there ground truth the system can check itself against as it works? Tests, tool results, a source of record.

Your shape

The Prompt

One well-built call, with the effort spent on context: retrieval, examples, structure. No loop, no framework. This is the majority answer in production, and choosing it is not settling. It is the simplest structure that passes.

Your shape

The Graph

If the flowchart holds for every case, the flowchart is the design. Wire the calls on predefined paths and keep the determinism: same input, same route, same bill. Revisit yearly, because every edge you draw carries a freeze date.

Your shape

The Loop

Unpredictable path plus verifiable progress is the loop's exact habitat. Spend the effort on tools, context, and the stop condition, and give it a meter: loops run about four times the tokens of chat before anything goes wrong.

Not yet a shape

Get ground truth first

Without anything to check against, the loop is a prompt with a meter running, and no shape saves you. Build the verifier before the system: a test, a validator, a source of record. Until it exists, run a graph with a human at the merge.

// Where this is heading

The shapes are sliding

Structure is migrating out of the code and into the model. The 2023 pattern was graphs everywhere: frameworks, seven-node pipelines, flowcharts as a product category, because the models could not be trusted with the flow. The current pattern is the loop eating the graph from the inside: fewer edges, better tools, and Anthropic shipping its flagship coding agent as one disciplined feedback loop with most of the engineering in the harness around it.

fig. 4 · every model release deletes edges somewhere.

Real systems stack the shapes rather than choosing one. A loop is full of prompts. A serious graph has loops for nodes: Anthropic's research system is an orchestrator graph whose workers are loops. Claude Code is a loop that can spawn fresh loops. The taxonomy is not three products. It is three answers to who decides next, and a system can answer differently at different layers.

The bet, flagged as this guide's opinion: every edge you draw today is a short position on next year's model. Draw them where the process is the product, where sameness and auditability are what the customer is buying. Everywhere else, build the loop, and spend the saved effort on the three things that survive every model release: tools, context, and the stop condition.

For where the shapes sit in the broader stack, stages four and five of Agentic AI, in five stages are the companion read.