"Agent" is the most overloaded word in AI right now. In the last three weeks alone, OpenAI shipped Operator, an agent that uses its own browser, and deep research, an agent that spends 5 to 30 minutes browsing and writing a report. Every framework, startup and vendor deck promises agents. Meanwhile, most production LLM features that work well are not agents at all.

The clearest vocabulary for this comes from Anthropic's December 2024 post Building effective agents, written by Erik Schluntz and Barry Zhang from what they saw across dozens of customer teams. Their main finding is unglamorous: the most successful implementations "weren't using complex frameworks or specialized libraries" but "simple, composable patterns".

This post is a cheatsheet of those patterns: what each one is, when to use it, what it costs, how to write it in a few lines of plain Python, and how it fails. No framework required.

1. Workflows vs. agents

Anthropic calls everything in this space an agentic system, and splits it in two:

Workflow Agent
Definition LLMs and tools orchestrated through predefined code paths LLMs dynamically direct their own process and tool usage
Who decides the next step Your code The model
Number of steps Known in advance (or bounded by your code) Unknown; depends on what the model finds
Predictability High: same input, same path Lower: paths vary run to run
Debugging Like normal software Read transcripts
Best for Well-defined tasks that decompose cleanly Open-ended tasks where you cannot hard-code the path
Cost and latency Bounded Variable, often high

The rule that comes before any pattern: find the simplest solution that works, and add complexity only when it demonstrably improves outcomes. For many applications, a single well-prompted LLM call with retrieval and a few examples is enough. Agentic systems trade latency and cost for better task performance; that trade has to be worth it.

2. The building block: the augmented LLM

Every pattern below is built from one unit: an LLM with retrieval, tools and memory. The model writes its own search queries, chooses tools and decides what to remember.

Two things matter in practice: tailor the augmentations to your use case (fewer, better tools), and give the model a clear, well-documented interface to each one. One way to plug in tools and data sources is the Model Context Protocol (MCP), which Anthropic open-sourced on November 25, 2024: a standard where data and tools are exposed by MCP servers (pre-built ones were released for Google Drive, Slack, GitHub, Git, Postgres and Puppeteer) and consumed by MCP clients.

For the sketches below, everything goes through one function. Wrap your provider's SDK in it and the patterns stay provider-neutral:

import json
from collections import Counter
from concurrent.futures import ThreadPoolExecutor


def llm(prompt: str, system: str = "", model: str = "default") -> str:
    """Wrap your provider's SDK here. Every pattern below only uses this."""
    raise NotImplementedError

3. The patterns at a glance

Pattern Shape Use when Main cost
Prompt chaining Fixed sequence A -> B -> C, with checks between steps The task splits cleanly into fixed subtasks Latency (sequential calls)
Routing Classify, then send to a specialized prompt or model Distinct input categories that need different handling One extra call; misroutes
Parallelization: sectioning Independent subtasks run at once, results combined Subtasks are independent, or each aspect needs focused attention More calls (but faster wall-clock)
Parallelization: voting Same task run several times or with several prompts; aggregate You need higher confidence on a judgment N times the cost
Orchestrator-workers An LLM plans subtasks at runtime, workers execute, LLM synthesizes You cannot predict the subtasks in advance Planning errors, more tokens
Evaluator-optimizer Generate, critique, revise in a loop Clear evaluation criteria and refinement measurably helps Several rounds per output
Autonomous agent LLM uses tools in a loop with environment feedback until done Open-ended problems, unknown number of steps Highest cost, compounding errors

4. Workflows

Prompt chaining

Decompose the task into a fixed sequence of steps, where each LLM call processes the previous output. Add programmatic gates between steps to catch problems early. The goal is to trade latency for accuracy by giving each call an easier job.

Examples: write an outline, check it against criteria, then write the document; generate marketing copy, then translate it.

def chain(topic: str) -> str:
    outline = llm(f"Write a 5-point outline for a blog post about: {topic}")
    if outline.count("\n") < 4:                      # programmatic gate
        outline = llm(f"Expand this into exactly 5 points:\n{outline}")
    draft = llm(f"Write the post following this outline:\n{outline}")
    return llm(f"Translate to Spanish, keep the markdown:\n{draft}")

Gates are plain code: length checks, schema validation, regex, a unit test. They are the cheapest reliability you can buy.

Routing

Classify the input, then send it to a specialized prompt, toolset or model. This gives separation of concerns: optimizing one prompt for refund requests no longer hurts technical questions. It also enables cost routing: easy questions to a small, fast model, hard ones to a larger model.

ROUTES = {
    "refund": ("You handle refunds. Policy: ...", "small"),
    "technical": ("You are a support engineer. Docs: ...", "large"),
    "general": ("You answer general questions briefly.", "small"),
}


def route(query: str) -> str:
    label = llm(f"Classify into one of {list(ROUTES)}. Reply with the label only.\n\n{query}")
    system, model = ROUTES.get(label.strip().lower(), ROUTES["general"])  # safe fallback
    return llm(query, system=system, model=model)

Routing only works if classification is accurate. Measure the router on its own, and consider a traditional classifier if the categories are stable.

Parallelization: sectioning and voting

Two variants:

  • Sectioning: split into independent subtasks and run them at the same time. Example: one call answers the user while another screens the request for policy violations. Anthropic notes this tends to work better than asking one call to do both.
  • Voting: run the same task several times, or with different prompts, and aggregate. Example: several reviewers each look for a different class of vulnerability; flag the code if any of them finds a problem.
def answer_with_guardrail(query: str) -> str:
    with ThreadPoolExecutor() as pool:
        answer = pool.submit(llm, query)
        check = pool.submit(llm, f"Is this request allowed by policy X? Reply PASS or FAIL.\n\n{query}")
        if check.result().strip() != "PASS":
            return "Sorry, I can't help with that."
        return answer.result()


def is_vulnerable(code: str) -> bool:
    prompts = [
        f"Review only for injection flaws. Reply VULNERABLE or SAFE.\n\n{code}",
        f"Review only for access-control flaws. Reply VULNERABLE or SAFE.\n\n{code}",
        f"Review only for unsafe deserialization. Reply VULNERABLE or SAFE.\n\n{code}",
    ]
    with ThreadPoolExecutor() as pool:
        votes = Counter(v.strip() for v in pool.map(llm, prompts))
    return votes["VULNERABLE"] >= 1  # the threshold sets the false positive / negative balance

The vote threshold is a product decision: "any reviewer flags it" maximizes recall; "majority" reduces false alarms.

Orchestrator-workers

A central LLM breaks the task down at runtime, delegates subtasks to workers, and synthesizes the results. It looks like sectioning, but the subtasks are not predefined: the orchestrator decides them from the input. Typical uses: coding changes across an unknown number of files, research across several sources.

def orchestrate(task: str, max_subtasks: int = 8) -> str:
    plan = json.loads(llm(
        f"Task: {task}\nReturn a JSON list of subtasks, each "
        f'{{"file": ..., "change": ...}}. At most {max_subtasks}.'
    ))[:max_subtasks]                                 # validate and cap the plan
    with ThreadPoolExecutor() as pool:
        results = list(pool.map(lambda s: llm(f"Make this change to {s['file']}: {s['change']}"), plan))
    return llm(f"Task: {task}\nCombine these results and flag conflicts:\n" + "\n---\n".join(results))

In production, validate the plan against a schema, cap its size, and log it. A bad plan is the most common failure, and it is invisible if you only log the final output.

Evaluator-optimizer

One call generates, another evaluates against explicit criteria and gives feedback, in a loop. Two signs it fits: a human's feedback would demonstrably improve the output, and an LLM can give that kind of feedback. Examples: literary translation, complex search where the evaluator decides whether more searching is needed.

def refine(task: str, criteria: str, max_rounds: int = 3) -> str:
    draft = llm(task)
    for _ in range(max_rounds):
        review = llm(f"Criteria:\n{criteria}\n\nDraft:\n{draft}\n\n"
                     "If all criteria are met reply PASS. Otherwise list concrete fixes.")
        if review.strip().startswith("PASS"):
            break
        draft = llm(f"{task}\n\nPrevious draft:\n{draft}\n\nApply this feedback:\n{review}")
    return draft

Always cap the rounds. Evaluators can be endlessly picky, and generators can oscillate between two versions.

5. Agents

An agent is, mechanically, simple: an LLM using tools in a loop, based on feedback from the environment. It starts from a user command or conversation, plans, acts, observes the result ("ground truth" from tool outputs or code execution), and repeats until done, blocked or out of budget. It can pause for human input at checkpoints.

Use agents for open-ended problems where you cannot predict the number of steps or hard-code a path, and where you have enough trust in the model's decisions. Anthropic's own examples are a coding agent for SWE-bench tasks and its computer use reference implementation. The recent public examples fit the same definition:

  • OpenAI Operator (January 23, 2025, research preview for Pro users in the US): uses its own browser by looking at screenshots and typing, clicking and scrolling. It asks the user to take over for logins and payment details, and asks for confirmation before significant actions like submitting an order.
  • OpenAI deep research (February 2, 2025, Pro users first): powered by a version of the upcoming o3 model, it searches, reads and analyzes hundreds of sources and writes a cited report in 5 to 30 minutes. OpenAI lists its limitations openly: it can still hallucinate facts, struggles to tell authoritative information from rumors, and is weak at conveying uncertainty.

A framework-free agent loop. The model returns either a tool call or a final answer, in whatever format your provider uses:

TOOLS = {
    "read_file": lambda path: open(path).read()[:20_000],   # cap tool output size
    "run_tests": lambda path: run_pytest(path),
}
NEEDS_APPROVAL = {"write_file", "send_email", "deploy"}


def agent(task: str, max_steps: int = 20) -> str:
    messages = [{"role": "user", "content": task}]
    for _ in range(max_steps):                                 # hard stop condition
        reply = call_model(messages, tools=list(TOOLS))        # {"tool", "args"} or {"answer"}
        if "answer" in reply:
            return reply["answer"]
        name, args = reply["tool"], reply["args"]
        if name in NEEDS_APPROVAL and input(f"Allow {name}({args})? [y/N] ") != "y":
            result = "Denied by the user."
        elif name not in TOOLS:
            result = f"Unknown tool {name}. Available: {list(TOOLS)}"
        else:
            try:
                result = str(TOOLS[name](**args))
            except Exception as e:                             # errors are feedback, not crashes
                result = f"Error: {e}"
        messages += [{"role": "assistant", "content": json.dumps(reply)},
                     {"role": "user", "content": f"Tool result ({name}):\n{result}"}]
    return "Stopped: step limit reached. Transcript saved for review."

Everything that makes this safe is ordinary code around the loop: a step limit, a cost budget, approval gates on side effects, errors returned to the model as text, and logs of every step.

6. Tool design: the agent-computer interface

Anthropic's advice is to put as much effort into the agent-computer interface (ACI) as teams put into human-computer interfaces. When they built their SWE-bench agent, they spent more time optimizing the tools than the overall prompt.

Principle In practice
Write tools like docs for a junior developer Clear name, description, parameter meanings, example usage, edge cases, and how the tool differs from similar tools
Put yourself in the model's shoes If you would need to think hard to use the tool from its description, so will the model
Choose formats the model writes easily Prefer formats common in natural text. Code inside JSON needs escaping; diffs need line counts in headers. Rewriting a whole file or using markdown is often easier
Leave room to think Do not force a format that makes the model commit before it has reasoned
Poka-yoke the tools Change arguments so mistakes are hard. Anthropic's agent kept getting relative paths wrong after changing directory; requiring absolute paths fixed it
Test the tools with the model Run many example inputs, read the calls the model makes, and iterate on names and descriptions
Keep outputs small and informative Truncate, paginate, and return actionable error messages ("file not found; did you mean src/app.py?")
Few, distinct tools Overlapping tools cause wrong choices. Merge or sharpen them

7. Cost, latency and reliability trade-offs

Single call Workflow Agent
LLM calls per task 1 Fixed (2-10) Variable (often 10-100+)
Latency Seconds Seconds to a minute (less if parallel) Minutes (deep research: 5 to 30)
Cost per task Lowest Predictable multiple Unpredictable; needs budgets
Reliability Easiest to evaluate Each step testable Errors compound across steps
Debugging Read the output Inspect each step Read whole transcripts
Flexibility Low Medium High

Compounding is the key number to internalize. If each step of an agent succeeds 95% of the time and a task takes 20 dependent steps, the chance that all of them succeed is 0.95^20, about 36%. Agents survive this only because they get feedback from the environment and can correct course. That is why the best agent domains have cheap, objective feedback: tests for code, page state for browsers, a resolved ticket for support. Anthropic highlights exactly these traits for its two most promising applications, customer support and coding agents.

8. Failure modes and fixes

Failure Typical symptom Fix
Over-engineering An agent where a chain or a single call would do Start with one call; add steps only when evals show a gain
Framework opacity Cannot see the actual prompt or response Log raw prompts and responses; drop abstraction layers you do not understand
Compounding errors Early wrong assumption derails the whole run Gates, ground-truth checks after each action, human checkpoints
Infinite loops Agent repeats the same tool call; evaluator never says PASS Step and round limits, cost budgets, repeat detection
Bad plans Orchestrator creates too many, overlapping or impossible subtasks Schema-validate plans, cap their size, log them
Tool misuse Wrong tool, wrong arguments, relative paths Better tool docs, poka-yoke parameters, clear error messages
Context overflow Huge tool outputs push out the instructions Truncate and summarize tool results; store details outside the context
Unsafe side effects Agent sends, deletes, buys or deploys something unintended Approval gates, sandboxes, least-privilege credentials, dry-run modes
Prompt injection Instructions hidden in web pages, emails or files steer the agent Treat tool output as untrusted data; confirm sensitive actions with the user
Misrouting Router sends inputs to the wrong handler Evaluate the router alone; add a safe default route
Silent quality drift Output looks fine, accuracy dropped after a model or prompt change Evals on every change, for each step and end to end

9. Choosing a pattern

Your task Start with
One well-defined transformation (summarize, extract, classify) Single call, good prompt, retrieval if needed
Fixed multi-step process (outline, draft, translate) Prompt chaining with gates
Mixed inputs needing different handling or models Routing
Independent checks or aspects (guardrails, multi-aspect evals) Parallelization: sectioning
High-stakes yes/no judgments Parallelization: voting
Work whose subtasks depend on the input (multi-file code changes) Orchestrator-workers
Output quality improves with critique (translation, writing to a rubric) Evaluator-optimizer
Open-ended goal, unknown steps, good environment feedback, trusted sandbox Autonomous agent, with limits and checkpoints

The patterns combine. A support system might route, use a chain for refunds, and use an agent only for technical troubleshooting. Measure each piece separately.

Why this matters

  • Most value is in workflows. Predefined paths with LLM steps are easier to test, cheaper to run and good enough for most business tasks.
  • Agents are a loop, not a product. The hard work is tools, feedback, limits and evaluation, not the loop itself.
  • Frameworks are optional. Each pattern here is 10-30 lines of plain code. Use a framework only if you understand what it does underneath.
  • Tool design is prompt engineering. The agent-computer interface often matters more than the system prompt.
  • Autonomy needs feedback and guardrails. Agents work best where results can be checked automatically and side effects can be gated.

Final Thoughts

The agent hype will keep growing this year, and more products like Operator and deep research will show what open-ended autonomy can do. For most engineering teams, though, the useful question is not "how do we build an agent?" but "what is the least autonomy this task needs?"

Start with a single call. Add a chain, a router or a parallel check when evals show they help. Reach for an agent when the path truly cannot be written down in advance, and then spend your effort on tools, limits and feedback. Simple, transparent and well-tested beats impressive and fragile.

Sources