Agents

Why this week is positioned where it is. Week 6 closed with agentic retrieval — the LLM orchestrates retrieval as a tool. That's the simplest agent pattern. This week generalizes: the agent loop, tool design, planning approaches, memory, and the production patterns that have stabilized. The framing matters because too many teams reach for an agent when a single LLM call or a deterministic pipeline would have been faster, cheaper, and more reliable.

1. What an agent is (and isn't)

An agent is an LLM in a loop with tools. That's it. Strip away the marketing.

Concretely: 1. The model receives a task and a list of available tools. 2. The model decides what to do next: call a tool, or finish. 3. If a tool: the system executes it, gets the result, feeds it back to the model. 4. Loop until the model finishes (or hits a limit).

Everything else — planning frameworks, multi-agent setups, memory systems — is variation on this theme.

What makes an agent different from a single LLM call:

• Multiple turns of reasoning based on intermediate results.
• Side effects via tools — real interactions with the world, not just generation.
• Long-running behavior — tasks that take many minutes or hours.
• Adaptive trajectories — the path the agent takes is decided dynamically.

What an agent is not:

• Not "intelligence." It's a model with a loop.
• Not "autonomy." Every action is initiated by the user or a scheduled trigger; the model has no goals of its own.
• Not always the right answer. Most "agent use cases" are better served by RAG, a structured pipeline, or a single well-prompted call.

The 2026 mature agent products to know:

• Claude Code, Cursor, Windsurf, Cline — IDE-integrated code agents. Real, useful, in production.
• Anthropic's research mode, OpenAI Deep Research, Perplexity — research agents that produce long reports.
• Computer Use (Anthropic), Operator (OpenAI) — desktop and browser automation.
• Cognition Devin — autonomous engineer pitch; still mostly demoware in 2026 but improving.
• Sierra, Harvey, Hippocratic AI — vertical agents (customer service, legal, healthcare).

The unifying observation: the products that ship and stick all start narrow. "Agent for code refactoring in this language" works; "general autonomous agent" doesn't.

2. The agent loop in detail

Every agent runs some variation of:

def agent_loop(task, tools, max_steps=20):
    messages = [{"role": "user", "content": task}]
    for step in range(max_steps):
        response = llm.call(messages, tools=tools)
        if response.stop_reason == "end_turn":
            return response.content
        if response.stop_reason == "tool_use":
            tool_call = response.tool_use
            result = execute(tool_call)
            messages.append({"role": "assistant", "content": response.content})
            messages.append({"role": "user", "content": [{
                "type": "tool_result",
                "tool_use_id": tool_call.id,
                "content": result,
            }]})
    return "Hit max_steps without finishing"

The components:

• Messages list — the running context, growing with each turn. The agent's "memory."
• Tool definitions — function schemas the model can choose from. Names, descriptions, parameter schemas.
• Stop reasons — end_turn (model finished), tool_use (model wants a tool), max_tokens (context full).
• Step limit — bound on iterations. Prevents runaway loops. Always set one.

In production this loop runs surprisingly reliably for many tasks, fails dramatically on others, and the difference is mostly about tool design and prompt structure — not model choice.

The 2026 frontier is not making the loop smarter. It's making the tools better, the prompts tighter, and the context management more selective. Most agent improvements over the last 18 months come from those, not from new orchestration frameworks.

Parallel tool calls. Modern model APIs (Anthropic's tool_use, OpenAI's function calling) support returning multiple tool calls in one response. The system executes them in parallel and returns all results at once. This is a meaningful latency win for independent operations: searching three sources, reading five files, querying multiple databases. Use it where the model can identify independent work.

3. Tool design — the part that matters

Tools are functions the model can call. Each tool has:

• A name.
• A description — the model's only documentation.
• An input schema (typically JSON Schema).
• An execution function on your side.

What separates good tools from bad:

Names matter. search_documents is clear; query_data is vague. The model picks tools partly by name match. Choose names like you would for a public API.

Descriptions matter more. Models read tool descriptions like manual pages. A 200-word description with usage examples beats a 20-word one. Include: what the tool does, when to use it, what it returns, common failure modes, edge-case behavior. The cost of writing more is borne once; the cost of bad tool selection is borne every call.

Schemas should be tight. Required vs optional, value enums where applicable, examples in descriptions. Loose schemas cause hallucinated parameters — the model invents values for fields you didn't constrain.

Idempotent when possible. Tools that can be safely retried are forgiving of agent loops. A delete_user that errors on second call leads the agent to retry-and-fail; a delete_user that returns "already deleted" lets the agent recover gracefully.

Errors should be helpful. Return structured error messages the model can act on: "missing parameterregion" beats TypeError: NoneType has no attribute 'lower'. The model can fix what it can read.

Granularity is a design decision. Five small tools or one big one? Smaller tools compose better, but the model has to plan more steps. Larger tools are simpler but less flexible. Default to smaller; combine via prompt patterns when needed.

The 2026 baseline tool taxonomy most agents use some subset of:

• Search/retrieval — over a corpus or the web.
• Read — open a file, fetch a URL, query a DB.
• Write — create a file, send a message, post a transaction.
• Compute — run code, do math, call an external API.
• Communicate — message another agent or a human.

MCP (Model Context Protocol) has become the standard tool interface in 2026. Anthropic introduced it in late 2024; by mid-2025 it had broad adoption (OpenAI shipped MCP support, then Google, then most major clients). Tool servers expose capabilities via a standard protocol; any MCP-compatible agent can use any MCP server. The result: tool development has decoupled from agent platforms. Worth learning the spec.

4. Planning approaches

The model has to decide what to do at each step. Planning approaches differ in how much structure the prompt imposes on this decision.

ReAct (Reason + Act). The model produces a Thought: then an Action:. Each step alternates reasoning and tool calls. The simplest, most flexible. The 2026 default for general tasks. Most production agents are ReAct under the hood, often without calling it that.

Plan-and-Execute. The model writes a multi-step plan first, then executes step by step. Better for longer tasks where holistic planning matters (research reports, multi-file refactors). Brittle when reality diverges from the plan — the model has to know when to replan, and most prompts handle this poorly.

Tree-of-Thoughts. Explore multiple reasoning branches, evaluate each, pick the best. Mostly research; rare in production due to cost. Useful for problems with verifiable intermediate states (math proofs, puzzles).

Reflexion / Self-critique. After acting, the model critiques its own reasoning and tries again. Useful for tasks with verifiable outcomes (code that runs, math that checks out). Less useful for open-ended tasks where "is this good?" is itself a hard question.

Hierarchical planning. A high-level planner decomposes the task; sub-agents execute the pieces. The multi-agent pattern in disguise.

In 2026, ReAct is the working answer for most tasks. The fancier approaches help on specific problem shapes (long-horizon tasks, formal reasoning, code generation) but add prompt-engineering complexity that often doesn't pay off. Default to ReAct. Reach for more structure when measured eval says it helps on your task.

5. Memory and context management

An agent's "memory" is just its messages list. As the loop runs, it grows. Two failure modes:

Context overflow. The messages list grows past the context window. The agent dies, or — worse — your client silently truncates from the front and the agent loses the original task description.

Lost-in-the-middle. Week 2's failure mode resurfaces. As the messages list grows, important early instructions get less attention. The agent "forgets" what it was doing.

The fixes:

Summarization. Periodically replace old messages with a summary. Standard pattern: every N steps, summarize messages older than the last K. Costs an extra LLM call per summarization but keeps the context bounded.

External memory. Store reasoning, observations, and decisions in a vector DB or a structured scratch pad. Retrieve when relevant. The agent's "long-term memory."

Working memory vs episodic memory. Short-term (in-context) for the current task; long-term (external) for cross-task knowledge. Most production agents do both — and the design of the boundary is one of the harder agent problems.

Selective context. Don't pass every tool result back to the model. If read_file() returned 30,000 tokens but only one section matters, summarize before passing along. The "context engineering" lessons from Week 2 apply doubly to agents.

The 2026 frontier: agents that maintain explicit memory objects (lists, dicts, even small databases) and update them via tools. Cleaner than implicit summarization but requires more agent-design work — you're now designing schemas for the agent's mental state.

6. Multi-agent patterns

When one agent isn't enough:

Orchestrator / worker. A planning agent decomposes tasks; worker agents execute pieces. Communication via shared workspace, message passing, or just nested function calls.

Debate. Two agents argue different positions; a judge synthesizes. Useful for nuanced decisions where multiple perspectives genuinely differ.

Specialist roles. Different agents have different tools, prompts, and instructions. A team-with-specialties model.

Voting / ensemble. Multiple agents independently solve the same problem; a judge or a voting mechanism picks an answer. Reduces variance, useful for high-stakes decisions.

The 2026 reality: multi-agent setups are seductive in theory and brittle in practice. The communication overhead, prompt engineering across agents, and debugging complexity often dominate the marginal capability gain. Most production "multi-agent" systems are actually one orchestrator agent with tool calls that happen to invoke other LLM-powered services. That's fine — and clearer to reason about.

When multi-agent really helps: - Truly parallelizable subtasks (independent research streams, sharded data processing). - Tasks with formal coordination structure (debate, voting, adversarial review). - Specialization where one model can't reasonably carry all roles (e.g., code agent + design agent + product agent for a feature).

When it doesn't: - Most other things. Single-agent with good tools is usually enough, and the few-percent quality gain from multi-agent often disappears against the eng-cost increase.

7. Evaluation

Evaluating agents is harder than evaluating LLMs. The output is a trajectory, not just a final answer.

The core metrics:

• Goal-completion rate — did it finish the task correctly? Binary or graded.
• Trajectory quality — was the path the agent took reasonable, or did it wander?
• Tool-call success rate — what fraction of tool calls succeeded?
• Cost / latency per task — production budget metric.
• Hallucinated calls — calls to non-existent tools, or with bad arguments.
• Step efficiency — did it finish in fewer steps than alternatives?

Approaches:

• Human evaluation. Labelers grade trajectories. Expensive but the ground truth. Worth it for the eval set you'll iterate against.
• LLM-as-judge. Another LLM evaluates the trajectory. Cheaper, noisier; works well for clear success criteria.
• Programmatic. For tasks with verifiable outcomes (code passes tests, math checks, file diff matches expected). The cleanest eval signal.
• Counterfactual. Would this trajectory have succeeded? Hard to assess; rarely done well.

The 2026 benchmarks worth knowing:

• SWE-bench — code agents on real GitHub issues. The reference benchmark for code agents.
• GAIA — general assistant tasks across domains. Hard, multi-step.
• AgentBench — multi-domain agent capabilities.
• WebArena, VisualWebArena — browser automation.
• Cybench — security-focused tasks.

The benchmarks are useful comparison points. Build your own eval set first — it'll match your task distribution far better than any public benchmark.

8. The agent design playbook

When to reach for an agent:

1. The task requires multiple steps that depend on intermediate results.
2. The required tools or actions can't be fully predetermined — the path varies by input.
3. The user can tolerate higher latency (several seconds to minutes) for higher capability.
4. You have a budget for the multi-call cost.

When to skip:

1. Single-shot LLM works. Most tasks are this. Try it first.
2. RAG is sufficient. Q&A over docs. Don't add a loop you don't need.
3. A deterministic pipeline works. Most automation is this. Pipelines are predictable, debuggable, and cheap.
4. Latency or cost are tight. Each agent step is a model call. The math adds up.

When you do build an agent:

1. Start with ReAct + a few tools. No planning frameworks, no multi-agent. Add complexity only if measured eval demands it.
2. Build trace logging from day one. You will need to inspect trajectories. Cheaper to add early than retrofitted.
3. Set hard step limits. Always. Agents loop, even good ones. Step limits are the safety net.
4. Make tool errors helpful. This is the highest-leverage design move you can make.
5. Iterate on tool design more than on model choice. Most "model isn't smart enough" complaints are tool-design issues.
6. Eval on trajectory, not just final answer. A correct answer reached via 30 wandering steps and an irrelevant final answer reached via 3 steps need different fixes.

The honest summary of the 2026 agent landscape: agents are useful, agents are hard, and most teams reach for them too eagerly. The discipline is asking what's the simplest thing that could work before reaching for the loop. When the answer genuinely requires multi-step adaptive behavior, agents are the right tool. When it doesn't, they're an expensive way to get worse results than a structured pipeline.

Build this week

Pick at least two:

1. Build a simple ReAct agent. Use the Anthropic or OpenAI tool-use API with three tools (search, fetch, code-exec). Run it on a research task. Log every step. Inspect the trajectory.

2. Tool design audit. Take an existing agent (yours or someone else's) and rewrite the tool descriptions and error messages. Measure before and after on goal-completion rate. The wins are usually obvious.

3. Build a step-limit-aware agent. Implement summarization that triggers when context is over 80% full. Test on a task that needs more than the naive context budget.

4. Build an eval harness for an agent. 20 tasks with known correct outcomes. Run agent, score on goal-completion + trajectory quality (manual). Use as your iteration baseline.

5. Reproduce a known agent benchmark. Pick SWE-bench-Lite or a small subset of GAIA. Run a baseline agent. Compare to public results. The gap is usually larger than you'd expect — and that's the most useful lesson.

Read this

• Anthropic's "Building effective agents" post (Dec 2024). The single best practical write-up. Required reading.
• The MCP specification. Increasingly the standard tool interface. Worth learning.
• The original ReAct paper (Yao et al., 2022). Worth reading once for the framing.
• SWE-bench paper and leaderboard. The canonical code-agent benchmark.
• OpenAI's Operator and Anthropic's Computer Use docs. Best primary sources for browser/desktop agent design.
• "What is an Agent?" (Simon Willison's posts). Skeptical-practitioner takes; corrects most of the hype.

Interview prompts

1. Walk through the agent loop. What are the components and where do they typically fail?
2. Compare ReAct, Plan-and-Execute, and Reflexion. When is each appropriate?
3. What makes a tool well-designed vs poorly-designed? Give three concrete principles.
4. How do you evaluate an agent? What do you measure beyond final-answer correctness?
5. Your agent loops on a task — calls the same tool with the same arguments three times. What's the most likely root cause?
6. When does multi-agent actually help, and when is single-agent sufficient?
7. Walk through the memory-management options for a long-running agent. Tradeoffs?
8. What is MCP and why does it matter?
9. You're choosing between RAG and an agent for a Q&A task. What's the deciding question?
10. A teammate proposes a 5-agent system with planning, research, drafting, review, and finalization roles. What's your reaction and follow-up?

What "done" looks like

By the end of this week you should be able to:

• Build a working ReAct agent with appropriate tools, prompts, and step limits.
• Identify when a task does and doesn't warrant an agent vs a simpler approach.
• Diagnose the canonical agent failure modes from a trajectory log.
• Design tools that produce good agent behavior — names, descriptions, schemas, errors.
• Make principled tradeoffs between agent and pipeline architectures.
• Recognize that "we built a multi-agent system" is usually a yellow flag, not a green one.

If you can do those, you're ready for Week 8 — evaluation. The discipline that makes everything in Weeks 5-7 actually iterate forward instead of vibes-based.