I spent the last few weeks building a 16-module wiki on agentic systems, from what an LLM physically is up to multi-agent frontier work. I didn't write it by hand. I ran Claude Code workflows, a multi-agent research pipeline, to draft and cross-check it, then I studied the result until I understood it well enough to argue with it.
This post is what I took out of that. It's the condensed, practical version: what to prioritize and what to ignore if you want to build agents that actually hold up. Everything here is distilled from that wiki, and every section links back to the module where the full argument and the graded sources live.
The uncomfortable part is that almost nothing on the priority list is about the model.
Run the p^t arithmetic before you write any code
Here's the number nobody runs. If each step of your agent succeeds independently with probability p, and the task takes t steps, the task succeeds with probability p^t. Errors don't add. They multiply.
A 10-step task where each step is 90% reliable lands at 0.9^10 ≈ 35%. Push every step to 95% and run a 20-step task and you're still at 0.95^20 ≈ 36%. Human intuition badly under-counts this, which is why so many agents demo beautifully (a short happy path) and die in production (a long real one).
This single curve routes every other decision. It gives you exactly three levers: raise p (per-step reliability), shrink t (shorten the chain), or break the chain into segments that each reset the error. "Use a smarter model" is a weak way to nudge p and does nothing for t. Most things that "need an agent" are better off as a deterministic workflow or a single call once you've seen what the math demands of you. That decision is the first thing the agent-as-policy module makes you confront.
Break the chain: verify against ground truth
If reliability decays geometrically, the highest-impact thing you can build (after knowing your number) is something that resets the error mid-chain. Put a verification gate in the middle and one catastrophic p^20 becomes two friendlier p^10 segments.
The catch: the verification has to come from outside the model. A real tool result. A test that passed or failed. The row that actually committed to the database. Asking the model "did you succeed?" and trusting the answer does not work, because the evaluator shares the generator's blind spots. Intrinsic self-correction does not reliably improve reasoning and can make it worse. Ground truth is the only thing that genuinely resets the per-step error, which is why the control-loop module treats observable feedback as the load-bearing part of the loop, not the prompt.
Own the loop in code; the model is a reducer
The instinct is to hand the whole flow to a framework that "just loops the LLM." That's backwards. Write the outer loop yourself, in plain testable code, and let the model decide only the things you genuinely cannot hardcode: which tool to call, how to phrase the reply.
Think of the model as a stateless reducer: (state, observation) → (thought, tool_call). Everything durable (locks, token budgets, routing, persistence, retries) is boring software you can unit-test and kill on demand. Moving it off the stochastic plane is where demo-to-production reliability actually comes from. The model has no brakes, no steering, and no memory between turns. You supply all three.
The guard in that diagram is the condition everyone skips. Without a no-progress check, an agent will happily burn its entire token budget repeating one failing call. Termination needs multiple conditions: goal reached, budget spent, and no progress since last turn. More on the loop shape in the agent-loop module.
Tools are the action space: design them as UX, not an API wrapper
The model can only be as accurate as the tools you give it. It reasons over the names, schemas, and (this is the part people miss) the error strings you wrote. It reacts to text. It cannot step through a debugger.
The most common mistake is mirroring your database: one tool per table, raw UUIDs in and out, an 8K-token blob back. That maximizes the number of sequential calls the model has to orchestrate, which (see the arithmetic) maximizes the ways it can fail. Collapse N calls into one tool that completes a whole workflow, return values a human could read, and write errors as instructions the model can act on.
// One tool per table. The model has to orchestrate the whole dance,
// hold UUIDs in context, and guess the right order every time.
get_user(id) -> { user_id: "8f3c...", tz_id: 42 }
list_slots(resource_id, day) -> [ 5000-token blob of raw rows ]
create_booking(user_id, slot_id) -> 201
// On a bad input you get: 400.Every arrow is another p. Five tables, five chances to drop the chain.
// One tool for the workflow the agent actually completes.
// Resolves, checks availability, and books, idempotently.
schedule_event({ customer: "Ana", service: "haircut", when: "Tue 5pm" })
-> { status: "booked", with: "Ana", at: "2026-06-30 17:00", id: "evt_91" }
// On a bad input, the error IS the next prompt:
-> { error: "use ISO-8601 for `when`; you sent '5pm'. Did you mean 17:00?" }One arrow. Semantic returns. The error tells the model how to fix itself.
A tool that hands back 8K tokens didn't succeed, it poisoned the next several turns. Cap and paginate output, namespace large catalogs, and make every write idempotent. The full treatment is in a tool is UX for an LLM.
Spend context like an attention budget
A 1M-token window makes "just put it all in" look free. It isn't. Attention is roughly O(n²), and recall sags on a U-curve well before the nominal limit. Every irrelevant token both dilutes the signal the model is attending to and quadruples cost. More context is not more intelligence.
Treat the window as the agent's belief state and curate it every turn. Keep lightweight identifiers in-window and pull the heavy data just-in-time through a tool. Evict stale tool outputs once you've distilled what mattered. Put the load-bearing facts at the very start and the very end, where recall is best. When the agent starts hallucinating, repeating actions, or losing the thread, those are belief-state pathologies. You fix them by eviction, not by enlarging the window and hoping. The context-engineering module names the failure modes so you can diagnose which one you're looking at.
Your eval is the control loop
Your hit rate is capped by your eval's fidelity, full stop. If the eval reads noise, you're driving blind no matter how fast you iterate. And the part that surprises people: the eval grader, the RL reward, and any self-improvement fitness are mathematically the same object. A grader the model can game is a grader that actively trains your agent to fool you.
Two rules carry most of the value. First, build graders bottom-up from error analysis on real traces (open-code ~50 of them into failure themes, then write graders for the frequent and costly categories) rather than top-down against the metrics you imagined. The observed failure distribution almost never matches your a-priori mental model. Second, gate releases on pass^k, not pass@1. One green run is not reliability.
Why pass^k, and why it falls faster than you'd think
pass@1 is a best-case average. pass^k asks whether the agent succeeds k times in a row, which is what "reliable" actually means for anything irreversible. It decays exponentially: a task that passes 90% of the time drops to 0.9^8 ≈ 43% over eight consecutive runs. And step errors are positively correlated in practice (a bad context poisons every following step), so reality usually falls faster than the independent-error math suggests. Grade the persisted state (the row in the DB), never the model's claim that it did the thing. The evaluation module walks through trajectory grading and code-based graders versus LLM-as-judge.
Security and human approval are architecture, not a better prompt
Prompt injection has no model-level fix. The LLM concatenates your system prompt and the poisoned data into one undifferentiated stream with no privilege boundary, the classic confused deputy. In one systematic study, 12 published behavioral defenses fell to adaptive attacks at over 90% success. Spotlighting and "treat documents as data, not instructions" cut measured injection from ~50% to under 2%, which sounds great until an adaptive attacker erases the residual. Useful as defense-in-depth. Never the thing standing between untrusted text and a write.
The boundary has to live in deterministic code. Default-deny capabilities scoped to the task. An outbound egress allowlist (often the single edge whose removal snaps the whole attack chain). The Agents Rule of Two: never leave the agent unsupervised with more than two of these three at once: untrusted input, sensitive data, and a state-changing or egress action. Defend against the adversary the model could become under injection, not the polite one it benchmarks as today.
Human-in-the-loop is the same discipline pointed at irreversibility. Don't confirm everything (that just trains the human to rubber-stamp, and the one action that mattered gets stamped with the rest). Gate only the roughly 1% irreversible-write tail. And close the gap almost nobody checks:
Both are general architectural principles, not framework features. The detail lives in agentic security and human-in-the-loop.
Make every step cheap and safe to re-run
Distributed delivery is at-least-once. A worker can finish a write and die before recording that it did, the queue redelivers, and the step runs again. A naive retry is two bombs at once: a token-cost bomb (you re-charge every model step) and a double-booking bomb (you re-charge the customer's card).
Wrap every step in durable execution so completed steps are memoized, and derive a deterministic idempotency key from (workflow_id, step_id) that you pass to every side-effecting call. On retry the engine replays cached results and the external API dedupes on the key. Model a human approval as a durable suspend, not a worker blocked for three hours. This composes with the p^t math: when every step is cheap and safe to re-run, retrying a flaky chain stops being scary. The production module frames the whole agent as a durable distributed workflow, which is what it is.
What to ignore (or at least defer)
Equally important is where not to spend the afternoon. Every row here is something that feels like progress and mostly isn't.
| Tempting move | Why it's a trap | Do instead |
|---|---|---|
| Reach for multi-agent 'because it scales' | ~79% of failures are coordination, not model limits; fan-out burns ~15× the tokens for parallel compute, not better reasoning | Single linear agent + compaction. Fan out only read-only exploration; keep every write on one thread |
| Trust chain-of-thought as an audit log | Models act on hints they never verbalize and rationalize after the fact; the visible chain is often non-causal | Build evals, gates, and observability on the trajectory of tool calls (what it DID, not what it said) |
| Fine-tune first | Teaches form over substance, ~1.5× inference premium, and training on new facts raises hallucination | Walk the ladder: prompt → few-shot → context/harness → RAG → fine-tune last. The harness is ~90% of ROI |
| Buy a bigger context window | Recall sags on a U-curve before the limit; irrelevant tokens dilute attention even when it all fits | Minimize signal density, not fit. Measure your real effective window |
| Stand up a vector DB by default | An embedding pipeline for 50 docs is a staleness and security liability for nothing | Corpus under ~200k tokens: load it in-context with caching. Lexical (BM25/grep) for identifiers and codes |
| Treat spotlighting as the security boundary | Cuts injection ~50%→<2% but an adaptive attacker erases the residual | Put the privilege boundary in deterministic code; keep spotlighting as defense-in-depth only |
| Crank thinking/effort to max 'just in case' | Past the elbow you hit an overthinking regime: accuracy flattens or drops while you pay per reasoning token | Measure the accuracy-vs-tokens curve per route and set effort at the elbow |
The pattern across all seven: each one is the model's job dressed up as architecture, or architecture dodged with a prompt. The seductive moves point at the LLM. The moves that work point at the code around it.
The questions you're actually asking
Do I even need an agent for this?
Run the p^t math first. If your per-step reliability and step count don't clear your target, decompose the horizon or drop to a deterministic workflow or a single call. Most tasks that "need an agent" don't.
Should I use a multi-agent system?
Not by default. Around 79% of multi-agent failures are coordination, not model limits, and fan-out burns roughly 15× the tokens for parallel compute, not better reasoning. Use a single linear agent with compaction, fan out only read-only exploration, and keep every write on one thread.
Is fine-tuning worth it for my agent?
Usually it's the last resort, not the first. Walk the ladder: prompt, few-shot, context and harness engineering, RAG, then fine-tune. The harness is about 90% of the ROI, and training on new facts actually raises hallucination.
Will a bigger context window make my agent more reliable?
No. Recall sags on a U-curve well before the nominal limit, and irrelevant tokens dilute attention even when everything fits. Minimize signal density instead, and measure your real effective window.
A high hit-rate agent is won by making every step cheap and safe to re-run, and by breaking the error chain with ground-truth verification. Not by a smarter prompt, and not by a smarter model.
If you read only one thing into your hands tomorrow, make it the arithmetic. Open a notebook, write down your p and your t, and look at p^t before you architect anything. The number will tell you whether you need an agent at all, where to put the verification gate, and which of these priorities is actually yours to fix. Everything else in this post is downstream of that one multiplication.
The full version, with every source graded and the proofs behind the claims, is the agentic-systems wiki. This was the map. That's the territory.
Discussion
Comments are hosted on GitHub Discussions — sign in with GitHub to reply.