Tim WilcoxsonLiberalisby Tim Wilcoxson

Paper · Independent research synthesis

Designing and Building Agentic Systems: A 2026 Survey of the State of the Practice

Tim Wilcoxson · 2026.07.09

Abstract

The hard part of building an agentic system has moved out of the model and into the system around it. This survey distills 118 authoritative sources, gathered and independently verified through 9 July 2026, into a single account of how practitioners and researchers actually design agents today: when to build one at all, how to engineer context, tools, memory, and retrieval, how to reason and plan across long horizons, when multi-agent architectures help versus hurt, how to make long-running agents reliable and durable, how to evaluate and secure them, and what separates deployments that ship from the 95 percent of pilots that do not. Every claim carries an inline citation to a source whose quoted wording was confirmed verbatim on the live page. The recurring finding is that context, architecture, and organization, not raw model intelligence, are the binding constraints.

Introduction

The first question on any agent project is whether to build an agent at all, before choosing a model or framework, and that is where the 2026 primary literature has settled. The vendors who sell agent frameworks now open their own guides by warning you away from agents. The most consistent message across the sources is that the difficulty of building useful agents has moved out of the model and into the system wrapped around it.

Four claims, each independently sourced, define the current consensus. First, context is the binding constraint rather than intelligence: Anthropic’s guidance treats context as “a finite resource with diminishing marginal returns” [S8] because “as the number of tokens in the context window increases, the model’s ability to accurately recall information from that context decreases” [S8], and Cognition, which ships a production coding agent, calls context engineering “effectively the #1 job of engineers building AI agents” [S12]. Second, reliability is an architectural property: a Princeton-led paper argues it “cannot be inferred from mean task success alone” [S68], and MLflow puts it plainly, “reliability comes from modular design, rigorous state management, and deterministic guardrails, not from better prompts alone” [S67]. Third, simplicity comes first: Anthropic recommends “finding the simplest solution possible, and only increasing complexity when needed” [S1]. Fourth, most enterprise value comes from organization rather than the model: Stanford’s study of 51 successful deployments concludes “The difference was never the AI model. It was always the organization” [S92].

The numbers are large. A well-designed multi-agent research system beat a single agent by 90.2 percent [S16]. Moving tool orchestration into code cut token use by 98.7 percent in one Anthropic example [S30] and 99.9 percent in Cloudflare’s Code Mode [S33]. The failure numbers are just as clear: an agent that is 85 percent reliable at each step completes a 10-step workflow “only about 20% of the time” [S66].

Scope and method. This survey covers text and tool-based agentic workflows: the design decision, the pattern library, context, tools and the Model Context Protocol, memory, reasoning and planning, multi-agent architecture, coding harnesses, frameworks and durable orchestration, reliability and observability, security, evaluation, agentic retrieval, architectural theory, deployment reality, and instruction design. Voice, multimodal, and computer-use agents are underrepresented, and framework coverage leans toward the labs’ own SDKs.

The underlying corpus was built as a three-stage pipeline. Sixteen theme-scoped research agents searched the live web, ranked sources by authority (lab and framework primaries and peer-grade arXiv first, recognized practitioners second), and saved each fetched page. An independent verification pass then re-fetched every cited URL and checked each quoted string character by character against the live page; confirmed and wording-corrected quotes survived, anything not found was dropped, and sources left with zero surviving quotes were discarded. The synthesis was written only from surviving quotes. The one rule the collection enforces is traceability: every claim below carries an inline [S#] that resolves in the References to a title, organization, date, and URL, and every quoted string was confirmed verbatim on that source. The material postdates the model’s training cutoff, so it comes from the fetched literature, not from memory. The corpus is 118 distinct sources and 495 verified quotes, the large majority tier-1 primaries.

The first architectural decision: agent, workflow, or neither

The core distinction is who controls the flow of execution. Anthropic defines the two halves: “Workflows are systems where LLMs and tools are orchestrated through predefined code paths” [S1], while “Agents, on the other hand, are systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks” [S1]. LangGraph compresses it: “Workflows have predetermined code paths and are designed to operate in a certain order. Agents are dynamic and define their own processes and tool usage” [S4]. The key word is dynamic. There is also a third category most teams sit in without noticing. OpenAI draws the line: “Applications that integrate LLMs but don’t use them to control workflow execution, think simple chatbots, single-turn LLMs, or sentiment classifiers, are not agents” [S2].

Agency is a matter of degree. Hugging Face’s smolagents guide frames it as a spectrum, “as you give more or less power to the LLM on your workflow” [S5], and argues the default should be low: “For the sake of simplicity and robustness, it’s advised to regularize towards not using any agentic behaviour” [S5]. Before an agent is warranted, the use case has to pass a threshold test. Google Cloud states it: “If your workload is predictable or highly structured, or if it can be executed with a single call to an AI model, it can be more cost effective to explore non-agentic solutions” [S3]. OpenAI makes it a hard checkpoint: “Before committing to building an agent, validate that your use case can meet these criteria clearly. Otherwise, a deterministic solution may suffice” [S2]. Agents are worth using “where traditional deterministic and rule-based approaches fall short” [S2].

When you do add agency, there is a named pattern library to climb one rung at a time. LangGraph catalogs the workflow patterns: prompt chaining, where “each LLM call processes the output of the previous call” [S4]; parallelization, running independent subtasks at once or the same task several times “to check for different outputs” [S4]; and evaluator-optimizer, where “one LLM call creates a response and the other evaluates that response” [S4]. Routing sits alongside these, and smolagents advises hand-coding the simplest rungs rather than importing a framework: “For some low-level agentic use cases, like chains or routers, you can write all the code yourself” [S5]. The pattern that starts to look agentic is orchestrator-workers, whose “key difference from parallelization is its flexibility, subtasks aren’t pre-defined, but determined by the orchestrator based on the specific input” [S1]. The practical rule: if you can enumerate the subtasks before you see the input, parallelize, because it is cheaper and predictable; if you cannot, pay for an orchestrator. Once agency is warranted, OpenAI’s escalation rule holds: “maximize a single agent’s capabilities first” [S2] before spawning more. Frontier work pushes past the static ladder, arguing that fixed templates “apply the same configuration regardless of query difficulty, leading to brittle behavior and wasted compute” [S6] and that “learning per-query agent configurations is a powerful alternative” [S6]. The mature reading is that simplest is a property of each query rather than of the whole app.

Context engineering

Context engineering is deciding, at every step of a run, which tokens occupy the window and which stay out. The field renamed itself from prompt engineering for a concrete reason: a prompt is a one-shot act, but an agent accumulates tool outputs and messages across hundreds of turns, and someone has to curate the growing pile. “Context refers to the set of tokens included when sampling from a large-language model” [S8], and the working definition is “the art and science of filling the context window with just the right information at each step of an agent’s trajectory” [S9].

The reason curation matters is empirical. Chroma tested the common assumption that a model “should handle the 10,000th token just as reliably as the 100th” [S10] across 18 frontier models and found it false: “models do not use their context uniformly; instead, their performance grows increasingly unreliable as input length grows” [S10]. It is not only length. Even one irrelevant chunk hurts: “Even a single distractor reduces performance relative to the baseline” [S10]. Practitioners treat this “context rot” as settled, “a result of models making less intelligent decisions at longer and longer context lengths” [S13].

LangChain’s framework gives four moves. Write means “saving it outside the context window” [S9]; select means “pulling it into the context window” [S9]; compress means “retaining only the tokens required to perform a task” [S9]; isolate means “splitting it up” [S9] into separate windows. The production primitives implement these. Compaction is “taking a conversation nearing the context window limit, summarizing its contents, and reinitiating a new context window with the summary” [S8]. Structured note-taking is the write move made durable, “notes persisted to memory outside of the context window” [S8]. Just-in-time retrieval is the select move done lazily: agents “maintain lightweight identifiers (file paths, stored queries, web links, etc.) and use these references to dynamically load data into context at runtime” [S8].

Manus adds the cost lens. It argues “the KV-cache hit rate is the single most important metric for a production-stage AI agent” [S11], which means you never rewrite the prefix of your context, because a cache miss re-bills every earlier token. So Manus grows its action space by masking tool logits rather than editing the tool set [S11], pushes state out to files [S11], fights goal drift by reciting a todo list into the end of the context [S11], and keeps failures in the window because “Erasing failure removes evidence” [S11]. Isolate is where judgment is hardest, and where the field changed its mind: Cognition argued in 2025 to “Share context, and share full agent traces, not just individual messages” [S12], then in 2026 concluded isolation is safe only in specific shapes, mainly read-only subagents and clean-context reviewers, where “A clean-context reviewer catches bugs the coder can’t see” [S13].

Memory and retrieval

A language model is a pure function: tokens in, tokens out, nothing retained. Memory is the layer that lets an agent carry facts, preferences, and learned procedures across sessions, and the naive fix of a bigger window was tested and failed. The LoCoMo benchmark showed that past a handful of sessions, “LLMs exhibit challenges in understanding lengthy conversations and comprehending long-range temporal and causal dynamics” [S40], and that “long-context LLMs or RAG can offer improvements but these models still substantially lag behind human performance” [S40].

A memory system runs a write, manage, read loop, “dynamically extracting, consolidating, and retrieving salient information” [S38]. What gets stored splits into three types, and builders under-invest in the third: episodic (“what happened”), semantic (“what is known”), and procedural (“Production agents also need a third: procedural memory”) [S36]. The 2026 architectures disagree on shape. Mem0 treats memory as a managed fact store and reports against a full-context baseline “a 91% lower p95 latency and saves more than 90% token cost” [S38]. Zep/Graphiti argues for a graph “maintaining historical relationships” [S39]. Letta makes memory an agent-editable block, “an elegant abstraction for context window management” [S41]. Anthropic externalizes it to files plus context editing, reporting that in a 100-turn evaluation, editing “enabled agents to complete workflows that would otherwise fail due to context exhaustion, while reducing token consumption by 84%” [S42]. A-MEM targets organization, letting memory evolve so old entries get rewritten rather than merely stacked [S43].

Two cautions carry across the memory literature. Measure accuracy per token rather than accuracy alone: “A system that scores well on accuracy but requires 26,000 tokens per query is not production-viable” [S36], and “Accuracy without a token budget is a half-finished score” [S37]. And writable memory is a new attack surface: it “introduces a qualitatively different threat landscape” and its security “cannot be retrofitted at retrieval or execution time alone” [S44]. Treat memory writes like database writes: auditable, attributable, recoverable.

Retrieval is the sibling discipline, and the 2026 reframing is to treat search as something an agent decides to do rather than a fixed pipeline stage. Below roughly 200,000 tokens, skip it entirely: “you can just include the entire knowledge base in the prompt” [S101]. Above it, prefer a hybrid of small up-front context plus just-in-time tool loading [S8], and make retrieval a conditional tool that grades its own results and rewrites the query when they are irrelevant [S102]. Retrieval quality still matters: Contextual Retrieval “reduced the top-20-chunk retrieval failure rate by 49%” [S101], and combined with reranking, 67 percent [S101]. Here the sources disagree: Pi-Serini shows that for a capable agent in a loop, a tuned lexical retriever can suffice, with “BM25 tuning improves answer accuracy by 18.0%” [S107], because the agent compensates by reading and re-querying. For global questions that map to no specific chunk, GraphRAG builds an entity graph and community summaries to handle “query-focused summarization” [S103], and corpus navigation helps only when the corpus has “a recoverable topical taxonomy” [S106].

Tools and the Model Context Protocol

Tools are how an agent acts on the world, and by 2026 building them has split away from ordinary API design because the caller is a model rather than a programmer who reads docs. Anthropic frames tools as “a new kind of software which reflects a contract between deterministic systems and non-deterministic agents” [S29]. The reorientation is to stop porting your existing API surface and instead “design them for agents” [S29], which means consolidation, because “More tools don’t always lead to better outcomes” [S29]. Tool descriptions are the only spec the model gets, and “Even small refinements to tool descriptions can yield dramatic improvements” [S29]. Two mechanisms keep context clean: cap output (Claude Code restricts tool responses to 25,000 tokens by default [S29]), and turn errors into steering, prompt-engineering error responses “to clearly communicate specific and actionable improvements, rather than opaque error codes or tracebacks” [S29].

The Model Context Protocol is the wire standard that in 2026 matured from convention to production infrastructure. It “now runs in production at companies large and small” [S32], and the defining architectural change is that “MCP is now stateless at the protocol layer” [S31] to let servers run as remote services behind load balancers, though “Removing the protocol-level session does not mean your application has to be stateless” [S31]. This is flagged as an exceptional breaking change with a final spec dated after this survey’s cutoff, so treat those details as subject to change.

MCP’s success created its own scaling problem: “every tool added fills the model’s context window” [S33], and at the extreme, agents connected to thousands of tools “need to process hundreds of thousands of tokens before reading a request” [S30]. The 2026 answer is to stop exposing tools directly and let the agent write code that calls them. Cloudflare’s Code Mode fronts an entire API “With just two tools, search() and execute()” consuming “only around 1,000 tokens” [S33], reducing input tokens by 99.9 percent [S33]; Anthropic reports the same pattern cutting a task “from 150,000 tokens to 2,000 tokens, a time and cost saving of 98.7%” [S30], with a privacy dividend because “intermediate results stay in the execution environment by default” [S30]. The efficiency wins do not close the reliability gap: MCP-Bench connects models to “28 representative live MCP servers spanning 250 tools” [S35] and finds “persistent challenges” [S35] for 20 advanced models on multi-step coordination.

Reasoning, planning, and test-time compute

Reasoning and planning split into two concerns: the in-context reasoning a model does per step, and the cognitive architecture an engineer builds to keep a long task coherent. The foundational pattern is ReAct, which interleaved “reasoning traces and task-specific actions” [S46] so that reasoning and acting inform each other, beating prior methods “by an absolute success rate of 34% and 10%” [S46] with only one or two in-context examples. By 2026 the framing is a paradigm shift toward agents that “plan, act, and learn through continual interaction” [S51], because static reasoning “struggle[s] in open-ended and dynamic environments” [S51].

ReAct’s weakness is its open-endedness. LangChain’s field observation is that as the agent accumulates observations, “the context window growing… can cause the LLM to get ‘distracted’ and perform poorly” [S47], which is why “nearly all the advanced ‘agents’ we see in production actually have a very domain specific and custom cognitive architecture” [S47]. The fix is to move planning out of the model, “essentially removing the planning responsibilities from the LLM to us as engineers” [S47].

Test-time compute (spending more tokens or samples at inference) improves answers, but it changes shape for agents. You cannot simply sample and vote, because “each attempt produces an extended trajectory of actions, observations, errors, and partial progress” [S50]. The thesis is that “test-time scaling for long-horizon agents is fundamentally a problem of representation, selection, and reuse” [S50], and it delivers a measurable gain: Claude-4.5-Opus “improves from 70.9% to 77.6% on SWE-Bench Verified” [S50]. On selection, compare candidates against each other rather than scoring in isolation, since “the list-wise method performs best” [S48]. Reflection should be gated rather than always on, because “knowing when to reflect is important for agents” [S48]. For horizons that outrun a single window, externalize state: “start the session by reading the progress notes file and git commit logs” [S49] so a fresh context reconstructs where it is. Genuine long-horizon planning under hard constraints is still unsolved; benchmarks that test it find “even frontier agentic LLMs struggle” [S52].

Multi-agent systems: when they help, when they hurt

A multi-agent system splits a task across several LLM instances. In 2026 this is contested because the same architecture delivered a large win in one domain and fragile systems in another. Anthropic’s research product is the clearest published win: a lead agent delegating to parallel subagents “outperformed single-agent Claude Opus 4 by 90.2% on our internal research eval” [S16]. The mechanism explains when the win transfers: “token usage by itself explains 80% of the variance” [S16], meaning subagents help mostly because each has a fresh context window, so the system reads more source material than one agent could hold. Simon Willison, initially skeptical, arrived at exactly this: “The key benefit is all about managing that 200,000 token context limit” [S21].

One day before that post, Cognition published the opposite headline, calling the orchestrator-splits-into-subagents pattern “very fragile” [S12] because “Actions carry implicit decisions, and conflicting decisions carry bad results” [S12]. The academic evidence agrees the hype outran results: the Berkeley/Databricks MAST study, built on “1600+ annotated traces” [S18], produces a 14-mode failure taxonomy in which two of three categories are coordination failures, not model-capability failures [S18].

LangChain resolved the contradiction in one sentence: “read actions are inherently more parallelizable than write actions” [S17]. Reads can run in any order; writes encode decisions that can conflict. So “Multi-agent systems designed primarily for ‘reading’ tasks tend to be more manageable than those focused on ‘writing’ tasks” [S17], which is why research suits fan-out while coding does not, a point Anthropic conceded: “most coding tasks involve fewer truly parallelizable tasks than research” [S16]. By 2026 Cognition had evolved to a scoped yes: “multi-agent systems work best today when writes stay single-threaded and the additional agents contribute intelligence rather than actions” [S13], in a “map-reduce-and-manage” [S13] topology. The counterintuitive exception is verification: a generator and verifier “work best when the coding and review agents do not share any context beforehand” [S13], because a reviewer primed with the author’s reasoning inherits its blind spots. Cost is significant: “multi-agent systems use about 15× more tokens than chats” [S16], and a topology benchmark found hierarchical supervisor-worker “occupy the most favorable position on the cost-accuracy Pareto frontier (F1 0.921 at 1.4x cost)” [S20] while reflexive loops buy the last few points at 2.3x cost.

Coding agents, harnesses, and architectural theory

Coding is the best-studied agentic domain because a change either compiles and passes tests or it does not. That verifiability made it the field’s laboratory for one question: how much of an agent’s competence lives in the model versus the harness, the scaffolding around each model call. The answer is stark. “The core of the system is a simple while-loop that calls the model, runs tools, and repeats” [S81], and everything interesting sits outside it: a permission system, a compaction pipeline, extensibility mechanisms, subagent orchestration [S81]. Community analysis estimates “only about 1.6% of Claude Code’s codebase constitutes AI decision logic, with the remaining 98.4% being operational infrastructure” [S81]. Treat 1.6 percent as directional, but the division of labor is the lesson: “the model reasons about what to do; the harness is responsible for executing actions” [S81], and “the scaffolding code that surrounds the language model… increasingly determines how the agent behaves” [S80].

Source-level analysis dismantles the old habit of naming agents by architecture. A study of 13 scaffolds found that “Five loop primitives (ReAct, generate-test-repair, plan-execute, multi-attempt retry, tree search) function as composable building blocks” and that “11 of 13 agents compose multiple primitives” [S80]. The design choices are tuning decisions. SWE-agent named the underlying insight in 2024: agents “represent a new category of end users” [S84] who benefit from a purpose-built agent-computer interface, and its contribution was a better interface to an existing model rather than a smarter model. For long-running work the recipe is an initializer plus incremental sessions that leave “clear artifacts for the next session” [S49], because a memoryless successor reconstructs state “with the claude-progress.txt file alongside the git history” [S49]. Splitting the generator from a skeptical evaluator helps because “tuning a standalone evaluator to be skeptical turns out to be far more tractable than making a generator critical of its own work” [S79], but the evaluator costs tokens, so it “is not a fixed yes-or-no decision” [S79]. The enduring discipline is that “every component in a harness encodes an assumption about what the model can’t do on its own” [S79], and those assumptions go stale as models improve.

Architectural theory hardened from convenience lists into a discipline. The reference point is CSIRO Data61’s catalogue of “18 architectural patterns” with “a decision model for selecting the patterns” [S89]. The 2026 critique is that flat catalogs “lack a rigorous systems-theoretic foundation” [S85], answered by decomposing an agent into functional subsystems before placing patterns against them [S85]. The sharpest contribution is that one axis is not enough: “the same Orchestrator-Workers topology can implement Plan-and-Execute, Hierarchical Delegation, or Adversarial Verification, three patterns with fundamentally different failure modes” [S86], so patterns must be selected along both execution topology and cognitive function. Every empirical survey hits the same wall: prototypes arrive fast, but “variability in LLM behaviour… leads to challenges in transitioning from prototype to production maturity” [S87].

Frameworks and durable orchestration

Building an agentic system means two independent decisions beginners collapse into one: which framework defines the agents, and what substrate keeps execution alive across crashes and multi-hour runs. Frameworks differ mainly in how much of the loop they run for you. The Claude Agent SDK is the low-effort end, giving “Claude with built-in tool execution” [S22] with control returned through hooks and subagents. OpenAI’s SDK calls delegation handoffs [S23]; Google’s ADK composes “multiple specialized agents in a hierarchy” [S24]; CrewAI splits Flows from Crews [S113]; AG2 builds everything on the ConversableAgent [S115] with a GroupChat where “all agents contribute to a single conversation thread” [S114]; Microsoft’s Agent Framework combines “AutoGen’s simple abstractions… with Semantic Kernel’s enterprise-grade features” [S116]; and LlamaIndex frames workflows as “an event-driven, step-based way to control the execution flow” [S118] in plain Python, positioned against DAG-style frameworks.

The reason orchestration is separate is compound failure. Temporal argues that “the odds an AI agent experiences some kind of failure increase with each marginal step” until “failure becomes not just possible but asymptotically certain” [S96], and Inngest notes these workflows “are long-running by nature” [S100]. Framework-native persistence is not a complete answer. LangGraph’s default in-memory savers “store checkpoints in RAM. When the process restarts, all checkpoints are lost” [S27], and a vendor with a durable-execution stake argues “checkpointing is not production-grade durability” [S25] because there is “no supervisor, no watchdog, no heartbeat mechanism” [S25]. Durable-execution engines close the gap by journaling and replay: Restate “remembers all the steps it did in a journal, and will resume… by replaying the journal” [S99], and the guarantee that matters is exactly-once, so “the planning and search steps don’t re-execute” [S100] on resume.

The most important trap here is that classical durable execution assumes a retried call is identical to the original, “an assumption that holds for traditional programs but fails for LLM agents, which re-synthesize subtly different requests after restore” [S98]. The consequence is a security bug: servers “treat these re-generated requests as new, enabling duplicate payments, unauthorized reuse of consumed credentials” [S98], which ACRFence names semantic rollback attacks and mitigates with “replay-or-fork semantics” [S98]. Atomix attacks the same seam transactionally, classifying effects by reversibility and letting “irreversible effects leave the gate” [S97] only once conflicting work is exhausted. Before enabling replay, gate every irreversible tool effect behind replay-or-fork or transactional commit.

Reliability and observability

Reliability and observability decide whether an agent that performed well in a notebook survives production traffic. The main shift is to stop reporting a single accuracy number. The Princeton-led reliability science gives “twelve metrics that decompose agent reliability along four key dimensions” [S68], because a mean “ignores whether agents behave consistently across runs, withstand perturbations, fail predictably, or have bounded error severity” [S68]. The underlying arithmetic: even at 85 percent per step, a 10-step workflow succeeds only about 20 percent of the time, because per-step reliability multiplies, roughly 0.8510≈0.200.85^{10}\approx0.20 [S66]. That collapse happens “not because the model got something wrong, but because the system had no way to checkpoint progress” [S66]. Checkpointing is the mechanism: a checkpointer “saves a snapshot of graph state at each super-step” [S65] and stores pending writes from nodes that succeeded, so a resume replays only what failed [S65].

The most useful recent primitive names where reliability is determined. Srinivasan’s methodology defines “the stochastic-deterministic boundary (SDB): a four-part contract among a proposer, verifier, commit step, and reject signal” [S69] and argues “the SDB is the load-bearing primitive of production agent runtimes” [S69]. It surfaces a failure unique to durable, event-sourced agents: “replay divergence, in which LLM-based consumers of a deterministic event log produce different downstream outputs under model-version or prompt changes” [S69]. The forward-looking point is that “as model variance decreases, pattern choice and SDB strength become increasingly important levers” [S69], so architecture matters more as models converge.

Observability makes the property visible. Because AI is nondeterministic, “debugging your application without any observability tool is more like guesswork” [S64], so tracing “records the flow of a request through your system, preserving causal relationships” [S64]. The OpenTelemetry GenAI conventions standardize the span tree, “the top-level invoke_agent span with child chat spans for each LLM call and execute_tool spans” [S63], with two defaults worth engineering around: content capture is opt-in, since “By default, no prompt content or tool arguments are captured” [S63], and ephemeral instance ids are kept off attributes to avoid cardinality blowups [S62].

Evaluation

Evaluation is where confident numbers are least trustworthy. The most consequential distinction is between “can it ever do this” and “will it do this every time.” Anthropic defines the pair: “pass@k measures the likelihood that an agent gets at least one correct solution in k attempts” [S54], a capability ceiling, while “pass^k measures the probability that all k trials succeed” [S54], the reliability bar. The mechanism is probability compounding: if per-trial success is pp, then passk≈pk\text{pass}^k \approx p^k, so an agent at 90 percent single-shot drops to about 59 percent over five trials. A demo reflects pass@1 on one favorable path, while production requires pass^k across a stochastic user base.

The benchmarks worth knowing each close a specific hole. SWE-bench Verified is “A human-filtered subset of 500 instances” [S59] reviewed so tasks are “solvable given the available information” [S59]. GAIA targets general assistant ability with questions “conceptually simple for humans yet challenging for most advanced AIs: 92% vs. 15% for GPT-4 equipped with plugins” [S56], holding out answers to guard against contamination [S56]. τ²-bench closes the assumption that the user is passive, modeling a “dual-control” domain where both agent and user act, and finding “significant performance drops when agents shift from no-user to dual-control” [S55].

The judging problem is that automated graders barely agree with humans. AgentProp-Bench found substring judging agrees with humans “at kappa=0.049 (chance-level)” while a three-LLM ensemble reaches only “kappa=0.432 (moderate)” [S60], and that a single wrong tool argument propagates to a wrong final answer “with human-calibrated probability approximately 0.62” [S60], so grading only the endpoint hides where the agent broke. Consistency does not make an LLM judge valid: the largest study, roughly 541,000 judgments, found “high test-retest reliability (>0.95) coexists with severe position bias” [S58], a repeatable judge that is still biased by answer order. Two more failure modes close the picture: simulated users are unreliable proxies, with success rates “varying up to 9 percentage points across different user LLMs” [S61] and worse rates for AAVE speakers [S61]; and aggregate scores hide behavior, which is why the Holistic Agent Leaderboard used “LLM-aided log inspection to uncover previously unreported behaviors, such as searching for the benchmark on HuggingFace instead of solving a task” [S57]. The reliable evaluation signal is the full transcript rather than the aggregate score.

Security and safety

Security for an agent is a different problem than for a chatbot, because an agent reads attacker-controllable text and then acts. Simon Willison’s lethal trifecta is the clearest model: access to private data, exposure to untrusted content, and “The ability to externally communicate in a way that could be used to steal your data” [S70]. Combine all three and injected instructions can read private data and exfiltrate it; remove any leg and the chain breaks. The mitigation follows: “once an LLM agent has ingested untrusted input, it must be constrained so that it is impossible for that input to trigger any consequential actions” [S70].

The unifying thesis is to contain capability rather than behavior. Anthropic supervises “what it’s able to do by enforcing access boundaries through, for example, sandboxes, virtual machines, and egress controls” [S71], because training “shape[s] only what the agent tends to do, not what it is theoretically capable of doing” [S71]. The defense that holds against exfiltration is “egress controls that block the POST regardless of intent” [S71]. Microsoft’s red team taxonomy, grounded in “12 months of red team engagements” [S72], reports two findings that should reshape priorities: “Cross-domain prompt injection delivered via external content remained the most reliable initial access vector” [S72], and “HitL bypass was the most consistently exploited failure mode, at very high frequency” [S72]. The control teams reach for first, a human in the loop, is the one most reliably defeated, and memory poisoning “requires only a single successful injection, which the agent then propagates across subsequent sessions” [S72].

New attack surfaces follow. OWASP’s MCP Top 10 defines schema poisoning, where “the attacker doesn’t exploit a code bug directly, they change the contract so legitimate agents behave incorrectly” [S73], mitigated by signing schemas so “Agents must verify signatures before accepting or using a schema” [S73]. By-design defenses move robustness off the prompt. OpenAI’s instruction hierarchy addresses the root cause, that models treat system and untrusted text “to be the same priority” [S77], and reports it “drastically increases robustness” [S77], though it is still probabilistic. CaMeL adds a system layer that “explicitly extracts the control and data flows from the (trusted) query” [S75] so untrusted data “can never impact the program flow” [S75], at a measured cost of “77% of tasks with provable security (compared to 84% with an undefended system)” [S75]. Human approval keeps a scoped place, with reviewers able to “Modify the tool arguments before execution” [S76], but given the bypass finding it should supplement environmental and system-layer controls rather than replace them.

Deployment reality

Deployment reality is the gap between a demo and a system that survives a real organization. The most-cited backdrop is that “95% of generative AI pilot programs fail to produce measurable financial impact” [S92], and Stanford’s study of the other side found “The difference was never the AI model. It was always the organization” [S92]. The hard work is invisible: “77% of the hardest challenges were invisible and intangible costs: change management, data quality, and process redesign” [S92], which is exactly why a clean-sample demo performs well and then fails to produce impact.

The autonomy pattern sets ROI. Agentic implementations “showed 71% median productivity gains versus 40% for high-automation but represented only 20% of cases” [S92], and the oversight pattern is itself a lever: “Escalation-based models (AI handles 80%+ autonomously, humans review exceptions) delivered 71% median productivity gains versus 30% for approval models” [S92]. In an approval model the human is the bottleneck; in an escalation model human judgment is spent only on exceptions. Default to escalation, reserve approval for unrecoverable actions.

Teams that succeed reuse infrastructure rather than build from scratch. LinkedIn found “our messaging system best emulated the characteristics we wanted in a multi-agent orchestrator” [S91] and adopted then adapted LangGraph [S91]. Klarna runs the same stack at “over 85 million active users” [S94] and “Reduced average customer query resolution time by 80%” [S94]. Uber shows governance at scale, brokering “short-lived, scoped tokens for every hop” [S90] to shrink blast radius, with “P99 latency for the STS Token Exchange API… consistently below 40 milliseconds” [S90]. Adoption is now mainstream, “57.3% now have agents running in production” [S28], but “Quality remains the biggest barrier to production” [S28], and for coding agents that ceiling “is set by context, not model quality” [S93]: they “reliably do the visible 80% of a task and miss the invisible 20% that lives outside their context window” [S93].

Instruction and prompt design

Instruction design governs the durable text an agent re-reads on every one of many turns, where small wording choices compound. The organizing idea is altitude: prompts should present ideas “at the right altitude for the agent” [S8], neither hardcoding brittle logic nor gesturing vaguely. The target is “the minimal set of information that fully outlines your expected behavior” [S8], teaching the long tail with “diverse, canonical examples” [S8] rather than exhaustive edge-case lists. Persistent files move this into the repository: AGENTS.md is “A simple, open format for guiding coding agents, used by over 60k open-source projects” [S108] with clear precedence, “The closest AGENTS.md to the edited file wins; explicit user chat prompts override everything” [S108], and the failure mode is length, since “Bloated CLAUDE.md files cause Claude to ignore your actual instructions!” [S83]. Anything that must happen belongs in code, because “hooks are deterministic and guarantee the action happens” [S83] while prose instructions are advisory.

The empirical evidence complicates this. A factorial study targeting the exact beliefs practitioners hold about file size, position, and structure found a null result: “None of the four structural variables or three two-way interactions produces a detectable contrast after multiple-testing correction” [S112]. What did matter was time-in-run: “each additional function the agent generates is associated with approximately 5.6% lower odds of compliance per step” [S112]. The resolution is not that instruction design is useless, altitude and precedence remain sound, but that rearranging the file is low-yield while adherence decays as the run grows. The lever that matters is shorter sessions and re-anchored instructions rather than section order.

Discussion: the open tensions

The corpus converges on a great deal, but three tensions remain open.

Multi-agent skepticism versus the measured win. The read-versus-write rule reconciles the 2025 debate: parallelize reads, serialize writes [S17]. But the strongest pro-multi-agent number, the 90.2 percent gain [S16], and the token multipliers are self-reported on an internal eval, and Cognition’s 2026 reversal from “don’t build” to “map-reduce-and-manage” [S13] is recent and single-vendor. The honest position is that multi-agent helps when the work decomposes into independent, context-hungry reads and hurts whenever independent agents make conflicting write decisions, and that the benefit comes from parallel context windows rather than additional model reasoning. Anyone reaching for a swarm before exhausting a single agent with tools [S2] is likely paying coordination cost for no read-parallelism benefit.

Reliability and the durable-execution gap. There is strong agreement that reliability is architectural and multi-dimensional [S68][S67] and that durable execution beats bare checkpointing [S25][S96][S100]. The unresolved edge is that durable execution was not built for non-deterministic replay. ACRFence and Atomix [S98][S97] both show that LLM re-synthesis breaks the idempotency assumption underneath replay-and-retry, turning a correctness convenience into a security hazard (duplicate payments, reused credentials). These mitigations are single, recent academic results whose real-world overhead and adoption are not yet independently replicated, and the vendor framing of “checkpoints are not durable execution” [S25] comes from a party selling a durable runtime. The safe reading is to gate irreversible effects behind replay-or-fork today and watch this area, because the primitives are not settled.

The evaluation gap. This is the least comfortable tension in the survey. The field grades agents with tools that agree with humans at chance level (substring judging at kappa 0.049 [S60]) or with LLM judges that are repeatable yet biased [S58], on benchmarks whose scores partly reflect which model played the user [S61] and which can be gamed by an agent that searches for the answer key [S57]. Two of the most quotable numbers, the substring kappa and the 0.62 cascade probability [S60], come from a single-author preprint. If evaluation is this shaky, then every self-reported gain in the corpus inherits that uncertainty, which is why confidence throughout should be read as directional for vendor numbers and single studies, and why reading transcripts and grading intermediate steps [S57][S60] is the only defensible practice.

The meta-principle ties the survey together. The durable work of an AI engineer is harness engineering rather than model selection; it “doesn’t shrink as models improve. Instead, it moves” [S79]. Context, tools, memory, control, and containment are where the design decisions that matter get made, and they are the same whether the base model is this year’s or next year’s.

References