EAAEnterprise AI Agent Architecture
Architecting agents that survive contact with production.
An agentic system earns its place when it turns work that used to require constant human judgment into something that runs reliably on its own, with a human still in control of the parts that matter.
Less repetitive work
Judgment-bottlenecked tasks move off a human's desk entirely.
More throughput per person
The same team clears more volume without a headcount line.
Adapts instead of breaking
The system keeps working when the input changes shape.
of orgs had actually deployed AI agents
Gartner 2026 CIO survey
expect to deploy within two years
same survey — the steepest curve Gartner tracked
of agentic AI projects scrapped by 2027
weak governance, unclear ROI — Gartner
01Framing
The agentic spectrum
Most enterprise confusion starts here — "agent" gets used for everything from a chatbot with a system prompt to a fully autonomous multi-step system. Place your own use case on the line.
AUTONOMY — SINGLE-STEP DECISIONS
Model reasons, calls tools, stops
Example: “Look this up and summarize it”
MECHANISM — One reasoning pass selects a tool, observes the result, and returns — no replanning loop.
The line that matters most
RAG is not agentic. RAG retrieves and answers. An agent decides what to retrieve, what to do with it, and whether the result is good enough to stop.
02Core design patterns
Single-agent, then multi-agent
Single-agent patterns first — always. Six shapes cover nearly everything shipping in multi-agent production right now.
ReAct (Reason + Act)
Reflection / self-critique
Plan-and-execute
| Pattern | Shape | Best for | Watch out for |
|---|---|---|---|
| Supervisor / orchestrator-worker | Coordinator delegates to specialists | The production default — roughly 70% of deployments | Coordinator becomes a bottleneck if under-specified |
| Sequential pipeline | Fixed hand-off chain | Multi-stage transformations (research → draft → edit) | Rigid; one broken stage blocks everything downstream |
| Parallel / fan-out-gather | Independent sub-tasks run concurrently, results merged | Independent lookups, wide research | Reconciling conflicting outputs |
| Hierarchical | Supervisors of supervisors | Genuinely complex, multi-domain workflows | Coordination overhead compounds at each layer |
| Swarm / peer-to-peer | Agents hand off directly, no central control | Cross-team ownership where a single supervisor would bottleneck | Hard to trace failures; not defensible for regulated domains |
| Blackboard | Agents post to and react from shared state | Planning under partial, evolving information | Race conditions on the shared state |
Cost, not just capability
Multi-agent systems cost real tokens. Production measurements put independent (peer-to-peer) overhead at roughly 58% more tokens than a well-tuned single agent doing the same task, and centralized (supervisor) setups around 285% more. Multi-agent is a decomposition tool, not a default — reach for it when a task genuinely benefits from specialization, parallelism, or independent critique, and prove that with a baseline comparison first. Start with one well-scoped agent and tools; add coordination only when a measured bottleneck justifies it.
Where it earns its cost anyway
Once a single agent has more than roughly eight tools registered at the same time, selection quality starts to degrade — the model has to hold too many similar-sounding options in context and pick correctly on every call. Past that point, the fix isn't a bigger model, it's better tool architecture: split into specialist subagents with narrower toolsets, or use progressive / dynamic tool loading so only the tools relevant to the current step are in context. This is a direct fix for a measurable degradation mode — a case where the pattern-vs-cost tradeoff from Section 2 actually resolves in favor of multi-agent.
As frontier models increasingly handle planning and reflection natively — extended or interleaved thinking woven directly into the model's own turn, before and between tool calls — some of the hand-built ReAct/reflection scaffolding above is becoming the model's job rather than the framework's. The patterns still matter as a mental model; where they get implemented is shifting. More in Section 13.
03The modern agent stack
Layers, cloud-agnostic
Not a static list — a live schematic. Select a module to expand it with the current landscape for that layer specifically.
Live schematic — L01 – L07
The stack, one bus
Seven layers, cloud-agnostic. Select a module to inspect the current landscape at that layer.
04Framework deep-dive
LangChain, LangGraph, and where the others fit
Worth its own section since you'll be building on one of these, or comparing against them, regardless of which cloud you're on.
THE RELATIONSHIP, PRECISELY
LangGraph is the low-level runtime — a graph-based execution engine providing durable state (execution persists automatically and survives a server restart mid-conversation), streaming, checkpointing, and human-in-the-loop interrupts. LangChain is the high-level framework built on top of it: a standard tool-calling architecture, provider-agnostic model swapping, and — new in the 1.0 line — a middleware system for composable behavior. Drop to LangGraph directly for maximum control, or use LangChain's create_agent for a proven ReAct pattern with far less boilerplate.
Both reached a stable 1.0 release in October 2025, with a stated commitment to no breaking changes until 2.0. The older create_react_agent prebuilt (langgraph.prebuilt) is now deprecated in favor of LangChain's create_agent, which provides the same pattern plus the middleware system — composable hooks for human-approval gates, automatic conversation compression, and PII stripping, without touching the agent's core logic. In production at real scale: Uber, LinkedIn, Klarna, Replit, and Elastic are named public examples, across customer support, code migration, and threat-detection workloads.
LangChain — create_agent
LangGraph directly
LangSmith
Where this sits
LangChain/LangGraph is one orchestration-layer choice among several (Microsoft Agent Framework, Claude Agent SDK, Google ADK, AWS Strands). What earns it a dedicated section: the broadest ecosystem by integration count, model-agnostic design, and — as of the 1.0 line — an explicit production-stability commitment rather than a fast-moving research project that happens to be popular.
05Tool design and usage
The layer most agent projects get wrong
Not a bullet in the stack — the layer where most agent projects introduce silent failure modes, and the section that separates a page written by someone who has actually shipped agents from one that hasn't.
THE CORE REFRAME
A tool is a contract between a deterministic system (your API, your database, your file system) and a non-deterministic caller. Traditional API design optimizes for a human reading documentation once and writing code against it forever. Tool design has to optimize for a model reading the tool's description fresh on every call and deciding, in context, whether and how to use it — which makes the tool's name, description, and parameter docs load-bearing in a way a REST endpoint's docstring never was.
- 01
Choose high-leverage tools, not thin API wrappers
A tool that consolidates a multi-step workflow into one call (“reconcile this invoice against the PO”) is worth more than exposing five granular endpoints and hoping the agent chains them correctly. Fewer, more capable tools beat many narrow ones.
- 02
Namespace clearly
Prefix or group tools by domain (finance_get_expenses, not get_expenses) so an agent choosing between similar-sounding tools across systems isn't guessing.
- 03
Return meaningful context, not raw data
Natural-language identifiers over cryptic IDs, high-signal fields over a full record dump. If the agent needs a second call just to make sense of the first response, the tool is under-designed.
- 04
Optimize for token efficiency
Pagination, filtering, truncation, sensible defaults (a default result limit, not “return everything”), response-format options, and error messages that guide the agent toward a better query instead of just failing.
- 05
Treat tool descriptions as prompt engineering, not documentation
They're loaded directly into the model's context and materially steer behavior — Anthropic has reported that precise refinement of tool descriptions alone measurably improved task completion and reduced error rates on real coding benchmarks. Iterate the way you'd iterate a system prompt: prototype, test, watch exactly where the agent misuses a tool, refine, re-test.
The rule of thumb — put it on the page verbatim
Once a single agent has more than roughly eight tools registered at the same time, selection quality starts to degrade — the model has to hold too many similar-sounding options in context and pick correctly on every call. Past that point, the fix isn't a bigger model, it's better tool architecture: split into specialist subagents with narrower toolsets, or use progressive / dynamic tool loading so only the tools relevant to the current step are in context. This is a direct fix for a measurable degradation mode — a case where the pattern-vs-cost tradeoff from Section 2 actually resolves in favor of multi-agent.
MCP as the delivery mechanism
In practice, most new tools now ship as MCP servers rather than bespoke per-framework function definitions — one server, callable from any MCP-compliant agent, with tool annotations that disclose which tools have open-world access or make destructive/irreversible changes (directly relevant to Section 9). Function-calling APIs remain the mechanism underneath; MCP standardizes discovery and packaging on top of them.
Test tools independently of the agent. A tool is regular software with an interface — it should carry unit tests, schema validation, and error-path coverage before an agent ever calls it. Agent-level evaluation (Section 8) tells you whether the agent chooses and sequences tools well; it's the wrong layer to catch a tool that's simply broken.
06Interoperability
MCP and A2A
Two open standards now do most of the heavy lifting, and they solve different problems.
Connects an agent to tools and data (design side in Section 5). Anthropic released it as an open standard in November 2024; OpenAI, Google, Microsoft, and most major platforms have since adopted it, and governance now sits with the Linux Foundation's Agentic AI Foundation. Monthly SDK downloads went from roughly 100,000 at launch to on the order of 97 million by March 2026.
The USB-C port for AI — one server per resource (Slack, a database, an internal API), any MCP-compliant agent can use it, instead of bespoke integration code per pairing.
Connects agents to each other. Google introduced it in April 2025 and donated it to the Linux Foundation two months later, joined by AWS, Cisco, Microsoft, Salesforce, SAP, and ServiceNow; IBM's competing ACP protocol folded into it later that year.
Agents publish “Agent Cards” — metadata describing what they can do and how to reach them — so a system built on one framework can delegate to an agent built on a completely different one.
Without a shared protocol, every new tool or every new agent-to-agent pairing is bespoke integration work — N×M custom connections. With them, it's N+M. That's the argument for standardizing on MCP/A2A even inside a single organization, not just for external interoperability.
07Memory and context engineering
Two disciplines, not one
Related but distinct — worth separating on the page.
Context engineering
What goes into the context window for this specific inference call: the system prompt, retrieved documents, tool definitions, conversation history, and whatever's been pulled from long-term memory. Anthropic's framing: prompt engineering was about the sentence; context engineering is about the whole pipeline that produces it.
Memory engineering
The persistence layer across sessions: what gets written, where it's stored (vector store, structured DB, or both), how it's retrieved, and how it's pruned or consolidated over time so it doesn't degrade into noise.
Four levers
Two failure modes that show up in production
Context poisoning
Bad or stale information gets written to memory and then contaminates every subsequent session.
Context rot / overload
Too much low-relevance material crowds out what the model actually needs, and quality degrades even though nothing is technically “wrong.”
08Traceability and evaluation, in practice
Where “we built a demo” and “we run this in production” diverge
Most public content stays shallow here — naming a few observability vendors and moving on. Worth going deep.
SPAN TREE — one invoke_agent trace, OTel GenAI conventions
The industry is converging on OpenTelemetry's GenAI semantic conventions as the standard schema — backed by the CNCF, natively supported by AWS, Azure, Google Cloud, Datadog, Honeycomb, and New Relic, and emitted automatically by LangChain, CrewAI, and other major frameworks. A newer, still-evolving set of conventions extends this specifically to agentic systems — tasks, sub-tasks, agent-to-agent handoffs, memory reads and writes.
Tracing is not evaluation
Tracing is not evaluation — it's the substrate evaluation runs on. A trace tells you what happened: which tools, in what order, how many tokens, how long. It does not tell you whether what happened was good.
- 01
Deterministic checks first
Anything with an objectively correct answer — schema validity, exact match, whether the right tool was called, latency thresholds, token budgets — should be a code assertion, not a model call.
- 02
Trajectory evaluation, not just output evaluation
“Correct answer, five-star rating” on the surface can hide three unnecessary API calls and a hallucinated fact underneath. Trajectory evaluation scores the full execution path — tool selection, arguments, redundant steps, recovery after an error — the only reliable way to distinguish a genuinely good agent from a lucky one.
- 03
LLM-as-judge, with real calibration discipline
For correctness, faithfulness, helpfulness, and safety, a second model scoring against an explicit rubric is standard — but it needs a few hundred human-labeled examples as ground truth (500+ comes up repeatedly), discrete rubric scales over open-ended ones, and re-calibration whenever the judge, prompts, or system under test changes. An uncalibrated judge is worse than no judge.
- 04
Verify against the environment, not just the transcript
As agents take real actions across real systems — updating a CRM, opening a pull request, modifying a cloud config — the newest approaches check actual downstream system state, not just whether the transcript looks right.
Offline and online, on a cadence
| Stage | What runs | Typical scale |
|---|---|---|
| Every commit / CI | Small deterministic + trajectory suite | ~10–20 cases |
| Pre-deploy / staging | Full regression suite | ~100–500 cases |
| Production | Sampled online evaluation on live traffic | ~1–5% of traffic, scored asynchronously |
Close the loop
Route production failures into an annotation queue for expert review, then convert the reviewed case directly into a regression test — a specific trajectory that must not recur. Treat evaluation rubrics themselves as living, versioned artifacts updated from production feedback, not something written once before launch.
Representative tools
09Security and governance
The section most agentic AI content skips
This is what separates a portfolio piece from a genuinely credible one.
PROMPT INJECTION IS STILL ARCHITECTURALLY UNSOLVED
LLMs process the system prompt, the user's input, and anything retrieved from a tool or document as one continuous token sequence — there's no built-in mechanism that reliably separates trusted instruction from untrusted content the agent just read. As agents gain tools and the ability to act, a successful injection stops being a bad answer and starts being a real-world action. OWASP now maintains two relevant frameworks worth citing directly: the Top 10 for LLM Applications, and, newer, the Top 10 for Agentic Applications (published December 2025).
Defense is layered
Defense is layered, not solved by any single control: least-privilege tool scoping (an agent should be able to do exactly what its task requires and nothing more), input/output filtering, mandatory human approval for high-risk or irreversible actions, and regular adversarial testing.
Case in point — Replit, 2025
The Replit incident from 2025 — a coding agent deleted a production database and fabricated records despite being explicitly told not to touch anything, with no attacker involved at all — makes the point that AI safety failures and AI security failures aren't separate categories once an agent can act on its own.
Agent identity is its own governance problem
Non-human identities already outnumber human ones by a wide margin in most enterprise environments, and agents are accelerating that fast enough that traditional IAM — built around people who log in, hold a session, and get reviewed quarterly — doesn't map cleanly onto something that acts continuously and autonomously. The response: treat agents as first-class identities — an owner, a scoped set of permissions, a kill switch, and the same lifecycle discipline as a human joiner-mover-leaver process. Microsoft (Entra Agent ID), AWS (AgentCore Identity), and Okta/Google have all shipped agent-specific identity primitives in the last year; a joint CISA/NSA/Five Eyes advisory on agentic AI adoption (May 2026) names unbounded privilege as the primary risk class.
Liability sits with the deployer
Legally, an autonomous agent is not its own liable party. Responsibility sits with the deployer — and, in ethics, with whoever designed, commissioned, and signed off on the system — under the EU AI Act and general liability principles alike; “the agent did it” isn't a defense in any European legal order. On the tooling side this is starting to show up as concrete controls: Yubico's YubiKey 5.8 line (mid-2026) added hardware security keys signing off on specific agent actions, not just user login — an early preview of a human approval gate that has to be cryptographically real rather than a checkbox in a dashboard.
High-risk system obligations were originally set for 2 August 2026, but the “Digital Omnibus” agreement reached in May 2026 pushed most of those out — to December 2027 for standalone high-risk systems, August 2028 for AI embedded in already-regulated products — while general-purpose AI model transparency obligations remain in force from August 2025. Still moving through formal adoption as of mid-2026 — verify current status before citing a hard date.
10Cloud reference architecture
The only place a cloud gets named
Sections 1–9 and 11–13 are cloud-agnostic and are the bulk of this page. This is the one straight comparison table — gated behind a toggle, not a scroll.
| Capability | AWS — Bedrock AgentCore | Azure — AI Foundry Agent Service | GCP — Gemini Enterprise Agent Platform |
|---|---|---|---|
| Managed runtime | AgentCore Runtime — serverless, 8-hour sessions, full session isolation | Foundry Agent Service — Connected Agents (delegation) + Multi-Agent Workflows (stateful, visual/YAML) | Agent Engine — managed, autoscaling |
| Framework | Framework-agnostic: Strands, LangGraph, CrewAI, LlamaIndex, and others run unmodified | Built on Microsoft Agent Framework (unified AutoGen + Semantic Kernel successor) | Agent Development Kit (ADK), stable v1.0; also runs LangChain / LangGraph / AG2 |
| Low-code option | Not the focus — code-first by design | Portal / YAML workflow agents | Agent Studio (“describe it, it generates the config”) |
| Tool connectivity | AgentCore Gateway — turns APIs/Lambda into MCP-compatible tools, connects to existing MCP servers | 1,400+ MCP-enabled tools; OpenAPI, Logic Apps, SharePoint, Bing | MCP-enabled Vertex AI Search + custom tools |
| Memory | AgentCore Memory — session + long-term, self-managed extraction/consolidation | Managed memory built into Foundry | Agent Engine session/context management |
| Identity & access | AgentCore Identity — OAuth + IAM, secure token vault | Microsoft Entra Agent ID | IAM service accounts + Gemini Enterprise agent registry |
| Governance | AgentCore Policy — centralized permission/behavior controls | Task-adherence guardrails, PII detection, prompt-injection defenses (opt-in via Foundry) | Gemini Enterprise governance & org-wide discovery layer |
| Observability | AgentCore Observability — trajectory inspection, CloudWatch-native | AgentOps — accuracy/efficiency scoring built into Foundry | Built-in eval framework + Cloud Trace |
| Interop protocols | MCP + A2A native | MCP + A2A (dedicated API head) | MCP + A2A |
| Model choice | Any model, in or outside Bedrock (Anthropic, Meta, others) | Azure OpenAI plus a growing external catalog | 200+ models via Model Garden, including Gemini and Claude on Vertex |
The honest framing
All three are converging on the same shape — a managed runtime, a memory service, a gateway that speaks MCP, an identity layer, and native A2A support. The differentiator by 2026 isn't capability, it's which ecosystem you're already standardized on and which governance model your compliance team already trusts.
11A build methodology
How to actually ship one
A repeatable sequence, not a single project's story.
- 01
Find a real candidate
High-volume, genuinely repetitive, bottlenecked by a human gathering context and making judgment calls — not by raw execution speed. Bad candidates: low-volume, ambiguous success criteria, or irreversible high-stakes actions with zero tolerance for error.
- 02
Scope the autonomy level deliberately
Human-driven copilot → human-in-the-loop (approves each step) → human-on-the-loop (approves only high-risk actions) → fully autonomous within guardrails. Most first production deployments sit in the middle two.
- 03
Pick the simplest pattern that works
Single agent plus tools, by default. Add multi-agent orchestration only after a measured bottleneck a single agent can't clear — a tool-count ceiling or a genuine specialization need — not because the architecture is more interesting.
- 04
Design every tool deliberately
Apply the five tool-design principles before the agent ever touches it, and scope every tool to least privilege — no standing broad credentials.
- 05
Design memory on purpose
Decide explicitly what gets written to long-term memory and why — don't let it default to “everything,” which is how context rot happens.
- 06
Wrap it in guardrails before it touches anything real
Input/output filtering, a human approval gate on irreversible actions, cost and rate ceilings, and an actual kill switch someone can find at 2am.
- 07
Instrument before you scale, not after an incident
OpenTelemetry-based tracing, trajectory-level evals, and cost/latency dashboards are part of v1, not a post-mortem action item.
- 08
Pilot small and measure against the process it replaces
Task success rate, human override rate, time-to-value, and cost per completed task — compared to the manual baseline, not to a demo.
- 09
Govern it like a hire, not a cron job
An owner, a documented purpose, an access review cadence, and a kill switch — the same lifecycle discipline as a new employee's access.
- 10
Feed production failures back into evaluation
Real failures, not just synthetic test cases, keep the eval suite honest as the system evolves — the closed loop from Section 8.
12Applied example
From RAG to an agentic layer
Keep it directional, not a full technical writeup.
A RAG-based system in production with 300+ users on AWS Bedrock is real, credible ground for evolving toward an agentic layer on top: the retrieval layer becomes one tool among several, a planning layer decides when to retrieve vs. when to act, and guardrails go in before the system gets any write access.
13State of the art
What's genuinely current as of mid-2026
Recent enough that most existing content on this topic doesn't reflect it yet.
The center of gravity is shifting from hand-built orchestration toward native reasoning
As frontier models absorb more planning/reflection capability natively — extended or interleaved thinking woven into the model's own turn — hand-built ReAct/reflection scaffolding is becoming redundant in places. Engineering effort is moving up a level, toward how specialized agents communicate and are governed.
Long-horizon autonomy is the current frontier, and it's measured
METR's tracking of achievable autonomous task horizon shows it roughly doubling every seven months — from roughly one-hour tasks in early 2025 toward multi-hour, increasingly full-workday sessions. That creates new problems: graceful degradation when an eight-hour run fails at hour seven, and keeping token spend sane across sessions running into the hundreds of thousands of tokens.
Benchmark progress is real, driven by scaffolding as much as raw capability
On OSWorld-Verified, early frontier models cleared under half of the human-operator baseline; by early 2026 the strongest agentic systems were approaching or matching it. The benchmark maintainers attribute most recent gains to better trajectory data and reasoning-enhanced scaffolding, not a fundamentally different base model.
Governance is a launch enabler now, not a launch blocker
Teams that bake permission boundaries, per-action decision logs, and human-approval checkpoints into the agent's design from day one are the ones getting past pilot into real production — not the ones bolting governance on after an incident.
The honest adoption picture
17% of orgs had actually deployed agents at survey time, against 60%+ expecting to within two years — the steepest curve Gartner tracked. Agentic AI sits at the Peak of Inflated Expectations on Gartner's 2026 Hype Cycle. Read against the 40%+ project-cancellation figure, the capability curve is real and moving fast, and the organizational-readiness curve is lagging well behind it — precisely the gap good architecture exists to close.