In 2023, the stack was simple: pick a Large Language Model (LLM), wrap it in a chat User Interface (UI), ship. In 2026, production artificial intelligence (AI) looks like a five-layer cake:
Model → Agent → MCP → Tools → Infrastructure
The model reasons. The agent plans, remembers, and enforces policy. Model Context Protocol (MCP) standardizes how the agent reaches the outside world. Tools are the actual systems of record: GitHub, Jira, Customer Relationship Management (CRM), databases, browsers. Infrastructure is where it all runs: gateways, identity, sandboxes, Kubernetes (K8s), observability, and horizontally scaled MCP servers (Model Context Protocol, n.d.; TrueFoundry, 2026; Microsoft, 2026).
Teams that treat "we use GPT" as the whole architecture build demos. Teams that map all five layers build systems that survive contact with enterprise reality.
Key Abbreviations in This Post
- LLM (Large Language Model): A neural network trained on text for language understanding and generation.
- SLM (Small Language Model): A compact model optimized for on-device or low-cost inference.
- MCP (Model Context Protocol): Open standard connecting AI hosts to tools, resources, and prompts (Model Context Protocol, n.d.).
- API (Application Programming Interface): Programmatic access to software systems.
- JSON-RPC (JSON Remote Procedure Call): Protocol MCP uses for host-server communication.
- A2A (Agent-to-Agent): Google's protocol for agent discovery and delegation (Google, n.d.).
- CRM (Customer Relationship Management): Software for customer records and sales workflows.
- IAM (Identity and Access Management): Systems governing who or what can access resources.
- NHI (Non-Human Identity): Credentials used by service accounts, agents, and automation.
- K8s (Kubernetes): Container orchestration platform for scaling services.
- SEP (Specification Enhancement Proposal): Formal MCP spec change document.
The One-Minute Version
- Model: The reasoning engine. Increasingly swappable and partially commoditized.
- Agent: The product layer. Harness, memory, orchestration, discovery, governance.
- MCP: The standard wire format between agents and capabilities. Stateless as of July 2026 (Model Context Protocol, 2026).
- Tools: Real integrations that mutate state in your organization.
- Infrastructure: Identity, scaling, audit, sandboxes, and policy enforcement outside the model.
- Moat in 2026: Layers 2 through 5, not layer 1 (TrueFoundry, 2026; Knowlee, 2026).
Old Stack vs New Stack
| Era | Stack | What broke |
|---|---|---|
| 2023 chatbot | Model + UI | Human copy-paste integration, no memory, no audit |
| 2024 copilot | Model + plugins/RAG | Custom glue per integration, tool sprawl in prompt |
| 2026 agent platform | Model + Agent + MCP + Tools + Infra | Complex, but composable and governable |
Layer 1: Model (The Engine, Not the Product)
The model is where most conversations start and where least durable advantage lives in 2026. GPT-6 Astra, Claude Opus-class, Gemini, Phi-4-reasoning, local Small Language Models (SLMs) on laptop Neural Processing Units (NPUs): all are interchangeable inputs to the same agent harness (OpenAI, 2026; Microsoft Research, 2025; Chandra & Krishnamoorthi, 2026).
What the model provides:
- Language understanding and generation.
- Reasoning over context (quality varies by tier).
- Tool-call intent: which function to invoke with what arguments.
What the model does not provide:
- Persistent memory across sessions (unless the agent layer adds it).
- Reliable multi-step workflow execution.
- Authentication, authorization, or audit trails.
- Guaranteed correct tool selection from 400 similar schemas.
Smart teams pick models by task routing: frontier cloud for hard reasoning, local SLM for high-volume classification, mid-tier for cost-sensitive agent loops (Chandra & Krishnamoorthi, 2026). The model is the engine. You do not sell the engine. You sell the car.
Layer 2: Agent (Where the Product Lives)
An agent is not a model with a system prompt. It is a runtime that wraps the model in a loop:
- Observe context (user message, memory, tool results).
- Reason and plan next action.
- Call tools or delegate to sub-agents.
- Update state, retry on failure, escalate to humans.
- Repeat until done or bounded.
The agent layer includes components that separate demos from production (MindStudio, n.d.; Agent Operating System, 2026):
- Harness: Retry logic, context management, subagent spawning, evaluators (OpenAI, 2026).
- Memory: Session, user, and org knowledge that survives turns and tabs.
- Orchestration: LangGraph-style workflows with explicit state, not hidden prompt chains (TrueFoundry, 2026).
- Tool discovery: Progressive loading, semantic routing, MCP-Zero-style catalogs instead of dumping 2,800 schemas into context (Xu et al., 2025; Spring AI, n.d.).
- Governance: Human approval gates, policy checks, misalignment monitoring (OpenAI, 2026; OWASP GenAI Security Project, 2025).
Microsoft's Windows agent runtime, OpenAI Codex, and Amazon Bedrock AgentCore all sit at this layer. The model inside can change. The harness is what users and enterprises actually depend on (Microsoft, 2026; Eluna, 2026).
Layer 3: MCP (The Standard Wire)
Before MCP, every agent platform invented its own plugin format. Salesforce had one shape. GitHub Copilot another. Internal platforms a third. MCP replaces that fragmentation with a shared protocol (Model Context Protocol, n.d.).
Three roles:
- Host: The AI application (Cursor, Claude Desktop, Copilot, your agent platform).
- Client: One JSON-RPC client per server connection, inside the host.
- Server: Your integration exposing tools, resources, and prompts.
MCP servers expose:
- Tools: Callable functions (
create_ticket,run_query). - Resources: Readable data (files, database views).
- Prompts: Reusable templates.
The July 2026 spec made remote MCP stateless: no session handshake, no sticky pods. Each HTTP request carries client identity in _meta. Gateways read Mcp-Method and Mcp-Name headers for routing (Model Context Protocol, 2026). That turns MCP from a local dev trick into production infrastructure behind load balancers.
MCP is not the only protocol. Google's Agent-to-Agent (A2A) handles delegation between agents. Many teams use both: MCP for tools, A2A for multi-agent handoffs (Google, n.d.). Think of MCP as USB-C for capabilities. A2A is the office phone system between departments.
Layer 4: Tools (Where State Actually Changes)
Tools are the reason agents matter. A tool call creates a Jira ticket, charges a card, deletes a file, merges a pull request. This is where blast radius lives (Cloud Security Alliance, 2026).
Tools arrive through several patterns in 2026:
- MCP servers: Packaged integrations (official GitHub MCP, internal CRM MCP).
- Agent Connectors: Microsoft Windows on-device registry for File Explorer, Settings, and third-party connectors (Dave R, 2026).
- Native computer use: GPT-6 Astra driving browser and desktop directly, bypassing formal tool schemas for some tasks (OpenAI, 2026).
- Legacy APIs: Wrapped behind MCP servers so agents never touch raw REST (Representational State Transfer) diversity.
Tool design principles that survive production:
- Small surface area. Prefer
search_issuesandupdate_issueover one mega-tool that does everything. - Explicit schemas. Clear names reduce wrong-tool errors when catalogs grow (Spring AI, n.d.).
- Idempotent where possible. Agents retry. Non-idempotent tools need guardrails.
- Backend enforcement. Never rely on the model to respect "do not delete" in the system prompt (Cloud Security Alliance, 2026).
MCPTox showed 72.8% success poisoning tool metadata on real MCP servers. Tool definitions are part of your attack surface, not documentation (MCPTox, 2026).
Layer 5: Infrastructure (Where Trust Is Enforced)
Infrastructure is everything that must work even when the model is wrong. It sits below MCP and tools and enforces reality (Agent Operating System, 2026; Microsoft, 2026):
- Identity and access. Dedicated Non-Human Identity (NHI) per agent, just-in-time credentials, tested revocation (Cloud Security Alliance, 2026; IDC, 2026).
- API gateways. Rate limits, auth, routing for stateless MCP replicas on Kubernetes, Cloud Run, or Lambda.
- Sandboxing. Microsoft Execution Containers (MXC), agent workspaces, cloud PCs for isolated execution (Microsoft, 2026).
- Observability. Tool-call logging with principal, arguments, and outcome to a sink the agent cannot modify (Cloud Security Alliance, 2026).
- Policy engines. Block destructive actions outside declared scope regardless of model intent.
- Data plane. Databases, queues, and secrets stores where tool side effects land.
Eluna runs production warehouse agents on Amazon Bedrock AgentCore with graph-guided Standard Operating Procedure (SOP) execution. The model reasons. The infrastructure guarantees procedural compliance and auditability (Eluna, 2026). That split is the whole point of the five-layer model.
How a Request Flows Through the Stack
Example: "Refund order 4821 and notify the customer."
- Model parses intent and proposes a plan: lookup order, validate policy, issue refund, send email.
- Agent loads memory (prior interactions with this customer), checks approval policy (refunds over $500 need human gate), selects tools via discovery (not all 200 schemas).
- MCP routes
tools/callto the Stripe MCP server and the email MCP server with authenticated client identity in_meta. - Tools execute: Stripe API refund, CRM status update, SendGrid message.
- Infrastructure logs each call, enforces OAuth scopes, rate-limits the agent identity, and stores audit records for compliance.
If the model hallucinates a refund amount, infrastructure and tool backends catch it. If MCP metadata is poisoned, gateway inspection and schema pinning catch it. If the agent loops, orchestration checkpoints and budgets stop it. No single layer carries the full burden.
What Breaks When You Skip a Layer
| Missing layer | Symptom |
|---|---|
| No agent layer | Chatbot that cannot sustain multi-step work or recover from failures |
| No MCP (ad-hoc plugins) | N integrations = N custom wire formats, no shared discovery or audit |
| No real tools | Agent generates text about actions but nothing in systems of record changes |
| No infrastructure | Prompt injection becomes data breach; no identity, scale, or kill switch |
| Model-only focus | Benchmark wins, production losses when tool catalog exceeds context |
Real-World Use Cases (Problem → Cause → Effect)
1. IDE assistant that only swapped models
Problem: Team upgraded from GPT-5.6 Sol to GPT-6 Astra expecting 2x productivity. Gains were marginal.
Cause: Harness was unchanged: no parallel sessions, no git worktree isolation, no context notes across windows. Model swap without agent layer investment (OpenAI, 2026; Microsoft, 2026).
Effect: GitHub Copilot app with agent-native harness (parallel sessions, CI babysitting) captured the value. Model was necessary; harness was sufficient.
2. Enterprise MCP behind a load balancer
Problem: Remote MCP worked on localhost, failed in Kubernetes with session errors on the second request.
Cause: Pre-2026 stateful MCP pinned clients to pods. Sticky sessions were fragile (Model Context Protocol, 2026).
Effect: Stateless MCP (July 2026 spec) let any replica serve any request. Infrastructure layer finally matched REST-style deployment patterns.
3. Support agent with 400 tools in prompt
Problem: Agent picked wrong Jira action 30% of the time. Context window half-consumed before user spoke.
Cause: No discovery layer. Model layer alone cannot navigate large catalogs (Spring AI, n.d.; Xu et al., 2025).
Effect: Progressive MCP discovery cut initial context 70 to 90% and improved selection accuracy. Agent layer fix, not model fix.
4. "Local AI" app that synced everything to cloud
Problem: Marketing claimed on-device inference. Security review found customer data leaving the device.
Cause: Model ran locally; agent layer still posted outputs to a cloud analytics endpoint (Beagle, 2026).
Effect: Stack audit required tracing all five layers. Privacy lives in infrastructure and agent design, not model location alone.
Build Order for Teams Starting Today
- Pick one bounded workflow. Refund processing, ticket triage, code review. Not "general assistant."
- Wrap it in an agent harness. Explicit state, retries, human gates, memory scope.
- Expose tools via MCP. One server per domain (billing, CRM, email). Stateless HTTP for production.
- Implement backend enforcement on tools. Scopes, idempotency keys, rate limits.
- Add infrastructure before scale. Agent identity, gateway, logging, sandbox. OWASP ASI Top 10 as checklist (OWASP GenAI Security Project, 2025).
- Choose model last and route by task. Swap freely once layers 2 through 5 are solid.
The new AI stack is not a model picker. It is an architecture:
Model → Agent → MCP → Tools → Infrastructure
The model thinks. The agent acts with memory and policy. MCP connects. Tools mutate real state. Infrastructure enforces trust when the model is wrong, the tool is poisoned, or the agent loops too far.
In 2023, buying GPT access was the strategy. In 2026, the strategy is building layers 2 through 5 so well that the model becomes a replaceable part. The teams winning on agents are not winning on parameters. They are winning on stack discipline.
Map your system against all five layers. Fix the missing floor before you upgrade the engine.
References
- Agent Operating System. (2026). The agent operating system (AOS): A reference operating architecture for distributed agentic systems. arXiv. https://arxiv.org/abs/2608.03214
- Beagle. (2026). Understand Microsoft Aion 1.0 before you plan your deployment. https://www.heybeagle.com/blog/understand-microsoft-aion-1-0-before-you-plan-your-deployment
- Chandra, V., & Krishnamoorthi, R. (2026). On-device LLMs: State of the union, 2026. Meta. https://v-chandra.github.io/on-device-llms/
- Cloud Security Alliance. (2026). Autonomous by design, uncontrolled by default. https://labs.cloudsecurityalliance.org/wp-content/uploads/2026/09/CSA_whitepaper_agentic_ai_loss_of_control_20260908-csa-styled.pdf
- Dave R. (2026). Inside the Windows agent platform. ITNEXT. https://itnext.io/inside-the-windows-agent-platform-how-microsoft-turned-the-os-into-a-secure-runtime-for-1-3-a980ac1d58b0
- Eluna. (2026). Eluna: An agentic LLM system for automating warehouse operations. arXiv. https://arxiv.org/html/2607.08960v2
- Google. (n.d.). Agent2Agent (A2A) protocol. https://google.github.io/A2A/
- IDC. (2026). IDC research finds AI agents outpacing security teams. https://ittech-pulse.com/news/idc-research-finds-ai-agents-outpacing-security-teams-sponsored-by-guidepoint-security/
- Knowlee. (2026). The agentic operating system. https://www.knowlee.ai/blog/agentic-operating-system-business
- MCPTox. (2026). MCPTox: A benchmark for tool poisoning on real-world MCP servers. AAAI. https://ojs.aaai.org/index.php/AAAI/article/view/40895
- Microsoft. (2026). Microsoft Build 2026: Be yourself at work. https://blogs.microsoft.com/blog/2026/06/02/microsoft-build-2026-be-yourself-at-work/
- MindStudio. (n.d.). What is an agentic operating system? https://www.mindstudio.ai/blog/what-is-agentic-operating-system
- Model Context Protocol. (2026). Specification (2026-07-28 stateless update). https://modelcontextprotocol.io/
- Model Context Protocol. (n.d.). Architecture overview. https://modelcontextprotocol.io/docs/concepts/architecture
- OpenAI. (2026). GPT-6 Astra: A new generation of intelligence. https://openai.com/index/gpt-6-astra/
- OWASP GenAI Security Project. (2025). OWASP Top 10 for agentic applications 2026. https://genai.owasp.org/download/52117
- Microsoft Research. (2025). Phi-4-reasoning technical report. https://www.microsoft.com/en-us/research/publication/phi-4-reasoning-technical-report/
- Spring AI. (n.d.). Tool calling and MCP integration. https://docs.spring.io/spring-ai/reference/api/tools.html
- TrueFoundry. (2026). Agent orchestration in production. Referenced via industry deployment patterns.
- Xu, Y., et al. (2025). MCP-Zero: Proactive tool discovery for LLM agents. arXiv. https://arxiv.org/abs/2505.07874


