Stateless MCP: What Changed, Why It Matters, and How to Build for Scale

 

If you deployed a remote MCP server in 2025, you probably hit the same wall I did: it worked fine on localhost, then fell apart behind a load balancer. Request two landed on a different pod and came back with a session error. The fix was always some combination of sticky sessions, Redis, or a custom gateway. That was not a bug in your code. It was the protocol.

The 2026-07-28 Model Context Protocol (MCP) specification changes that. MCP is now stateless at the transport layer. Each Hypertext Transfer Protocol (HTTP) request carries everything the server needs. Any replica can answer it. For teams treating MCP as production infrastructure, not a local dev trick, this is the biggest shift since remote MCP launched.

Key abbreviations in this post

• API — Application Programming Interface
• AWS — Amazon Web Services
• HTTP — Hypertext Transfer Protocol
• JSON — JavaScript Object Notation
• JSON-RPC — JSON Remote Procedure Call
• K8s — Kubernetes
• LLM — Large Language Model
• MCP — Model Context Protocol
• MRTR — Multi Round-Trip Requests
• OAuth — Open Authorization
• OpenTelemetry — open standard for traces, metrics, and logs
• PEP — Python Enhancement Proposal
• REST — Representational State Transfer
• RFC — Request for Comments
• RPC — Remote Procedure Call
• SaaS — Software as a Service
• SDK — Software Development Kit
• SEP — Specification Enhancement Proposal
• SSE — Server-Sent Events
• SKU — Stock Keeping Unit
• stdio — standard input/output

MCP in 60 Seconds

The Model Context Protocol (MCP) is an open standard for connecting AI applications to external tools, data, and prompts. Think of it as a structured Application Programming Interface (API) layer built for Large Language Model (LLM) hosts rather than generic Representational State Transfer (REST) clients.

The architecture has three roles (Model Context Protocol, n.d.):

• Host — the AI app you actually use (Cursor, Claude Desktop, an internal agent platform).
• Client — one client per server connection, living inside the host. It speaks JSON Remote Procedure Call (JSON-RPC) 2.0 on behalf of the host.
• Server — your integration. It exposes tools (callable functions), resources (readable data), and prompts (templates).

Local servers usually run over stdio (standard input/output): the host spawns a subprocess and talks over stdin/stdout. Remote servers use Streamable HTTP (Hypertext Transfer Protocol) on a real port. That is the deployment path that needed statelessness.

That is the whole stack in brief. The rest of this post is about what changed when MCP stopped pretending every HTTP (Hypertext Transfer Protocol) deployment was a single long-lived conversation.

What "Stateless MCP" Actually Means

Before spec version 2026-07-28, a remote MCP client had to:

1. Send an initialize request with protocol version and capabilities.
2. Receive an Mcp-Session-Id header from the server.
3. Include that session ID on every later call, often over a held-open Server-Sent Events (SSE) stream.

The server kept session state in memory (or Redis). The client was pinned to whichever instance created that session. Scale-out required sticky routing and shared session stores (Model Context Protocol, 2026; Van Gent & Blount, 2026).

Stateless MCP removes that contract entirely. Each change below is tracked as a numbered Specification Enhancement Proposal (SEP), MCP's formal design document for spec changes (similar to a Python Enhancement Proposal (PEP) in Python or a Request for Comments (RFC) on the web):

• SEP-2575 (Specification Enhancement Proposal) retires the initialize / initialized handshake.
• SEP-2567 (Specification Enhancement Proposal) removes the Mcp-Session-Id header.
• Every request includes client identity and capabilities in a _meta object.
• Optional server/discover Remote Procedure Call (RPC) replaces "connect first, then ask what you can do."
• SEP-2243 (Specification Enhancement Proposal) puts Mcp-Method and Mcp-Name in HTTP (Hypertext Transfer Protocol) headers so gateways can route without parsing JSON (JavaScript Object Notation) bodies.

Working process step by step

1. The host decides to call a tool (for example, search_inventory with a Stock Keeping Unit (SKU)).
2. The MCP client builds a JSON Remote Procedure Call (JSON-RPC) tools/call payload and attaches _meta with protocol version, client name, and capabilities.
3. The client POSTs to /mcp with headers MCP-Protocol-Version, Mcp-Method, and Mcp-Name.
4. A load balancer forwards the request to any healthy server instance.
5. The server validates headers against the body, runs the tool handler, and returns a JSON response. No session lookup.
6. If the tool needs user input mid-flight, the server returns a Multi Round-Trip Requests (MRTR) inputRequired result with a requestState blob. The client retries on any instance with answers attached (Model Context Protocol, 2026).

One nuance worth stating clearly: stateless protocol does not forbid application state. If your workflow spans multiple tool calls, mint an explicit handle (a task ID, a draft ID) from the first tool and pass it back as an argument. That is visible to the model and survives load balancing. Hidden session state in the transport was the part that had to go.

Why We Needed This

MCP exploded as the default way to wire agents to Software as a Service (SaaS) APIs, databases, and internal services. Downloads across Tier 1 Software Development Kits (SDKs) crossed into the hundreds of millions per month (Model Context Protocol, 2026). Most of that growth pointed at remote HTTP (Hypertext Transfer Protocol) servers, not local stdio (standard input/output) subprocesses.

The old session model created real production pain (Van Gent & Blount, 2026; New Relic, 2026):

• Broken round-robin. Standard load balancers sent the second request to a pod that never saw the session.
• Sticky session tax. Affinity rules skew traffic and fight autoscaling.
• Fragile deploys. Rolling updates dropped in-memory sessions and surfaced transient 400 errors to active chats.
• Extra infrastructure. Teams bolted on Redis or gateway packet inspection just to keep MCP alive at scale.
• Gateway blind spots. Rate limits and audit rules could not key off method or tool name without parsing JSON (JavaScript Object Notation) bodies.

Stateless MCP makes remote servers behave like normal HTTP (Hypertext Transfer Protocol) microservices. That sounds obvious. For a protocol born on stdio (standard input/output), it is a necessary maturation step.

Stateful vs Stateless MCP

Aspect	Stateful MCP (pre-2026-07-28)	Stateless MCP (2026-07-28)
Connection setup	initialize handshake required	No handshake; optional server/discover
Session tracking	Mcp-Session-Id on every request	No protocol-level session ID
Client context	Sent once at connect time	Sent in _meta on every request
Load balancing	Sticky sessions or shared session store	Plain round-robin across replicas
Serverless / scale-to-zero	Poor fit; sessions die on cold start	Natural fit (Cloud Run, Lambda, Workers)
Gateway routing	Often requires JSON (JavaScript Object Notation) body inspection	Mcp-Method and Mcp-Name headers
Mid-call user prompts	Open bidirectional Server-Sent Events (SSE) stream	Multi Round-Trip Requests (MRTR) with requestState retry
Long-running work	Block connection or custom hacks	Tasks extension with poll-based status
Payload size trade-off	Smaller per-call payloads after handshake	Slightly larger requests (metadata repeated)
Best transport locally	stdio (standard input/output) still fine either way	stdio (standard input/output) unchanged; HTTP (Hypertext Transfer Protocol) gains the most

Real Use Cases

Stateless MCP is not academic. It unlocks patterns teams were already hacking together:

• Enterprise tool gateways. One MCP endpoint fronting dozens of internal APIs (Application Programming Interfaces), rate-limited per tool name at the edge (Microsoft Foundry, Amazon Web Services (AWS) Bedrock AgentCore, and others ship variations of this).
• Multi-tenant SaaS (Software as a Service) MCP. GitHub and similar providers removed Redis session stores after upgrading, cutting latency on every call (Van Gent & Blount, 2026).
• Global deployment. Any region, any replica, same request shape. No cross-region session replication.
• Agent fleets at scale. Thousands of concurrent agent sessions hitting the same tool server without affinity tuning.
• Serverless MCP. Spin down to zero between bursts; the next request is self-contained.
• Compliance-friendly auditing. Log Mcp-Name and auth subject per request without holding connection state.

Deployment Architecture

A production layout looks familiar if you have shipped REST (Representational State Transfer) APIs before. The protocol no longer fights you.

Benefits in this layout:

• Horizontal pod autoscaling on Central Processing Unit (CPU) or request rate, not session count.
• Zero-downtime deploys without draining sessions.
• Open Authorization (OAuth) at the gateway with Request for Comments (RFC) 9207 issuer validation (SEP-2468 (Specification Enhancement Proposal)).
• OpenTelemetry traces correlated per request, not per opaque session.
• ttlMs (time-to-live in milliseconds) and cacheScope on list responses (SEP-2549 (Specification Enhancement Proposal)) so clients cache tool catalogs safely.

Example: Stateless MCP Server in Python

The v2 Python Software Development Kit (SDK) targets spec 2026-07-28. The high-level Application Programming Interface (API) is MCPServer. Transport options belong on run(), not the constructor (Model Context Protocol, n.d.-b).

# server.py — stateless inventory MCP server
from mcp.server import MCPServer

mcp = MCPServer(
    "InventoryTools",
    instructions="Look up product stock by SKU.",
)

@mcp.tool()
def search_inventory(sku: str) -> str:
    """Return stock level for a warehouse SKU."""
    # Replace with a real DB/API call in production.
    catalog = {"SKU-1001": 42, "SKU-2002": 0}
    qty = catalog.get(sku.upper())
    if qty is None:
        return f"No product found for {sku!r}."
    return f"{sku.upper()}: {qty} units in stock."

if __name__ == "__main__":
    mcp.run(
        transport="streamable-http",
        host="0.0.0.0",
        port=8000,
        stateless_http=True,
        json_response=True,
    )

Install and run:

pip install "mcp[cli]>=2.0.0"
python server.py
# or: uv run mcp dev server.py

What each piece does:

• @mcp.tool() registers a callable the host can invoke. Docstrings become tool descriptions for the model.
• transport="streamable-http" exposes the server on a port at /mcp for remote clients over Hypertext Transfer Protocol (HTTP).
• stateless_http=True creates a fresh transport per request with no session tracking. This aligns the SDK (Software Development Kit) with the stateless protocol core.
• json_response=True returns one JSON body per POST instead of a Server-Sent Events (SSE) stream. Good default for simple tool servers behind load balancers.

A client call (from the same Software Development Kit (SDK)) looks like this:

import asyncio
from mcp import Client

async def main() -> None:
    async with Client("http://localhost:8000/mcp") as client:
        result = await client.call_tool(
            "search_inventory",
            {"sku": "SKU-1001"},
        )
        print(result.structured_content)

asyncio.run(main())

Each call_tool is an independent HTTP (Hypertext Transfer Protocol) round trip. Hit the server from three terminals at once and any replica can serve any call. That is the behavior you want in staging before you trust it in production.

Migration Checklist

If you are upgrading from MCP v1 SDKs (Software Development Kits) or pre-2026 HTTP (Hypertext Transfer Protocol) servers:

1. Bump to Software Development Kit (SDK) v2 (Python mcp>=2.0, TypeScript @modelcontextprotocol/server@beta or later).
2. Replace FastMCP imports with MCPServer.
3. Move stateless_http and port settings from the constructor to run().
4. Delete sticky-session config and shared Redis session stores used only for MCP transport.
5. Replace session-keyed application state with explicit IDs returned from tools.
6. Rewrite elicitation flows around Multi Round-Trip Requests (MRTR) instead of open back-channels.
7. Add ttlMs (time-to-live in milliseconds) / cacheScope to list handlers where catalog caching helps.

Important Links and Repositories

• Official spec (2026-07-28): modelcontextprotocol.io/specification/2026-07-28
• Release announcement: The 2026-07-28 Specification
• Architecture overview: MCP Architecture Docs
• Python SDK (Software Development Kit) (v2): github.com/modelcontextprotocol/python-sdk
• Python SDK docs: Running your server
• TypeScript SDK (Software Development Kit): github.com/modelcontextprotocol/typescript-sdk
• Go SDK (Software Development Kit): github.com/modelcontextprotocol/go-sdk
• Google deep dive on stateless updates: Scaling AI Agent Infrastructure with MCP Stateless Updates
• Microsoft Azure perspective: MCP Just Went Stateless on App Service

Bottom Line

MCP started as a clean way to plug tools into a local AI assistant. Stateless HTTP (Hypertext Transfer Protocol) turns it into something you can run the way you run the rest of your stack: behind a load balancer, across regions, on serverless, with ordinary observability.

The trade is slightly larger requests and a migration sprint if you built on sessions. The payoff is boring scaling, which is exactly what you want once agents stop being a demo and start being a product surface.

If you are starting fresh, build on 2026-07-28 and skip the session workarounds entirely. If you are maintaining an existing remote server, treat this like any other protocol upgrade: staged rollout, benchmark your payload sizes, delete the Redis session layer when receipts prove you do not need it.

Bibliography

• Model Context Protocol. (2026, July 28). The 2026-07-28 specification. Model Context Protocol Blog. https://blog.modelcontextprotocol.io/posts/2026-07-28/
• Model Context Protocol. (n.d.). Architecture. https://modelcontextprotocol.org/docs/learn/architecture
• Model Context Protocol. (n.d.-b). Running your server. MCP Python SDK. https://py.sdk.modelcontextprotocol.io/v2/run/
• New Relic. (2026). MCP is going stateless: What the new spec means for AI agents. https://newrelic.com/blog/ai/mcp-is-going-stateless
• Van Gent, K., & Blount, A. (2026, August 5). Scaling AI agent infrastructure with the MCP stateless updates. Google Developers Blog. https://developers.googleblog.com/scaling-ai-agent-infrastructure-with-the-mcp-stateless-updates/

Read More

The Hidden Climate Cost of AI Data Centers in India

Artificial intelligence is becoming a central part of India’s growth story. From chatbots and recommendation systems to healthcare analytics and smart mobility, AI is shaping how individuals and industries function. It often feels intangible, almost weightless, as if it exists purely in the digital world.

But behind every AI interaction lies something very physical: data centers.

These are not small server rooms tucked away in offices. Modern AI data centers are massive, industrial-scale facilities filled with thousands of high-performance machines running continuously. As India accelerates its adoption of AI, the rapid expansion of these facilities is beginning to have a noticeable impact on the environment, particularly on energy, water, and local climate conditions.

AI Runs on Electricity, Not Just Algorithms

When people think about AI, they usually imagine software, models, and code. What is often overlooked is the scale of electricity required to run these systems.

An AI data center functions like a factory that never stops operating. Thousands of processors handle computations every second, responding to user queries, training models, and processing data streams. Unlike traditional computing workloads, AI tasks are significantly more energy-intensive.

To understand the scale, consider a single large data center consuming electricity comparable to tens of thousands of homes. Now imagine multiple such facilities operating in and around major Indian cities like Bengaluru, Hyderabad, Mumbai, and Chennai. These are already regions where electricity demand is high, especially during peak summer months.

In India, a substantial portion of electricity is still generated from coal. This means that as AI usage grows, the indirect carbon emissions associated with that usage also increase. What appears to be a simple digital interaction is, in reality, linked to a much larger energy system that has environmental consequences.

The Overlooked Resource: Water

While energy consumption is widely discussed, water usage remains one of the least understood aspects of data center operations.

Servers generate heat as they process data, and without effective cooling, they cannot function reliably. Many data centers rely on water-based cooling systems to manage this heat. These systems can consume enormous quantities of water on a daily basis.

To put this into perspective, a large AI data center can use as much water in a day as a small residential community. In a country like India, where water scarcity is already a pressing issue in many regions, this raises serious concerns.

Cities such as Chennai and Bengaluru have experienced significant water shortages in recent years. Groundwater levels have been declining, and urban demand continues to rise. Introducing water-intensive infrastructure into such environments creates competition between industrial use and essential human needs like drinking water and agriculture.

This is not a distant or theoretical issue. It is a practical challenge that cities may increasingly face as more data centers are built.

Heat Generation and Its Local Effects

Another important but less visible impact of data centers is heat.

Every machine inside a data center produces heat while operating. Cooling systems remove this heat and release it into the surrounding environment. When multiple data centers are concentrated in urban areas, this can contribute to localized warming.

In cities that are already experiencing high temperatures, this additional heat can intensify what is known as the urban heat island effect. This phenomenon occurs when built environments trap heat, causing cities to remain warmer than surrounding rural areas.

The consequences are tangible. Higher temperatures increase the demand for air conditioning in homes and offices. This, in turn, raises electricity consumption, which can lead to even greater emissions if the energy comes from non-renewable sources. Over time, this creates a feedback loop where cooling demands drive more energy use, which then contributes to further warming.

The Environmental Cost Beyond Operations

The impact of AI data centers extends beyond their day-to-day operations.

The hardware used in these facilities, including GPUs and specialized chips, requires complex manufacturing processes. These processes consume large amounts of water and energy and involve chemicals that must be carefully managed.

In addition, the lifecycle of AI hardware is relatively short. As newer, more powerful systems are developed, older equipment is replaced. This leads to the generation of electronic waste, which is one of the most challenging types of waste to handle due to its toxic components.

There are also emissions associated with construction. Building a data center requires materials such as steel and concrete, both of which have significant carbon footprints. Transportation of equipment and ongoing maintenance activities further add to the overall environmental impact.

Land Use and Long-Term Commitments

AI data centers require large parcels of land and robust infrastructure, including power supply systems, network connectivity, and backup facilities.

In some cases, this land may have previously been used for agriculture or may have supported local ecosystems. Once a data center is established, it represents a long-term commitment. These facilities are not easily relocated, and their presence shapes the surrounding environment for decades.

This makes site selection a critical decision. Choosing locations without considering environmental constraints can lead to long-term challenges that are difficult to reverse.

Why India Faces a Unique Challenge

Every country building AI infrastructure faces environmental trade-offs, but India’s situation is particularly complex.

The country has a large and growing population, increasing digital demand, and limited natural resources, especially freshwater. At the same time, it is striving for economic growth and technological leadership.

This creates a delicate balance. On one hand, data centers bring investment, jobs, and technological advancement. On the other hand, they place additional pressure on already strained resources.

In regions where water scarcity and energy demand are already concerns, the introduction of resource-intensive infrastructure can amplify existing challenges.

Building AI Infrastructure Responsibly

The question is not whether India should build AI data centers. These facilities are essential for supporting digital services and innovation.

The real question is how they should be built.

There are several approaches that can reduce environmental impact. Transitioning to renewable energy sources such as solar and wind can significantly lower carbon emissions. Using alternative cooling technologies, such as air cooling or advanced liquid cooling systems that minimize water usage, can address water concerns.

Locating data centers in regions with cooler climates or more abundant resources can also improve efficiency. Additionally, designing systems to reuse waste heat or recycle water can make operations more sustainable.

These solutions require planning, investment, and regulation, but they offer a path forward that balances technological growth with environmental responsibility.

To be Honest & Finally to Conclude this...

Artificial intelligence is often described as the future. However, its foundation is deeply rooted in physical infrastructure that interacts directly with the environment.

In India, the expansion of AI data centers represents both an opportunity and a challenge. These facilities can drive innovation and economic growth, but they also have the potential to strain energy systems, deplete water resources, and contribute to local and global climate change.

Understanding this dual impact is essential.

The long-term success of AI in India will not depend solely on advancements in algorithms or software. It will also depend on how thoughtfully the supporting infrastructure is designed and managed.

In the end, the true measure of progress will not just be how intelligent our systems become, but how sustainably we choose to build and operate them.

Bibliography

• International Energy Agency. (2025). Data centres and energy demand. Retrieved from https://www.iea.org
• Council on Energy, Environment and Water. (2024). Data centre infrastructure in India: Power and water use. Retrieved from https://www.ceew.in
• The Wire. (2024). India is betting big on data centres, but at what cost? Retrieved from https://www.thewire.in
• Press Information Bureau, Government of India. (2024). Growth of data centres in India and power demand. Retrieved from https://www.pib.gov.in
• Deccan Herald. (2024). Water impact of AI and data centres in India. Retrieved from https://www.deccanherald.com
• Socomec. (2024). AI energy consumption trends and future projections. Retrieved from https://www.socomec.co.in
• Environmental and Energy Study Institute. (2023). Data centers and water consumption. Retrieved from https://www.eesi.org

Read More

AI Hype vs Actual Use: Is the AI Bubble Still On?

AI is everywhere.

Every product is “AI-powered.”
Every roadmap has AI.
Every demo looks impressive.

But if you are building real systems, you already know:

AI in production is very different from AI in presentations.

The Hype

The story sounds simple:

• Add AI
• Get intelligence
• Scale instantly

Clean input. Smart output. Done.

The Reality

Nothing is clean.

• Data is messy.
• Sensors drift.
• APIs are inconsistent.
• Latency exists.

Before AI even starts, you are already fixing problems.

Most of the work is not AI. It is data and systems.

What Breaks First

Data

You do not get a dataset.
You build one. Slowly.

Models

They do not crash.
They quietly become less useful.

Real-time

Looks great in slides.
Feels slow in production.

Expectations

This is where things get interesting.

The Expectation Gap (After AI Tools Arrived)

Then came AI tools and AI IDEs.

Suddenly everything looked faster:

• Code generation in seconds
• Models built in minutes
• Demos ready almost instantly

From the outside, it feels like:

“Now everything should be faster.”

What Leadership Often Assumes

At a high level, it sounds logical:

• AI writes code
• AI builds models
• AI speeds up development

So naturally:

• Timelines should shrink
• Teams should do more with less
• Complexity should reduce

What Actually Happens on the Ground

AI helps. No doubt.

But it does not remove the hard parts:

• Understanding messy requirements
• Handling real-world data issues
• Debugging edge cases
• Integrating with existing systems
• Making things reliable

AI accelerates output, but it does not remove complexity.

The Silent Pressure

This creates an unspoken expectation:

• “Why is this taking so long?”
• “Can’t AI handle this?”
• “This should be quicker now, right?”

Teams end up:

• Prototyping faster
• Struggling the same in production

The Reality Check

AI IDEs can generate code.

They cannot:

• Guarantee correctness
• Fully understand business context
• Handle production edge cases

The last 20% still takes the most effort.

And that part decides success or failure.

Hard Truth

Most problems do not need AI.

A simple rule often works:

• Faster
• Cheaper
• Easier to maintain

Adding AI too early just adds complexity.

So… Is It a Bubble?

Partly.

There is hype:

• Overuse of “AI-powered”
• Solving simple problems with complex tools
• Chasing trends

That will settle.

What Is Actually Real

AI works when:

• Patterns are complex
• Data is large
• Rules stop working

That is where it shines.

Not everywhere.

What Actually Works

Start simple

Rules first.
AI later.

Combine approaches

Rules + statistics + AI
This works in real systems.

Keep it replaceable

Models will change.
Your system should not break.

Monitor everything

If you cannot see it, you cannot trust it.

The Cost Nobody Talks About

AI is not just a model.

It is:

• Data pipelines
• Infrastructure
• Monitoring
• Retraining

AI is a system commitment.

Better Question to Ask

Not:

“Where can we use AI?”

But:

“Where are we stuck without it?”

Finally to conclude 

AI is real.
The hype is real too.

Both are happening at the same time.

The winners will not be the ones who use AI everywhere.
They will be the ones who use it where it actually matters.

If You Are Building

Focus on:

• Clean data
• Reliable systems
• Clear problems

Then bring in AI.

Bibliography

• Artificial Intelligence: A Modern Approach
• Stuart Russell, & Peter Norvig. Artificial intelligence: A modern approach (4th ed.). Pearson.
• Designing Data-Intensive Applications
• Martin Kleppmann. Designing data-intensive applications. O’Reilly Media.
• McKinsey & Company. The state of AI: Global survey. Retrieved from https://www.mckinsey.com/
• IBM: What is artificial intelligence? Retrieved from https://www.ibm.com/topics/artificial-intelligence
• Stanford University. AI Index Report. Retrieved from https://aiindex.stanford.edu/

Read More

Building Smarter Robots with Small Language Models in Everyday Life

 🎉 Happy New Year to All My Readers 🎉

I hope this year brings health, learning, growth, and meaningful success to you and your loved ones.

A new year always feels like a clean slate. For technology, it is also a good moment to pause and ask a simple question:

Are we building things that are truly useful in daily life?

This is why I want to start the year by talking about something very practical and underrated
Small Language Models (SLMs) and how they can be used in robotics for everyday use cases in a cost-effective way.

Why We Are Considering Small Language Models (SLMs)

In real-world robotics, the goal is not to build the smartest machine in the world. The goal is to build a machine that works reliably, affordably, and efficiently in everyday environments. This is one of the main reasons we are increasingly considering Small Language Models instead of very large, general-purpose AI models.

Most robotic tasks are well-defined. A robot may need to understand a limited set of voice commands, respond to simple questions, or make basic decisions based on context. Using a massive AI model for such tasks often adds unnecessary complexity, higher costs, and increased latency. Small Language Models are focused by design, which makes them a much better fit for these scenarios.

Another important reason is cost efficiency. Robotics systems already require investment in hardware, sensors, motors, and power management. Adding large AI models on top of this quickly becomes expensive, especially when cloud infrastructure is involved. SLMs can run on edge devices with modest hardware, reducing cloud dependency and making large-scale deployment financially practical.

Reliability and control also play a major role. Smaller models are easier to test, debug, and validate. When a robot behaves unexpectedly, understanding the cause is far simpler when each model has a clearly defined responsibility. This modular approach improves safety and makes systems easier to maintain over time.

Privacy is another strong factor. Many robotics applications operate in homes, hospitals, offices, and factories. Running SLMs locally allows sensitive data such as voice commands or environment context to stay on the device instead of being sent to external servers. This builds trust and aligns better with real-world usage expectations.

Finally, SLMs support a long-term, scalable architecture. Just like microservices in software, individual AI components can be upgraded or replaced without rewriting the entire system. This flexibility is essential as AI technology continues to evolve. It allows teams to innovate steadily rather than rebuilding from scratch every few years.

For robotics in everyday life, intelligence does not need to be massive. It needs to be purpose-driven, efficient, and dependable. Small Language Models offer exactly that balance, which is why they are becoming a key building block in modern robotic systems.

From Big AI Models to Small Useful Intelligence

Most people hear about AI through very large models running in the cloud. They are powerful, but they are also expensive, heavy, and sometimes unnecessary for simple real-world tasks.

In daily robotics use, we usually do not need a model that knows everything in the world.
We need a model that can do one job well.

This is where Small Language Models come in.

SLMs are:

• Smaller in size
• Faster to run
• Cheaper to deploy
• Easier to control

And most importantly, they are practical.

Thinking of SLMs Like Microservices for AI

An Example architecture of Monolithic vs Microservices used in Software Inductries

In software, we moved from monolithic applications to microservices because:

• They were easier to maintain
• Easier to scale
• Easier to replace

The same idea works beautifully for AI in robotics.

Instead of one huge AI brain, imagine multiple small AI blocks:

• One model for voice commands
• One model for intent detection
• One model for navigation decisions
• One model for basic conversation

Each SLM does one specific task, just like a microservice.

This makes robotic systems:

• More reliable
• Easier to debug
• More cost-effective
• Easier to upgrade over time

Everyday Robotics Where SLMs Make Sense

Let us talk about real, everyday examples.

Home Robots

A home assistant robot does not need a giant model.
It needs to:

• Understand simple voice commands
• Respond politely
• Control devices
• Follow routines

An SLM running locally can do this without sending data to the cloud, improving privacy and reducing cost.

Office and Workplace Robots

In offices, robots can:

• Guide visitors
• Answer FAQs
• Deliver items
• Monitor basic conditions

Here, SLMs can handle:

• Limited vocabulary
• Context-based responses
• Task-oriented conversations

No heavy infrastructure needed.

Industrial and Warehouse Robots

Industrial robots already know how to move.
What they lack is contextual intelligence.

SLMs can help robots:

• Understand instructions from operators
• Report issues in natural language
• Decide next actions based on simple rules plus learning

This improves efficiency without increasing system complexity.

Healthcare and Assistance Robots

In hospitals or elderly care:

• Robots need predictable behavior
• Fast response
• Offline reliability

SLMs can be trained only on medical workflows or assistance tasks, making them safer and more reliable than general-purpose AI.

Why SLMs Are Cost-Effective

This approach reduces cost in multiple ways:

• Smaller models mean lower hardware requirements
• Edge deployment reduces cloud usage
• Focused training reduces development time
• Modular design avoids full system rewrites

For startups, researchers, and even individual developers, this makes robotics accessible, not intimidating.

The Bigger Picture

The future of robotics is not about giving robots human-level intelligence. It is about giving them just enough intelligence to help humans better.

SLMs enable exactly that.

They allow us to build robots that:

• Are useful
• Are affordable
• Are trustworthy
• Work in real environments

A New Year Thought

As we step into this new year, let us focus less on building the biggest AI and more on building the right AI.

• Small models.
• Clear purpose.
• Real impact.

Happy New Year once again to all my readers 🌟

Let us focus on building technology that serves people locally and globally, addresses real-world problems, and creates a positive impact on society.

Bibliography

• OpenAI – Advances in Language Models and Practical AI Applications
• Used as a reference for understanding how modern language models are designed and applied in real-world systems.
• Google AI Blog – On-Device and Edge AI for Intelligent Systems
• Referenced for insights into running AI models efficiently on edge devices and embedded systems.
• Hugging Face Documentation – Small and Efficient Language Models
• Used to understand lightweight language models, fine-tuning techniques, and deployment strategies.
• NVIDIA Developer Blog – AI for Robotics and Autonomous Systems
• Referenced for practical use cases of AI in robotics, including perception, navigation, and decision-making.
• MIT Technology Review – The Rise of Practical AI in Robotics
• Used for broader industry perspectives on how AI is shifting from experimental to everyday applications.
• Robotics and Automation Magazine (IEEE) – Trends in Modern Robotics Systems
• Referenced for understanding modular robotics architectures and intelligent control systems.
• Personal Industry Experience and Hands-on Projects
• Insights based on real-world development, experimentation, and system design experience in AI-driven applications.

Read More

TOON: The Future of Structured Data for AI - A Simpler, Lighter, Human-Friendly Alternative to JSON

For more than a decade, JSON has been the backbone of web APIs. It’s everywhere powering apps, microservices, logs, configs, and data pipelines. But as we enter a world dominated by AI agents, LLM workflows, and token-optimized prompts, JSON is starting to show its age.

Today’s AI systems don’t just consume data ; they interpret it, reason with it, and generate new structures from it.

Yet JSON, with its endless braces, commas, and quotes, wasn’t built for that kind of work.

So a new idea has emerged:

TOON : Token-Oriented Object Notation

A compact, human-readable, AI-friendly alternative to JSON that reduces token cost, improves model understanding, and simplifies structured prompt design.

And honestly?
It’s one of the most refreshing innovations in AI tooling I’ve seen in years.

The Problem with JSON in AI Workflows

Let’s be fair JSON is excellent for machines.
But for humans designing structured prompts, tool schemas, agent configs, and reasoning structures, JSON becomes:

• Too verbose
• Hard to read
• Token-inefficient
• Not friendly for mixing text + structure + examples
• Difficult to reference or reuse

Consider this: every {, ", :, and , you include in JSON becomes a token when passed to a language model. That is wasted budget, wasted context window, and wasted clarity.

The freeCodeCamp article puts it elegantly:

“JSON’s punctuation and quotes create unnecessary token bloat that doesn’t help the model understand your structure.”

And when your prompts or agent configs grow into the hundreds of lines, you feel that bloat.

Which brings us to…

Enter TOON: Token-Oriented Object Notation

TOON is a new notation format designed precisely for AI systems especially LLMs. It aims to solve JSON’s weaknesses while keeping the same underlying data model.

According to the official TOON GitHub repository:

“TOON is a compact, human-readable encoding of the JSON data model designed for LLM prompts. It provides a lossless serialization of objects, arrays, and primitives but with far fewer tokens.”

So you get the best of both worlds:

• JSON compatibility
• Human-friendly syntax
• LLM-optimized token efficiency

It’s like someone finally said:
“What if structured data didn’t have to look like a programming parse tree?”

TOON vs JSON: A Side-by-Side Look

JSON Example

{
  "users": [
    { "id": 1, "name": "Alice", "role": "admin" },
    { "id": 2, "name": "Bob", "role": "user" }
  ]
}

TOON Equivalent

users[2]{id,name,role}:
  1,Alice,admin
  2,Bob,user

Immediately you notice:

✔ No quotes
✔ No braces
✔ No commas between fields
✔ Cleaner structure
✔ Fewer tokens
✔ Easier for both humans and LLMs to interpret

This is the magic of TOON.

TOON is Not Just YAML or a Shorthand - It’s Purpose-Built for AI

People might ask:
“Is TOON just another YAML or HCL?”

Not at all.

TOON is designed with 3 AI-specific goals:

1. Minimize Token Count

JSON forces every key and value into quotes, every object into {}, and every list into [].
TOON eliminates most of that.

Why this matters:

• LLM context windows are limited
• Token cost affects your bill
• Structured prompts can get huge (tool definitions, agent descriptions, memory, etc.)

TOON often reduces token usage by 30–60%, according to early tests.

2. Improve Model Parsing & Predictability

Models don’t “see” braces and commas the way developers do. They see tokens.

TOON’s cleaner syntax helps models:

• Parse structure more reliably
• Respect fields more consistently
• Follow templates more accurately

This is especially useful for:

• Function calling
• Structured output enforcement
• Agent workflows
• Multi-turn reasoning setups

3. Make Prompts and Schemas Human-Readable

TOON is designed for the people actually building AI systems:

• Prompt engineers
• Data scientists
• Product teams
• LLM app developers
• Multi-agent workflow designers

You can read TOON like a clean, modern DSL.

A Real-World Example: Defining Tools and Agents in TOON

JSON Tool Definition

{
  "name": "get_weather",
  "description": "Fetch weather by city name",
  "schema": {
    "type": "object",
    "properties": {
      "city": { "type": "string" }
    },
    "required": ["city"]
  }
}

TOON Equivalent

tool get_weather:
  description: Fetch weather by city name
  args{city:string!}

That’s it.

• Cleaner
• Easier to edit
• Far fewer tokens
• Still maps 1:1 to JSON

TOON Supports Deep Structures Too

JSON:

{
  "article": {
    "title": "AI in 2025",
    "tags": ["ai", "future", "trends"],
    "author": {
      "name": "Ravi",
      "followers": 5000
    }
  }
}

TOON:

article:
  title: AI in 2025
  tags[3]: ai,future,trends
  author:
    name: Ravi
    followers: 5000

It looks like a hybrid of:

• JSON’s structure
• CSV’s compactness
• YAML’s simplicity

But behaves like token-optimized JSON under the hood.

Developer Experience: Using TOON in Your Apps

The official GitHub repo provides SDKs for:

✔ TypeScript / JavaScript

npm install @toon-format/toon

✔ Python

pip install python-toon

Convert JSON → TOON (CLI)

npx @toon-format/cli input.json -o output.toon

Convert TOON → JSON

npx @toon-format/cli input.toon -o output.json

You can integrate TOON anywhere you are already using:

• JSON configs
• AI prompts
• LLM tool schemas
• Agent definitions
• Structured output templates

Where TOON Really Shines

1. AI tool calling

Cleaner schemas, fewer tokens, better consistency.

2. Multi-agent ecosystems

Easier to define agent roles, memory, context, and routing rules.

3. RAG pipelines

Structured metadata is more readable and cheaper to embed.

4. Workflow orchestration

Tasks, edges, and dependencies look like a proper DSL instead of a JSON jungle.

5. Prompt engineering at scale

Prompts become easier to maintain, version, document, and share.

Limitations of TOON (Honest Assessment)

The GitHub repo outlines some limitations:

• Extremely irregular JSON might not compress well
• Round-trip conversion should be tested for edge cases
• JSON remains better for general web API interoperability
• Tools and libraries are still maturing

TOON is not a replacement for JSON everywhere, it is a better tool for AI-specific use cases.

Final Thought: TOON is JSON for the AI Era

AI changes how we think about data.

And TOON feels like the first serialization format truly designed for the LLM age where:

• Human readability
• Token efficiency
• Structured reasoning
• Model-friendliness

…all matter just as much as machine parsing.

TOON is not a buzzword, it’s a practical, elegant evolution in how we express structured information to AI systems.

If you work with prompts, agents, or structured LLM outputs, TOON will feel like a breath of fresh air i.e  simple, compact, powerful.

Bibliography

• freeCodeCamp. (2024). What is TOON? How Token-Oriented Object Notation Could Change How AI Sees Data. https://www.freecodecamp.org/news/what-is-toon-how-token-oriented-object-notation-could-change-how-ai-sees-data/
• TOON Format Team. (2024). TOON – Token-Oriented Object Notation (GitHub repository). https://github.com/toon-format/toon

Read More

The Data Engines Driving RAG, CAG, and KAG

AI augmentation doesn’t work without the right databases and data infrastructure. Each approach (RAG, CAG, KAG) relies on different types of databases to make information accessible, reliable, and actionable.

RAG – Retrieval-Augmented Generation

Databases commonly used

• Pinecone – Vector Database | Cloud SaaS | Proprietary license
• Weaviate – Vector Database | v1.26+ | Apache 2.0 License
• Milvus – Vector Database | v2.4+ | Apache 2.0 License
• FAISS (Meta AI) – Vector Store Library | v1.8+ | MIT License

How it works:

• Stores text, documents, or embeddings in a vector database.
• AI retrieves the most relevant chunks during a query.

Real-World Examples & Applications

• Perplexity AI → Uses retrieval pipelines over web-scale data.
• ChatGPT Enterprise with RAG → Connects company knowledge bases like Confluence, Slack, Google Drive.
• Thomson Reuters Legal → Uses RAG pipelines to deliver compliance-ready legal insights.

CAG – Context-Augmented Generation

Databases commonly used

• PostgreSQL / MySQL – Relational DBs for session history | Open Source (Postgres: PostgreSQL License, MySQL: GPLv2 with exceptions)
• Redis – In-Memory DB for context caching | v7.2+ | BSD 3-Clause License
• MongoDB Atlas – Document DB for user/session data | Server-Side Public License (SSPL)
• ChromaDB – Contextual vector store | v0.5+ | Apache 2.0 License

How it works:

• Stores user session history, preferences, and metadata.
• AI retrieves this contextual data before generating a response.

Real-World Examples & Applications

• Notion AI → Reads project databases (PostgreSQL + Redis caching).
• Duolingo Max → Uses MongoDB-like stores for learner history to adapt lessons.
• GitHub Copilot → Context layer powered by user repo data + embeddings.
• Customer Support AI Agents → Redis + MongoDB for multi-session conversations.

KAG – Knowledge-Augmented Generation

Databases commonly used

• Neo4j – Graph Database | v5.x | GPLv3 / Commercial License
• TigerGraph – Enterprise Graph DB | Proprietary
• ArangoDB – Multi-Model DB (Graph + Doc) | v3.11+ | Apache 2.0 License
• Amazon Neptune – Managed Graph DB | AWS Proprietary
• Wikidata / RDF Triple Stores (Blazegraph, Virtuoso) – Knowledge graph databases | Open Data License

How it works:

• Uses knowledge graphs (nodes + edges) to store structured relationships.
• AI queries these graphs to provide factual, reasoning-based answers.

Real-World Examples & Applications

• Google’s Bard → Uses Google’s Knowledge Graph (billions of triples).
• Siemens Digital Twins → Neo4j knowledge graph powering industrial asset reasoning.
• AstraZeneca Drug Discovery → Neo4j + custom biomedical KGs for linking genes, proteins, and molecules.
• JP Morgan Risk Engine → Uses proprietary graph DB for compliance reporting.

Summary Table

Approach	Database Types	Providers / Examples	License	Real-World Use
RAG	Vector DBs	Pinecone (Proprietary), Weaviate (Apache 2.0), Milvus (Apache 2.0), FAISS (MIT)	Mixed	Perplexity AI, ChatGPT Enterprise, Thomson Reuters
CAG	Relational / In-Memory / NoSQL	PostgreSQL (Open), MySQL (GPLv2), Redis (BSD), MongoDB Atlas (SSPL), ChromaDB (Apache 2.0)	Mixed	Notion AI, Duolingo Max, GitHub Copilot
KAG	Graph / Knowledge DBs	Neo4j (GPLv3/Commercial), TigerGraph (Proprietary), ArangoDB (Apache 2.0), Amazon Neptune (AWS), Wikidata (Open)	Mixed	Google Bard, Siemens Digital Twin, AstraZeneca, JP Morgan

Bibliography

• Pinecone. (2024). Pinecone Vector Database Documentation. Pinecone Systems. Retrieved from https://www.pinecone.io
• Weaviate. (2024). Weaviate: Open-source vector database. Weaviate Docs. Retrieved from https://weaviate.io
• Milvus. (2024). Milvus: Vector Database for AI. Zilliz. Retrieved from https://milvus.io
• Johnson, J., Douze, M., & Jégou, H. (2019). Billion-scale similarity search with GPUs. FAISS. Meta AI Research. Retrieved from https://faiss.ai
• PostgreSQL Global Development Group. (2024). PostgreSQL 16 Documentation. Retrieved from https://www.postgresql.org
• Redis Inc. (2024). Redis: In-memory data store. Redis Documentation. Retrieved from https://redis.io
• MongoDB Inc. (2024). MongoDB Atlas Documentation. Retrieved from https://www.mongodb.com
• Neo4j Inc. (2024). Neo4j Graph Database Platform. Neo4j Documentation. Retrieved from https://neo4j.com
• Amazon Web Services. (2024). Amazon Neptune Documentation. AWS. Retrieved from https://aws.amazon.com/neptune
• Wikimedia Foundation. (2024). Wikidata: A Free Knowledge Base. Retrieved from https://www.wikidata.org

Read More