Building a Production-Ready Remote MCP Server

The Model Context Protocol (MCP) is an open standard for connecting LLMs to external tools and data sources via a JSON-RPC interface. The key insight is that MCP gives you a proper, discoverable API: a model can list available tools, inspect input schemas and optional output schemas, and request tool calls through a host application. MCP defines separate primitives for tools, resources, and prompts; retrieving a prompt is not itself execution of a function.

I was in charge of creating a remote MCP server to allow our customers to use LLMs to access the data they already have on our platform. What seemed straightforward turned into a deep dive into protocol design, session management, and authorization at scale.

Session Problems in Our April 2025 Deployment

There was one problem: in April 2025, the Python MCP ecosystem was still very young. I needed a library that I could embed directly in a Starlette/FastAPI application — one that was lightweight, type-safe, and simple to reason about. The official SDK had significant issues with our infrastructure, particularly around session management in horizontally scaled environments:

GitHub discussion about MCP session loss in horizontally scaled environments

GitHub Discussion Link

Rather than working around these limitations, I decided to build a library from scratch with the characteristics mentioned above — and without the problems encountered in that deployment. This is a historical account, not a claim that current SDK releases require the same workaround. The official SDK now documents deployment and scaling options, including stateless operation.

Building http-mcp

http-mcp (published on PyPI as http-mcp) provides tools and prompts over HTTP and stdio. Its supported features and protocol versions should be checked against the client in use. I designed it around a few principles:

1. Pydantic-first, schema automatically generated. Tools are declared by wrapping a Python function in a Tool dataclass and pointing at its Pydantic input/output models. The JSON schema that MCP clients use for discovery is derived automatically from those models:

from http_mcp.types import Arguments, Tool
from pydantic import BaseModel, Field, UUID4


class FixMetadataInput(BaseModel):
    vulnerability_id: UUID4 = Field(description="Vulnerability uuid4 id")


class FixMetadataOutput(BaseModel):
    version: str = Field(
        description="Safe version of the dependency",
    )
    is_breaking_change: bool = Field(
        description="Whether upgrading to safe version is a breaking change",
    )


def get_fix_metadata(
    arguments: Arguments[FixMetadataInput],
) -> FixMetadataOutput:
    return FixMetadataOutput(
        version="2.3.4",
        is_breaking_change=False,
    )


tool = Tool(
    inputs=FixMetadataInput,
    output=FixMetadataOutput,
    func=get_fix_metadata,
)

With inputs as Pydantic models validation happens automatically before the data reaches the tool function. Inputs are rejected with structured errors if they do not match the schema. Another advantage is that every field is self-documenting — by using Field(description="..."), both the input and output schemas carry human-readable documentation that MCP clients can surface to users and models alike.

2. Starlette-native auth. Rather than inventing a new auth model, the library plugs into Starlette’s AuthenticationMiddleware. Scope-based access control just works: tools declare which scopes they require, and the framework filters them before listing or invocation.

3. Tool errors as model feedback. The http-mcp documentation describes return_error_message=True as converting ToolInvocationError exceptions into ErrorMessage results. This is a library feature, not a general guarantee about all Pydantic failures or HTTP status codes. A useful error can explain what the caller should correct, provided the client exposes that result to the model.

Tool(
    inputs=FixMetadataInput,
    output=FixMetadataOutput,
    func=get_fix_metadata,
    return_error_message=True,  # Return supported tool invocation errors as ErrorMessage results
    scopes=("mcp:read",),
)

This can make recoverable failures actionable. Whether a retry succeeds, or saves tokens, depends on the error and the client.

Stateful Sessions: A Scaling Problem

The early versions of the MCP Python SDK maintained session state across requests. This creates a critical problem in horizontally scaled environments: if your MCP server runs on multiple workers behind a load balancer, each request might hit a different worker instance. Session data stored on Worker A is invisible to Worker B, causing a session lookup to fail. That can interrupt tool access, but it does not inherently erase the model’s conversation history, which the host manages.

The difficulty is coordinating state, not the mere existence of state:

Requests can reach a worker that does not own an in-memory session.
Session affinity is one option, with failover trade-offs. Shared state and routing to the owning worker are other possible designs.
Stateless request handling simplifies horizontal scaling, but stateful services can also scale with appropriate coordination and recovery.

For our workload, avoiding per-worker session state was a useful trade-off.

The solution was to implement a stateless HTTP transport that treats each request as independent. Instead of maintaining session objects in memory, the transport sends all necessary context in each request. This stateless design became the foundation for http-mcp: https://pypi.org/project/http-mcp/

With the session in worker memory, the second request fails as soon as the balancer picks another worker. When each request carries its context, any worker can answer.

Remote MCP Architecture: Streamable HTTP vs stdio

The 2025-11-25 MCP specification defines two standard transports and permits custom ones:

Stdio Transport: The MCP server runs as a subprocess on the same machine, communicating via standard input/output. This is simple for local integrations but limits you to a single host.
Streamable HTTP: The MCP server runs as a separate service (often remote), communicating over HTTP. This enables true separation of concerns and horizontal scaling.

For production systems, HTTP transport is the practical choice. It allows separate deployment and scaling of the MCP server, easier monitoring and logging, integration with load balancers and service meshes, and language-agnostic communication between components.

The trade-off is complexity: handle network errors, timeouts, authentication, and any chosen session lifecycle. Streamable HTTP permits optional sessions; it does not require every server to be stateless.

Challenge 1: Errors the Caller Can Act On

Input-schema validation and tool execution are different stages. Malformed protocol requests can produce JSON-RPC errors; recoverable execution failures can be represented as tool results. MCP defines an isError flag for tool execution errors. See the tool error rules.

In http-mcp, the documented return_error_message option handles supported ToolInvocationError exceptions. For example, a handler can report that a requested record is unavailable, with enough context for the caller to change its arguments. Do not assume that every client displays errors identically or that a retry will succeed.

sequenceDiagram
participant LLM
participant Client as MCP client
participant Server as MCP server
LLM->>Client: request a record
Client->>Server: tools/call
Server-->>Client: tool result describing recoverable failure
Client-->>LLM: error details, if exposed by the host
LLM->>Client: revised arguments
Client->>Server: tools/call
Server-->>Client: tool result

A recoverable tool error can guide a revised request when the client passes its details to the model. The outcome depends on the client and the error.

Challenge 2: Context-Based Tool Exposure

Not all tools should be available to all callers. You might want:

Public tools (no auth required): A “get_server_status” tool anyone can use
Private tools (auth required): A “delete_user” tool only admins can access
Scoped tools: Different tools for different contexts or user roles

I solved this by leveraging Starlette’s scope system. Each HTTP request carries scope metadata (user, role, permissions), and tools declare which scopes they require. With the same models as before, the only addition is scopes:

Tool(
    inputs=FixMetadataInput,
    output=FixMetadataOutput,
    func=get_fix_metadata,
    scopes=("mcp:read",),  # Only callers with "mcp:read" scope can use this tool
)

This approach has several benefits: tools are self-documenting about their requirements, authorization is centralized and consistent, you can change permissions without modifying tool code, and the AI agent can query which tools are available in its current context — which reduces suggestions for unavailable tools. A client can still submit a hidden or stale tool name, so the server must authorize every invocation.

The agent calls tools/list with a bearer token that grants mcp:read.
A configured authentication backend validates the token and supplies the caller's scopes to Starlette.
Each tool declares the scopes it requires, and the server compares them with the caller's.
The response lists only the tools the caller may use. delete_user stays hidden.

Scope filtering limits discovery. The server must also reject unauthorized tools/call requests, even if the client knows a hidden tool name.

Lessons Learned

Building a production MCP server taught me several things:

Stateless by default. Design your server assuming instances are ephemeral. This simplifies request routing and avoids some session-related bugs.
Error messages are feedback loops. The AI agent can adjust subsequent calls using error messages. Make them helpful and specific — generic errors cause retries and wasted tokens. Some MCP clients do not even pass error responses to the LLM; supported tool errors can instead be returned as results via return_error_message=True.
Explicit permissions matter. Making scope requirements visible in tool definitions makes it easier to reason about security and to conditionally expose tools and prompts based on the caller’s identity.

References

Official MCP Specification: modelcontextprotocol.io
My HTTP Transport Implementation: http-mcp on PyPI
Stateless Sessions Discussion: Python SDK GitHub Issue #520

Recent Posts