MCP Gateway

AI GatewayMCP

The Problem

AI agents need tools. Those tools come from many places: internal APIs, third-party SaaS, and specialized MCP servers, each with its own endpoint, auth scheme, and connection lifecycle. As the number of tool providers grows, every agent has to track every endpoint, negotiate every auth flow, and manage every connection individually. That complexity belongs in the infrastructure, not the application.

How Zilla Solves It

Zilla functions as a production-grade MCP gateway. It presents a single Streamable HTTP endpoint to every AI agent and fans out requests to any number of upstream tool providers (MCP servers and HTTP APIs) based on the toolkit name in each tool call. The agent sees one endpoint with a unified tool list. Zilla handles routing, listing aggregation, auth enforcement, caching, and observability behind the scenes.

No custom middleware. No SDK wrapper. No protocol translation written by hand.

Architecture

The binding pipeline composes from standard, reusable pieces. Each layer does one thing:

Binding	Role
`mcp · server`	Terminates the inbound Streamable HTTP connection from the AI agent
`mcp · proxy`	Routes tool calls to upstream providers by toolkit name
`mcp · client`	Connects to an upstream MCP server over HTTP, WebSocket, or SSE
`mcp-http · proxy`	Exposes any HTTP API as MCP tools with no MCP server needed upstream

Toolkit-Based Routing

The proxy layer routes each tool call to the correct upstream using the toolkit field in the route condition. AI agents address tools with the toolkit prefix: bluesky__get_weather routes to the Bluesky toolkit, payments__charge routes to the payments toolkit, and Zilla dispatches accordingly.

routes:
  - exit: bluesky_mcp_client
    when:
      - toolkit: bluesky
  - exit: payments_mcp_client
    when:
      - toolkit: payments

Adding a new tool provider is a config change, not a code change.

Listing Cache

tools/list, prompts/list, and resources/list calls are cached at the gateway with a configurable TTL (default: 5 minutes). The agent gets a merged, deduplicated view of every upstream's capabilities in a single response. Upstream servers are not called on every listing request.

Auth Enforcement

The agent's session is validated directly on the mcp · server binding through its own options.authorization field, before a request ever reaches mcp · proxy or an upstream server: JWT bearer validates an inbound token against a JWKS endpoint, using the open-source jwt guard. Auth state (JWKS keys) is held in-memory. See Bearer Auth for how the flow is wired.

Observability

Every MCP method call is instrumented automatically. Zilla emits per-method counters and duration histograms dimensioned by toolkit and outcome, with no instrumentation required in the agent or the upstream server.

Metrics export to Prometheus or via OTLP to any OpenTelemetry-compatible backend.

Try It

Try the example

Walk through Get Started for the runnable steps: two upstream MCP servers, toolkit-based routing, a 5-minute listing cache, and Prometheus metrics, all from a single docker compose up.