Skip to main content

MCP Gateway Setup

Get Started brought up AI Gateway with two upstream MCP servers, bluesky-mcp and payments-mcp, routed by toolkit. This tutorial builds on that same running stack and adds a third upstream: a plain REST API with no MCP server in front of it at all.

The Virtual Server doc puts it directly: "Adding a new upstream is one new binding and one new route entry." This walks through exactly that, using the mcp-http · proxy binding so the new upstream doesn't need to speak MCP at all, Zilla exposes its REST endpoints as tools directly.

Note

This uses httpbin, a small public REST API test service, as a stand-in for "any REST API you want to expose." For the deeper, Kafka-specific version of this pattern, see Kafka as MCP Tools, which chains mcp-http · proxy in front of a Kafka-backed REST API instead.

Prerequisites

Add the Upstream Container

Open the docker-compose.yaml in zilla/examples/mcp.proxy (the one you started in Get Started) and add a third service alongside bluesky-mcp and payments-mcp:

Download httpbin-service.yaml

httpbin:
  image: kennethreitz/httpbin
  ports:
    - 8000:80

Route a Toolkit to It

Edit the same directory's zilla.yaml. Adding the upstream is three changes: a new route on north_mcp_proxy, a new mcp-http · proxy binding as its exit, and a small catalog entry for the one tool it exposes.

New Route

The last three lines are the addition; the bluesky and payments routes are unchanged.

north_mcp_proxy:
  type: mcp
  kind: proxy
  options:
    cache:
      store: cache
      ttl: PT5M
  routes:
    - exit: bluesky_mcp_client
      when:
        - toolkit: bluesky
    - exit: payments_mcp_client
      when:
        - toolkit: payments
    - exit: httpbin_http_proxy
      when:
        - toolkit: httpbin

The existing bluesky_mcp_client and payments_mcp_client bindings and their routes are untouched.

Full zilla.yaml config

Download zilla.yaml

name: mcp-gateway-setup

#region new_catalog
catalogs:
  httpbin_catalog:
    type: inline
    options:
      subjects:
        check_http_status_params:
          schema: |
            {
              "type": "object",
              "properties": {
                "code": {
                  "type": "integer"
                }
              },
              "required": [
                "code"
              ]
            }
#endregion new_catalog

stores:
  cache:
    type: memory

bindings:
  north_tcp_server:
    type: tcp
    kind: server
    options:
      host: 0.0.0.0
      port: 7114
    routes:
      - when:
          - port: 7114
        exit: north_http_server

  north_http_server:
    type: http
    kind: server
    options:
      access-control:
        policy: cross-origin
    routes:
      - when:
          - headers:
              ":path": /mcp
        exit: north_mcp_server

  north_mcp_server:
    type: mcp
    kind: server
    exit: north_mcp_proxy
    telemetry:
      metrics:
        - mcp.*
      attributes:
        method: ${mcp.method}
        tool: ${mcp.tool}
        outcome: ${mcp.outcome}

  #region mcp_proxy
  north_mcp_proxy:
    type: mcp
    kind: proxy
    options:
      cache:
        store: cache
        ttl: PT5M
    routes:
      - exit: bluesky_mcp_client
        when:
          - toolkit: bluesky
      - exit: payments_mcp_client
        when:
          - toolkit: payments
      - exit: httpbin_http_proxy
        when:
          - toolkit: httpbin
  #endregion mcp_proxy

  bluesky_mcp_client:
    type: mcp
    kind: client
    options:
      server: http://bluesky-mcp:3001/mcp
    exit: sys:http_client

  payments_mcp_client:
    type: mcp
    kind: client
    options:
      server: http://payments-mcp:3002/mcp
    exit: sys:http_client

  #region new_binding
  httpbin_http_proxy:
    type: mcp-http
    kind: proxy
    options:
      tools:
        check_http_status:
          description: Check what httpbin returns for a given HTTP status code.
          summary: "Checked httpbin status code"
          schemas:
            input:
              model: json
              catalog:
                httpbin_catalog:
                  - subject: check_http_status_params
                    version: latest
    routes:
      - when:
          - tool: check_http_status
        exit: sys:http_client
        with:
          headers:
            ":method": GET
            ":scheme": http
            ":authority": httpbin:80
            ":path": /status/${args.code}
  #endregion new_binding

telemetry:
  metrics:
    - mcp.initialize
    - mcp.initialize.duration
    - mcp.tools.list
    - mcp.tools.list.duration
    - mcp.tools.call
    - mcp.tools.call.duration
  exporters:
    prometheus_exporter:
      type: prometheus
      options:
        endpoints:
          - scheme: http
            port: 7190
            path: /metrics

Apply the Change

docker compose --project-directory mcp.proxy up -d

This starts the new httpbin container and reloads Zilla with the updated config.

Verify

Confirm httpbin answers directly:

curl http://localhost:8000/get

Confirm Zilla picked up the new config without errors:

docker compose --project-directory mcp.proxy logs zilla

Once an MCP client reconnects (or the listing cache's TTL expires), its aggregated tool list includes check_http_status under the httpbin toolkit alongside bluesky and payments, with no MCP server standing in front of the REST API it calls. No code was written and no existing upstream was restarted.

When you're done, stop the stack:

docker compose --project-directory mcp.proxy down

Next Steps