Observable AI Agents

Use CaseObservabilityAI Gateway

An agent that can call any tool behind the gateway is also an agent whose behavior needs to be watched: which tools it's calling, how often, how long each call takes, and whether calls are succeeding. That visibility shouldn't depend on the agent framework or the upstream server choosing to emit it.

The Problem

Instrumentation added inside an agent or an upstream MCP server only covers that one agent or that one server. A fleet of agents built on different frameworks, calling upstreams maintained by different teams, ends up with inconsistent or missing telemetry, exactly where an operator most needs a consistent view.

How Zilla Solves It

Every MCP method call passing through the gateway is instrumented automatically. Zilla emits per-method counters and duration histograms dimensioned by toolkit and outcome, with no instrumentation required in the agent or the upstream server. Because the gateway sits on every request path, the metrics it produces cover every agent and every upstream uniformly.

Metrics export to Prometheus or via OTLP to any OpenTelemetry-compatible backend.

Metrics and Attributes

Add metric names to telemetry.metrics and they're collected automatically:

telemetry:
  metrics:
    - mcp.initialize
    - mcp.initialize.duration
    - mcp.tools.list
    - mcp.tools.list.duration
    - mcp.tools.call
    - mcp.tools.call.duration

Attach telemetry.attributes to the mcp · server binding to dimension those metrics per request:

north_mcp_server:
  type: mcp
  kind: server
  exit: north_mcp_proxy
  telemetry:
    metrics:
      - mcp.*
    attributes:
      method: ${mcp.method}
      tool: ${mcp.tool}
      outcome: ${mcp.outcome}

method, tool, and outcome are populated per request by the gateway itself, no agent or upstream code needs to set them.

Exporters

Metrics and log events collected by AI Gateway are shipped by exporters, configured once at the top level of zilla.yaml:

telemetry:
  exporters:
    prometheus_exporter:
      type: prometheus
      options:
        endpoints:
          - scheme: http
            port: 7190
            path: /metrics
    stdout_logs_exporter:
      type: stdout

The Prometheus exporter exposes the standard /metrics scrape endpoint. The stdout exporter prints events to standard output, useful during local development or when a log aggregator is already watching a container's stdout. OTLP, AWS CloudWatch, and Syslog exporters are also available for pushing to an external backend.

Beyond Metrics: Events

The gateway also raises named log events for the MCP request lifecycle, connection handling, and guard decisions, for example an event fires when a tool call fails authorization for a guarded route. These events are most useful for troubleshooting, not dashboards, but they export through the same exporters as metrics.

For the full metrics table, attribute reference, event catalog, and exporter comparison, see Monitoring and Observability.

Try It

Try the example

Walk through Observable AI Agents for the runnable steps: make a few tool calls and watch the per-tool, per-outcome counters climb.