agentgateway

agentgateway

Expert guidance for agentgateway — the AI/agentic proxy for LLM routing, MCP server routing, and external processing on Kubernetes.

Overview

agentgateway is a Linux Foundation project that extends Kubernetes Gateway API for AI workloads. It provides:

• LLM Provider Routing: Ollama (local), OpenAI, Anthropic, Gemini, Vertex AI (GCP), Azure OpenAI, AWS Bedrock
• MCP Server Routing: Model Context Protocol endpoints via HTTP/SSE
• External Processing: Delegate request/response handling to gRPC services for custom mutations and routing
• Traffic Policies: Rate limiting (requests and tokens), CORS, authentication via AgentgatewayPolicy

Note: Starting v2.3.0, agentgateway diverges from kgateway into its own repo (agentgateway/agentgateway) and namespace (agentgateway-system). See references/helm-lifecycle.md.

Quick Commands

This plugin includes slash commands for gateway operations:

Command	Purpose
/gw-status	Show all gateway resources status
/gw-logs [count|controller]	Tail controller logs for debugging
/gw-debug	Full diagnostic with events and recommendations
/gw-backend <provider>	Generate backend YAML (ollama, openai, anthropic, gemini, vertexai, azureopenai, bedrock)
/gw-eval	Evaluate current config against best practices
/gw-versions	Compare installed vs latest helm chart versions
/gw-upgrade [component]	Guide helm chart upgrades with pre/post validation

The gateway-manager agent handles multi-step workflows: "add an Anthropic backend", "set up Vertex AI routing", "upgrade my gateway", "why isn't my route working".

Architecture

                    ┌─────────────────────────────────────┐
                    │         agentgateway proxy          │
                    │      (port 8080 - LoadBalancer)     │
                    └─────────────┬───────────────────────┘
                                  │
        ┌─────────────────────────┼─────────────────────────┐
   /ollama/*              /anthropic/*                 /mcp/*
   /openai/*              /gemini/*
        │                        │                          │
        ▼                        ▼                          ▼
┌───────────────┐       ┌───────────────┐         ┌───────────────┐
│AgentgatewayBackend    │AgentgatewayBackend      │AgentgatewayBackend
│   (AI type)   │       │   (AI type)   │         │  (MCP type)   │
└───────────────┘       └───────────────┘         └───────────────┘

Key Resources

AgentgatewayParameters

Configures the agentgateway proxy deployment (logging, resources).

AgentgatewayBackend

Routes to LLM providers or MCP servers. Two types:

• AI Backend: spec.ai.provider with host/port and provider type
• MCP Backend: spec.mcp.targets for MCP server endpoints

AgentgatewayPolicy

Traffic policies targeting Gateway or HTTPRoute:

• Rate limiting (requests and tokens)
• CORS configuration
• Authentication

Common Patterns

Local Ollama Backend

apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayBackend
metadata:
  name: ollama-backend
  namespace: kgateway-system
spec:
  ai:
    provider:
      openai:                    # Ollama is OpenAI-compatible
        model: "llama3.2"
      host: "host.internal"      # OrbStack host access
      port: 11434

Cloud Provider Backend (with Secret)

Create the secret first (preferred — avoids keys in output):

kubectl create secret generic openai-api-key \
  --from-literal=api-key="${OPENAI_API_KEY}" \
  -n kgateway-system --dry-run=client -o yaml | kubectl apply -f -

Then create the backend:

apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayBackend
metadata:
  name: openai-backend
  namespace: kgateway-system
spec:
  ai:
    provider:
      openai:
        model: "gpt-4o-mini"
  policies:
    auth:
      secretRef:
        name: openai-api-key

Version Awareness

The agent and /gw-versions command check for updates live:

• Fetches latest releases from agentgateway/agentgateway
• Compares against installed helm chart versions
• Warns about docs-vs-reality drift (features documented on main may not exist in released versions)
• Always inspect installed CRD to verify field availability

Troubleshooting Quick Reference

Error	Cause	Solution
backends required DNS resolution which failed	Backend hostname not resolvable	Use host.internal for OrbStack
processing failed: failed to parse request: EOF	GET request to LLM endpoint	LLM backends expect POST with JSON body
model 'X' not found	Wrong model in backend config	Update AgentgatewayBackend with correct model

Debug Commands

# Check agentgateway controller logs
kubectl -n kgateway-system logs -l app.kubernetes.io/name=agentgateway --tail=50

# Check backends and policies
kubectl get agentgatewaybackend,agentgatewaypolicy -n kgateway-system

# Describe a backend for detail
kubectl describe agentgatewaybackend <name> -n kgateway-system

Reference Files

When you need more detail, load these reference files:

• references/provider-backends.md: All 7 provider configurations (Ollama, OpenAI, Anthropic, Gemini, Vertex AI, Azure OpenAI, Bedrock), token lifecycle, secret management
• references/resource-patterns.md: AgentgatewayBackend, AgentgatewayPolicy, AgentgatewayParameters YAML examples
• references/mcp-routing.md: MCP server backend and routing patterns
• references/external-processing.md: External Processing (ExtProc) for custom request/response mutations and routing
• references/helm-lifecycle.md: agentgateway installation, upgrade, rollback, version checking
• references/lessons-learned.md: API structure gotchas, CORS pitfalls, local vs cloud deployment patterns

Installation Requirements

• kgateway v2.2.x+ required for agentgateway support (pre-v2.3.0)
• Registry: cr.agentgateway.dev (agentgateway)
• Enable agentgateway in kgateway: --set controller.enableAgentgateway=true (pre-v2.3.0)
• See references/helm-lifecycle.md for complete installation sequence