Kong AI Gateway: the AI plugins on top of the gateway you run

What Kong AI Gateway is

Kong describes it as a connectivity and governance layer for AI-native applications built on top of Kong Gateway. That phrasing is accurate and worth taking literally: there is no separate Kong AI Gateway product to install. You run Kong Gateway, you enable AI plugins on routes and services, and the result is what the marketing calls an AI gateway.

The consequence is a hard fork in the decision. If your platform team already runs Kong in front of your APIs, adding LLM traffic to it is a configuration change on infrastructure you operate, monitor, and know how to debug. If you do not run Kong, adopting it to route model calls means adopting an API gateway, and that is a much larger commitment than pointing a base URL at a hosted proxy.

The plugin catalogue

There are roughly two dozen AI plugins. They compose: a single route can proxy to a load-balanced pool, strip PII on the way in, check the prompt against a semantic deny list, serve from a semantic cache, and score the response with an LLM judge.

Providers AI Proxy speaks

The AI Proxy plugin has a min_version of Kong Gateway 3.6. Its provider list as of August 2026:

• OpenAI, Azure OpenAI
• Anthropic
• Amazon Bedrock
• Gemini, Vertex AI
• Cohere, Mistral, Hugging Face
• Llama, Ollama, vLLM (self-hosted)
• xAI, Alibaba Cloud DashScope, Cerebras, DeepSeek
• Databricks

The self-hosted three (Llama, Ollama, vLLM) are the ones that make Kong interesting for regulated deployments: the same policy layer covers a commercial API and your own inference server, with no code change in the client.

A worked configuration

Declarative config, applied with deck or via the Admin API. This routes one path to Anthropic through AI Proxy, with a prompt guard and rate limiting on top.

_format_version: "3.0"

services:
  - name: llm
    url: http://localhost:32000
    routes:
      - name: chat
        paths:
          - /chat
        plugins:
          - name: ai-proxy
            config:
              route_type: llm/v1/chat
              auth:
                header_name: x-api-key
                header_value: ${ANTHROPIC_API_KEY}
              model:
                provider: anthropic
                name: claude-sonnet-5

          - name: ai-prompt-guard
            config:
              deny_patterns:
                - ".*ignore (all )?previous instructions.*"
              allow_all_conversation_history: false

          - name: ai-rate-limiting-advanced
            config:
              llm_providers:
                - name: anthropic
                  limit: [200000]
                  window_size: [3600]

The shape to notice: route_type normalises the request into Kong's own schema, which is what lets AI Proxy Advanced load balance the same route across two providers later without touching the client. That normalisation is the actual value of the plugin, more than the auth handling.

What it costs

Self-managed Kong Gateway Enterprise is a separate licensing conversation from Konnect. If you already have that contract, the AI plugins may be inside it; confirm with your account team rather than assuming from the Konnect page.

When Kong is right

Right when

• Kong is already your API gateway. The marginal cost of governing LLM traffic there is a config change.
• You need one policy engine over REST, gRPC, LLM, MCP, and agent-to-agent traffic.
• Your compliance boundary means the gateway must run in your own network, in your own Kubernetes.
• You mix commercial APIs with self-hosted vLLM or Ollama and want identical policy over both.

Wrong when

• You do not already run Kong. Adopting an API gateway to route model calls is a large answer to a small question.
• You want a hosted wallet. Kong proxies your provider accounts; it does not sell you tokens.
• Your model catalogue is wide and your traffic is thin. Per-model pricing on Plus punishes exactly that shape.
• You need it working this afternoon. LiteLLM or a hosted gateway is minutes; Kong is a platform project.

Questions people ask

Keep reading

Continue through LLM gateways and routing with Portkey review: the gateway is now Prisma AIRS AI Gateway.