For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.
Providers
Configure agentgateway for first-class provider shortcuts and OpenAI-compatible fallbacks.
Learn how to configure agentgateway for a particular LLM.
First-class providers
Prefer the dedicated provider pages for upstreams that have a built-in preset or provider type. These providers include built-in host, path, and request-format defaults.
OpenAI-compatible providers
Many providers expose the OpenAI Chat Completions API but do not have a first-class type in the AgentgatewayBackend API. Examples include Cohere, DeepSeek, Groq, Mistral, Perplexity, Together AI, and xAI. Configure these providers with the ai.provider.openai shape. For the list of supported hosts and paths, see OpenAI-compatible providers.
Custom providers
Use custom providers for unsupported, non-standard, or self-hosted targets when you need to declare formats, upstream paths, or backend targets explicitly.
Tip
Compare provider prefixes, exact names, wildcards, and custom names in Choose how to expose models.
Amazon Bedrock
Configure Amazon Bedrock as an LLM provider for agentgateway.
Anthropic
Configure Anthropic Claude as an LLM provider for agentgateway.
Azure
Configuration and setup for Azure AI services provider
Gemini
Configure Google Gemini as an LLM provider for agentgateway.
GitHub Copilot
Configure GitHub Copilot with credentials from a Kubernetes Secret.
OpenAI
Configure OpenAI as an LLM provider for agentgateway.
OpenAI-compatible providers
Configure OpenAI-compatible providers with AgentgatewayModel presets or AgentgatewayBackend …
TypeSafe
Configure TypeSafe Jev as an LLM provider for agentgateway.
Vertex AI
Configure Google Vertex AI as an LLM provider for agentgateway.
Custom providers
Configure unsupported, self-hosted, and non-standard LLM providers with explicit API formats, paths, …
Nebius Token Factory
Configure Nebius Token Factory as an LLM provider for agentgateway.
Ollama
Configure agentgateway to route LLM traffic to Ollama for local model inference
vLLM
Configure agentgateway to route traffic to vLLM for self-hosted model inference
Multiple endpoints
Configure access to multiple OpenAI API endpoints like chat completions, embeddings, and models …
Backend tunnel proxy
Use the tunnel policy to route backend connections through an HTTP proxy for JWKS fetching and other …
Audio models
Serve self-hosted audio models such as Voxtral with an AgentgatewayModel, and record the usage and …
Mock LLM with httpbun
Set up httpbun as a mock OpenAI-compatible LLM backend for testing without API keys.
