ai-gateway

Summary

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

Release history

1. v1.5.0 Aug 29, 2026 · issue 011

   ai-gateway v1.5.0 makes the gateway importable as a Go library via new run.Main() and run.Run() APIs

   └──▷ GET THIS VERSION
   copy

   $ git clone --branch v1.5.0 https://github.com/ferro-labs/ai-gateway.git
   # already have the repo? check out this version:
   $ git checkout v1.5.0

   └──▷ USE IT
   Embed the gateway in a Go service so it shuts down cleanly when a parent context is cancelled — useful for test harnesses or multi-component binaries.
   go copy

   import (
       _ "your-org/your-plugin"
       "github.com/ferro-labs/ai-gateway/run"
   )
   
   func main() {
       ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt)
       defer cancel()
       if err := run.Run(ctx); err != nil {
           log.Fatal(err)
       }
   }

   • ›Adds run.Main() and run.Run(ctx, opts...) in the new public run package, letting Go programs embed the full ferrogw server — run.Run returns startup and listen errors instead of exiting, and honours context cancellation for graceful shutdown identical to SIGTERM.
   • ›Custom binaries can now be built by blank-importing plugins and calling run.Main(), enabling a plugin-composition workflow without forking the gateway binary.
2. v1.4.2 Aug 10, 2026 · issue 011

   ai-gateway v1.4.2 adds provider registration aliases, an embeddable HTTP facade, and build provenance on GET /health

   └──▷ GET THIS VERSION
   copy

   $ git clone --branch v1.4.2 https://github.com/ferro-labs/ai-gateway.git
   # already have the repo? check out this version:
   $ git checkout v1.4.2

   └──▷ TRY IT
   Confirm exactly which build is serving traffic without shelling into the container — useful after a rolling deploy.
   copy

   $ curl -s https://gateway.internal/health | jq '{version, commit, built}'

   Bind two sets of provider credentials to distinct routing targets so tenants can be isolated by alias rather than by separate gateway instances.
   go copy

   gw.RegisterProviderAs(openaiProvider, "openai-team-a")
   gw.RegisterProviderAs(openaiProvider, "openai-team-b")

   • ›Adds Gateway.RegisterProviderAs to register one provider under a distinct routing target, allowing multiple credentials for the same canonical provider while preserving all optional capabilities (streaming, embeddings, images, rerank, moderation, audio, discovery, batch, Responses, and generic pass-through).
   • ›Exposes httpgateway package to embedding applications, giving embedders access to Files/Batches, Responses, and generic pass-through handlers while keeping their own authentication and tenant policy middleware.
   • ›GET /health now returns version, commit, and built build metadata alongside provider status, sourced from internal/version (defaults: dev / none / unknown for unstamped local builds).
3. v1.4.0 Aug 7, 2026 · issue 011

   ai-gateway v1.4.0 adds native rerank, moderation, audio, Files, Batches, and Responses API routing with full gateway lifecycle support.

   └──▷ GET THIS VERSION
   copy

   $ git clone --branch v1.4.0 https://github.com/ferro-labs/ai-gateway.git
   # already have the repo? check out this version:
   $ git checkout v1.4.0

   └──▷ USE IT
   Declare a preview or self-hosted model ID so it appears in routing and /v1/models without waiting for catalog updates.
   yaml copy

   targets:
     - id: my-azure-preview
       provider: azure-openai
       base_url: https://my-resource.openai.azure.com
       models:
         - gpt-4o-2026-preview
         - gpt-4o-mini-regional

   Enable the OpenAI Batches API by pointing the gateway at a batch-capable backend so /v1/files* and /v1/batches* resolve without a 501.
   yaml copy

   batch_target: openai-prod
   
   targets:
     - id: openai-prod
       provider: openai
       api_key: ${OPENAI_API_KEY}

   • ›Adds POST /v1/rerank (Cohere v2 contract) as a natively routed surface supporting cohere, together, deepinfra, nvidia-nim, and bedrock — with top_n normalised across providers (0 caps to no results, negative values are rejected).
   • ›Adds POST /v1/moderations (OpenAI contract) as a natively routed surface supporting openai and mistral.
   • ›Adds POST /v1/audio/transcriptions and POST /v1/audio/translations (multipart upload, 25 MiB cap) as natively routed surfaces supporting openai, azure-openai, groq, together, sambanova, deepinfra, mistral, and fireworks.
   • ›Adds POST /v1/audio/speech (JSON in, binary audio out; input capped at 4096 characters) as a natively routed surface supporting openai, azure-openai, groq, together, deepinfra, and mistral.
   • ›Adds targets[].models field to declare models a target serves that are absent from the catalog or live discovery — new catalog IDs, regional/preview names, or self-hosted deployments — making them available in routing and GET /v1/models.
   +8 moreshow less

   • ›Adds batch_target config key to forward /v1/files* and /v1/batches* transparently to a configured backend (openai, azure-openai, groq, novita, qwen); returns 501 when batch_target is unset.
   • ›Adds POST /v1/responses (Responses API) as a governed, priced surface with plugins, guardrails, circuit breaker, concurrency, and request log; stateful sub-routes pin to responses_target and return 501 when it is unset.
   • ›All four new natively routed surfaces (rerank, moderations, audio, responses) carry the full gateway lifecycle — targets, routing strategy, plugins, circuit breaker, per-target concurrency, metrics, and request logging — replacing the generic pass-through.
   • ›Extends image generation support to gemini (via generateContent), deepinfra, and together; extends embeddings support to azure-foundry.
   • ›Adds a Tracing page to the dashboard over the gateway's OpenTelemetry output.
   • ›Adds a deploy/ compose stack running the gateway, a collector, a tracing backend, and a mock upstream for end-to-end observability without a provider key.
   • ›Agentic MCP tool loops now run before_request plugins (guardrails, rate limiters) on every turn, not only the first — transform and logging/metrics plugin types are deliberately excluded from per-turn execution.
   • ›Unifies routing internals across chat, streaming, embeddings, and image generation so retry (targets[].retry), circuit breaking, error classification, metrics, and request logging behave identically on every surface.
   └──▷ BREAKING ON UPGRADE

   • !targets[].retry was previously honoured only under fallback routing mode; it is now honoured under all modes, so a target that always fails will produce attempts upstream calls where it previously produced one.
   • !render.yaml now sets GATEWAY_ENV=production, enabling startup checks that refuse to boot when ALLOW_UNAUTHENTICATED_PROXY=true or CORS_ORIGINS contains *; a Render service carrying either setting will fail its next deploy.
   • !Outbound HTTP clients no longer follow upstream 3xx redirects — the redirect is returned to the caller as-is; requests that previously succeeded via redirect will now fail until the provider base URL is updated to the redirect target.
   • !Withheld config map keys in GET /admin/config are no longer returned verbatim; each entry is now replaced with [REDACTED_KEY_<n>] (sorted, stable index). A PUT body containing placeholder key names is rejected. Affected maps: mcp_servers[].env, mcp_servers[].headers, observability.exporters[].config, observability.tracing.headers, and undeclared plugin settings.