Bifrost

Bifrost is a backend-only, provider-agnostic AI gateway. Your clients keep talking the OpenAI (and Anthropic) wire format they already know; the gateway routes each request to the right upstream — OpenAI, Anthropic, Google, Azure, Vercel AI Gateway, OpenAI-compatible providers, or your own custom models — behind a single public model catalog with one auth, rate-limit, routing, logging, and cost layer.

The pitch in one line: point your existing OpenAI SDK at Bifrost and change nothing else.

from openai import OpenAI

client = OpenAI(base_url="http://localhost:4000/v1", api_key="<virtual-key>")
client.chat.completions.create(model="general", messages=[{"role": "user", "content": "Hi"}])

model is a public name you define (general, image-default, …), not a provider model id. Behind it sits a pool of one or more deployments that the gateway load-balances, retries, and falls back across.

Start here

New to the project? Read these in order:

1. Quickstart — from git clone to your first successful response in a few minutes.
2. Concepts — the mental model: public models, deployments, adapters, transports.
3. Architecture — the same request, traced through every layer in precise order.
4. First deployment — wire up one real provider, verified before you save it.

Setup

• Setup — requirements, environment, secrets, and choosing a database and Redis.
• First deployment / Creating deployments — the minimal path, and the full reference across every provider and operation type.
• Environment variables — every variable, its default, and what it controls.

Deploy

• Deployment — the Docker Compose stack and per-platform guides (Coolify, Portainer, Dokploy, Linux).

Inference API

• API overview — auth, error shape, and OpenAPI import, shared across every endpoint.
• Chat completions, Responses, Messages — the three text contracts over one canonical core.
• Images, Embeddings, Reranking, Audio transcriptions — the other operations.
• Streaming — the SSE shape shared across every streamable endpoint.
• Model discovery — GET /v1/models and its per-deployment sibling.
• Headers — every request/response header in one table.

Routing & resilience

• Routing — the eight balancing strategies and how each is measured.
• Parameter policy — what happens when a request asks for something a model doesn't support.
• Fallbacks — fallback chains per public model: reasons, retries, lifecycle.
• Caching — the opt-in response cache and its headers.

Access & governance

• Virtual keys — client keys, scopes, budgets, and rate limits.
• Security — the auth model, credential encryption, and what's redacted by default.
• Cost accounting — where pricing comes from and how spend is tracked.

Model catalog

• Model catalog — the shape of catalog.json, per-operation profiles, pricing, and custom models.
• Reasoning — the canonical effort ladder and every provider's native reasoning mechanism.

Providers

• Providers — every built-in adapter: credentials, default transport, and quirks, from OpenAI to Vercel AI Gateway to any OpenAI-compatible API.

Extend

• Runtime extensions — admin-uploaded ESM extensions, hooks, versioning, and failure handling.

Operate

• Operations — the runbook: health probes, secrets, data lifecycle, backups, shutdown.
• Observability — structured logs, durable operation logs, and OpenTelemetry.
• Production checklist — a condensed pre-launch checklist.
• Upgrades — migrations, breaking changes, and safe rollback.
• Troubleshooting — error shape, status codes, and known issues with exact fixes.

Reference

• Admin API reference — every admin route, grouped by area.
• Testing — test layers, common helpers, and the no-real-providers guarantee.
• Glossary — the project's canonical names. The source of truth for code, SQL, OpenAPI, logs, and docs.

Non-Markdown artifacts

• OpenAPI: apps/gateway/openapi.yaml
• JSON Schemas: schemas