An open-source AI gateway written in Rust: it includes an admin console in a single binary, sits between applications and various model services, and unifies APIs, routing, quotas, caching, and safety guardrails.
Stars 376 (+41 / 24h)
Reasons
- The backend can connect to OpenAI, Azure, Anthropic, Bedrock, Vertex, Gemini, Mistral, DeepSeek, xAI, and any OpenAI / Anthropic-compatible service (such as vLLM), with automatic conversion between client protocols
- Request-determined fallback chains, weighted traffic splitting, lowest-cost / lowest-latency routing, retries, hedging, and circuit breaking, with results logged in request logs
- Dollar or token budgets and rate limits across four levels: organization, team, project, and key; guardrails cover PII, secrets, JSON schema, tool permissions, and prompt injection detection
- The author claims that on the same 4 vCPU machine, at 5000 requests per second, the median overhead is 0.53 ms, whereas Bifrost is 19.7 ms and LiteLLM fails entirely
Usage
- docker run -p 8080:8080 -p 8081:8081 -v antirion-data:/var/lib/antirion eliasbiondo/antirion; the first startup prints the console address and a one-time admin token
- Add model services and models in http://localhost:8081/ui/, create an API key, and then point the base URL of any OpenAI SDK to http://localhost:8080/v1
Risks
- The performance figures are from the author's own benchmarks, with request logging and caching disabled in the minimal configuration; performance in production configurations is unknown
- The repository was only created in mid-10; production readiness remains to be seen; the license is Apache-2.0