gpt-load

Gpt-load is an open‑source, self‑hosted AI gateway that presents a single HTTP endpoint for accessing many different large‑language‑model and multimodal services. The project is distributed as a compiled Go binary and is released under the MIT License, so you can run it on any Linux server you control.

The gateway includes 35 built‑in channels that map to official APIs, cloud platforms and model‑service providers such as OpenAI, Anthropic, Gemini, Azure OpenAI, AWS Bedrock, Google Vertex AI, DeepSeek, Moonshot AI and others. Each channel can be reached through one of four conversation protocols: OpenAI Chat Completions (POST /v1/chat/completions), OpenAI Responses (namespace‑preserving pass‑through), Anthropic Messages (POST /v1/messages) and Gemini (generateContent and streaming variants). In practice, a client that already knows how to call an OpenAI‑compatible endpoint can keep the same request body and headers, change only the base URL to the gateway's address, and add the AccessKey header that authorises the call.

Credentials for the upstream services are stored locally, encrypted on disk, and can be managed in SQLite, MySQL or PostgreSQL. You can add API keys or subscription‑account credentials for providers such as Codex, Claude, Antigravity or Grok. The gateway pools these credentials and applies a scheduler that decides which credential to use for each request. Scheduling rules include weighted rotation, retry on failure, cooldown periods after errors and automatic blacklisting of a credential that repeatedly fails. This means that if a particular key hits a rate limit, the gateway will switch to another key without the client needing to implement its own fallback logic.

Every request that passes through the gateway is logged. The logs contain health information, the selected upstream route, usage counts and per‑model cost estimates down to the individual credential. The management UI, which is built into the same Go binary, lets you inspect these logs, view usage summaries and monitor the health of each channel. Because the gateway never forwards credentials to third parties, you retain full control over data privacy.

Typical inputs are JSON payloads that match the chosen conversation protocol; outputs are the corresponding responses from the upstream model, returned unchanged to the client. The system is suited for developers who run multiple AI‑enabled applications, need a unified endpoint, and want automatic credential rotation and observability.