GammaInfra API documentation

Last updated: 20 May 2026

Overview

GammaInfra is an intelligent LLM routing engine. The router classifies every prompt by task and dispatches to the best-fit model across every major LLM provider — delivered through one API.

If you already use OpenAI or any OpenAI-compatible SDK, switching to GammaInfra is a one-line change — update the base_url, keep everything else the same.

Quickstart

1. Get an API key

Sign up at gammainfra.com — email + password, one-time verification link, no credit card. New accounts come with $3.00 of free credit, enough to try the router end-to-end. Your API key is shown once after you click the verification link — store it somewhere safe. Need more keys later? Issue and revoke them from the dashboard.

2. Make your first call

curl -s -X POST https://api.gammainfra.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-gammainfra-..." \
  -d '{
    "model": "gammainfra/auto",
    "messages": [{"role": "user", "content": "Explain transformers in one paragraph."}]
  }'

The response is identical to OpenAI’s format. gammainfra/auto lets the router pick the best model for your prompt.

3. Drop-in replacement

from openai import OpenAI

client = OpenAI(
    api_key="sk-gammainfra-...",
    base_url="https://api.gammainfra.com/v1",
)

response = client.chat.completions.create(
    model="gammainfra/auto",
    messages=[{"role": "user", "content": "Hello"}],
)
print(response.choices[0].message.content)

The same pattern works with LangChain, LlamaIndex, and any other OpenAI-compatible library.

Ecosystem compatibility

If your code targets an OpenAI-compatible router, the migration is usually two strings — base_url and api_key. Both /v1/* and /api/v1/* prefixes are mounted with identical responses, so SDKs that hard-code either path keep working unchanged.

from openai import OpenAI
client = OpenAI(
    api_key="sk-gammainfra-...",
    base_url="https://gammainfra.com/api/v1",
)

Body fields

GammaInfra accepts the common ecosystem-extension request fields. The ones below change behavior; everything else is accepted silently for forward compatibility.

Model name aliases

GammaInfra uses the conventional vendor/slug format and accepts the common ecosystem aliases directly:

Unknown :suffix variants are stripped silently for forward-compat. For the full catalogue of native model IDs, see Model names below.

Compatibility endpoints

The auxiliary endpoints SDKs commonly call are mounted under both prefixes and return ecosystem-compatible JSON.

Headers

Send HTTP-Referer and X-Title on every request — GammaInfra stores them with each request log so per-app analytics work consistently. Both are best-effort and entirely optional.

What's not supported

• Web search (plugins:[{id:"web"}] and the :online variant) — not implemented. We reject explicitly so your prompts don't silently degrade.
• Free tier (:free) — we don't offer one. Every account starts with $3.00 of free balance; top up from /billing/checkout.
• Per-key spend limits (limit, limit_remaining, limit_reset) — not yet implemented; /key returns null for these fields.
• OAuth/PKCE key exchange — sign in via the dashboard and copy a key.

Authentication

Every authenticated request carries a Bearer token. The public endpoints are /v1/models, /v1/status, /health, /ready, and the signup/login routes; everything else needs a valid key.

Authorization: Bearer sk-gammainfra-...

Keys are prefixed sk-gammainfra-. The plaintext is only returned on creation — GammaInfra stores a bcrypt hash. Create additional keys or revoke old ones from the dashboard.

Smart routing

Send model: "gammainfra/auto" and the router classifies your prompt into one of 8 task labels (plus 2 deterministic capability flags), then dispatches to the best-fit model for that type. If the primary model is unavailable, GammaInfra falls back through a chain of 3–4 models automatically.

Capability flags are decided up-front from the request body, never from prompt text:

For everything else the prompt text is classified into one of these 8 labels:

Model names

Smart aliases (recommended)

Bare model names (logical)

Type a bare model name and GammaInfra's router picks the best endpoint that serves it. Useful when the same model is hosted by more than one provider (e.g. Claude Opus is reachable via the native Anthropic API and Amazon Bedrock).

claude-opus-4-7
claude-opus-4-6
claude-sonnet-4-6
claude-haiku-4-5
nova-pro
nova-2-lite
gpt-5-mini
gpt-5.4-mini
deepseek-v4-pro
mistral-large-2512
llama-3.3-70b-versatile
gemini-3.1-pro-preview
grok-4-1-fast-non-reasoning

Bare names that aren't in the registry return 404 model_not_found (no silent fallback). Use X-GammaInfra-Routing: literal to disable cross-host routing and pin the first registered endpoint instead.

Pin a specific model

Prefix any model with its provider slug:

openai/gpt-5.4
openai/gpt-5.4-mini
openai/gpt-5.4-nano
openai/gpt-5-mini
anthropic/claude-opus-4-6
anthropic/claude-sonnet-4-6
anthropic/claude-haiku-4-5
google/gemini-3.1-pro-preview
google/gemini-3-flash-preview
google/gemini-2.5-pro
google/gemini-2.5-flash
mistral/mistral-large-2512
mistral/mistral-small-2603
mistral/codestral-2508
mistral/devstral-2512
groq/llama-3.3-70b-versatile
groq/llama-3.1-8b-instant
groq/qwen/qwen3-32b
deepseek/deepseek-v4-pro
deepseek/deepseek-v4-flash
# Legacy V3 slugs — still routable via direct pin, retire 2026-07-24:
# deepseek/deepseek-chat, deepseek/deepseek-reasoner
grok/grok-4.20-0309-reasoning
grok/grok-4-1-fast-reasoning
grok/grok-4-1-fast-non-reasoning
bedrock/us.anthropic.claude-opus-4-7
bedrock/us.anthropic.claude-opus-4-6-v1
bedrock/us.anthropic.claude-sonnet-4-6
bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
bedrock/meta.llama3-70b-instruct-v1:0
bedrock/mistral.mistral-large-2402-v1:0
bedrock/us.amazon.nova-pro-v1:0
bedrock/us.amazon.nova-2-lite-v1:0

Note on Bedrock IDs: Most Bedrock models require the us. cross-region inference profile prefix (Anthropic Claude, Amazon Nova). A few older models (Meta Llama 3, Mistral Large 24.02) use the bare ID without the prefix. The exact strings above are what AWS Bedrock accepts; copy verbatim. Bedrock's catalog of newer Meta and Mistral models lags behind these providers' direct APIs — for the latest Llama and Mistral, use groq/llama-3.3-70b-versatile or mistral/mistral-large-2512 respectively.

For the full, authoritative list:

curl -s https://api.gammainfra.com/v1/models | jq .

Streaming

Streaming works exactly like OpenAI — set stream: true and read Server-Sent Events. All providers are normalised to the OpenAI SSE format, so your existing code works unchanged.

stream = client.chat.completions.create(
    model="gammainfra/auto",
    messages=[{"role": "user", "content": "Write a haiku about distributed systems."}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="", flush=True)

Headers

Request headers

Response headers

Preference precedence

You can express routing preference through five different channels — header, body, model suffix, or model shortcut. When more than one is set, the most-specific wins:

1. X-GammaInfra-Preference: latency — always wins (latency is orthogonal to cost/quality).
2. Body provider.sort (ecosystem-compat) — price → cost, throughput/latency → latency.
3. X-GammaInfra-Cost-Quality header — continuous 0.0…1.0 dial.
4. X-GammaInfra-Preference: quality | cost header — legacy preset.
5. Model-slug variant (:nitro → latency, :floor → cost) or the gammainfra/fast / gammainfra/cheap shortcuts.
6. Default: quality.

Balance & pricing

• Your balance is denominated in USD — no in-platform credits unit, no conversion math. The dashboard, ledger, and GET /v1/billing/balance all return dollars (e.g. 2.4738 means $2.4738 of remaining balance).
• New accounts start with $3.00 of free balance.
• 0% markup on tokens — per-request provider cost is passed through at list prices.
• 5% fee on managed top-ups (3% launch-window rate during the first 60 days).
• Minimum top-up: $10. No subscription. Balance never expires.

Approximate cost per 1M tokens

Costs above are provider list prices — GammaInfra passes them straight through. The only GammaInfra fee is the 5% we charge when you top up credits (3% during the launch window). For the full cost table and legal terms, see the Terms of Service.

Reasoning tokens on gpt-5 and DeepSeek V4

OpenAI’s gpt-5 family and DeepSeek’s V4 reasoner (deepseek-v4-pro) bill hidden “reasoning tokens” in addition to the visible output. Reasoning tokens are the model’s chain-of-thought and are not returned in the response but are counted in usage.completion_tokens.

GammaInfra silently caps gpt-5 reasoning at max_completion_tokens=2048 when the caller omits the parameter, and picks a conservative reasoning_effort based on the router’s logical label (chat → low, code/summarize → medium, reasoning/math → high). This prevents the “‘hi’ burned 320 reasoning tokens” pathology from reaching your bill, but you should still budget 2–4× visible output tokens for gpt-5-family calls in batch sizing. Inspect usage.completion_tokens_details.reasoning_tokens in any response to see the split.

Prompt caching and your bill

When a repeated prefix is served from cache, the provider charges a reduced rate for those tokens. GammaInfra passes the discount straight through — you are billed at the provider’s actual cache-read rate, never the full input rate. Your X-GammaInfra-Cost-USD header reflects the all-in cost including any cache-read discounts and cache-write premiums on the same call. For per-direction detail, see X-GammaInfra-Cache-Read-Tokens and X-GammaInfra-Cache-Write-Tokens in the response headers above.

Cache writes carry a small premium over standard input pricing for Anthropic and Bedrock (the provider charges extra to prime the cache). On first use the total cost is slightly higher; on repeated calls the cache-read savings more than offset the initial write. Set X-GammaInfra-Cache: off if you are sending one-off requests and don’t want the write overhead.

Check your balance

curl -s https://api.gammainfra.com/v1/billing/balance \
  -H "Authorization: Bearer sk-gammainfra-..."

{"balance_usd": 0.97, "customer_id": "..."}

Top up

Top up your balance from dashboard.gammainfra.com → Top up. You’ll be redirected to Stripe’s hosted checkout and back to your dashboard once payment clears. Card data is handled by Stripe — GammaInfra never sees it. Amount range: $5 – $1000; your balance updates within seconds of Stripe’s confirmation.

Bring your own key (BYOK)

Optional. By default GammaInfra uses its own provider API keys on your behalf — one GammaInfra key, every model. If you already have a direct relationship with a provider, add your own key at dashboard.gammainfra.com → Provider Keys and GammaInfra will route requests to that provider through your key instead.

• Your key is stored encrypted at rest (Fernet).
• The provider bills you directly (on your account with them) for the tokens a BYOK-routed request consumes. GammaInfra only charges a small per-request routing fee from a separate prepaid BYOK balance — see BYOK pricing below.
• Revoking or deleting your key falls back to managed routing (if the provider offers managed access) or skips that provider.
• Supported providers: openai, anthropic, google, mistral, groq, deepseek, grok.

Add a key

curl -s -X POST https://api.gammainfra.com/v1/provider-keys \
  -H "Authorization: Bearer sk-gammainfra-..." \
  -H "Content-Type: application/json" \
  -d '{"provider_name": "openai", "api_key": "sk-..."}'

List your keys

curl -s https://api.gammainfra.com/v1/provider-keys \
  -H "Authorization: Bearer sk-gammainfra-..."

Delete a key

curl -s -X DELETE https://api.gammainfra.com/v1/provider-keys/openai \
  -H "Authorization: Bearer sk-gammainfra-..."

Or manage all of this from dashboard.gammainfra.com → Provider Keys.

BYOK pricing — separate prepaid balance

BYOK traffic uses its own prepaid balance, distinct from your managed credits. Top it up from the dashboard's BYOK Balance tab or via POST /v1/billing/byok/checkout (minimum $5, no top-up fee). Each BYOK-routed request deducts a small per-request fee:

• 2% of the retail provider cost (standard rate).
• 1% during the launch window (first 60 days).
• When the BYOK balance hits $0, BYOK-routed requests return 402 byok_balance_empty — top up to resume. Managed credits are not touched.
• You'll receive one low-balance email when your BYOK balance drops below 20% of your last top-up (7-day cooldown between reminders).

curl -s -X POST https://api.gammainfra.com/v1/billing/byok/checkout \
  -H "Authorization: Bearer sk-gammainfra-..." \
  -H "Content-Type: application/json" \
  -d '{"amount_usd": 25.0}'

Check your BYOK balance:

curl -s https://api.gammainfra.com/v1/billing/byok/balance \
  -H "Authorization: Bearer sk-gammainfra-..."

Prompt caching

Repeating the same system prompt, tool definitions, or conversation prefix across multiple calls is the most common source of avoidable spend. GammaInfra automatically caches these prefixes on providers that support it, so cache-hit tokens on follow-up calls cost a fraction of full input tokens.

How it works

GammaInfra detects cacheable prefixes and, where supported, injects cache_control breakpoints before the request leaves the gateway:

• Anthropic (Claude): Gateway auto-injects breakpoints on the system prompt and tools prefix when the combined token count meets the minimum threshold (1024 tokens for most models, 2048 for Claude Haiku). You do not need to add cache_control markers yourself — the gateway does it for you.
• OpenAI, DeepSeek, Google Gemini: Caching is automatic on the provider side. The gateway reads the discounted token counts from the response and bills at the provider’s actual cache-read rate.
• Amazon Bedrock: Cache read tokens are billed at the provider’s discounted rate. Bedrock auto-inject is a planned fast-follow.
• Groq, Mistral, xAI: Do not report cache tokens. Always billed at standard input rates.

Controlling cache behaviour

Send the X-GammaInfra-Cache request header to override the default:

Unknown values for X-GammaInfra-Cache are silently ignored and fall back to the default — the header never returns a 400 error.

Verifying cache hits

Check the response headers on any call:

X-GammaInfra-Cache-Mode: auto
X-GammaInfra-Cache-Read-Tokens: 4096
X-GammaInfra-Cache-Write-Tokens: 512
X-GammaInfra-Cost-USD: 0.000041

X-GammaInfra-Cache-Read-Tokens is the number of input tokens served from cache on this call. X-GammaInfra-Cache-Write-Tokens is the number written to cache (priming cost). Both are absent when zero. X-GammaInfra-Cost-USD is always the all-in cost, inclusive of any cache-read discounts and write premiums.

When caching saves money

Caching wins when the same prefix is reused across at least two calls. On the first call GammaInfra writes the prefix to cache (small premium); on subsequent calls those tokens are served at the provider’s cache-read rate (significant discount). The break-even point is reached quickly for system prompts longer than ~1k tokens that are reused across many requests.

For truly one-off requests where the prefix will never repeat, set X-GammaInfra-Cache: off to skip the write cost.

Error codes

Error responses use a consistent JSON shape:

{
  "error": {
    "message": "Human-readable description",
    "type": "error_type",
    "code": "machine_readable_code",
    "request_id": "uuid"
  }
}

Rate limits

• Per-API-key cap: 240 requests per minute on /v1/chat/completions and /v1/completions (and their /api/v1/* aliases). Sliding 60-second window. Static endpoints (/v1/models, /v1/status, /health, /ready) don't count.
• Every response carries X-RateLimit-Limit, X-RateLimit-Remaining, and X-RateLimit-Reset (Unix seconds). When you hit the cap you get 429 with a Retry-After header in seconds.
• Provider-side rate limits also pass through as 429 with any Retry-After the provider returned.
• If you expect sustained traffic above the per-key cap, email hello@gammainfra.com — we raise caps for steady-state volume on request.

Status

Live per-provider uptime, latency, and error counts are published:

• status.gammainfra.com — human-readable HTML dashboard, auto-refreshes every 30 s
• GET /v1/status — same data as JSON, safe to poll from your own monitoring

Both endpoints are public (no auth). Each provider is marked operational, degraded, or outage based on the rolling 24 h request log plus a live health-check ping.

Support

Email support@gammainfra.com, or join our Discord and open a ticket in #support. Include the X-GammaInfra-Request-Id response header from any failing request — it lets us trace the exact path the request took through the router.

For policy and billing terms, see Terms and Privacy.

FAQ

Common developer questions about the API. For the conceptual overview see the FAQ on the landing page.