Route your model traffic through ModelGate.

Quickstart

Four steps, about five minutes:

1. Open an account and sign in to the dashboard.
2. Add a provider key. Under Provider credentials, paste your own OpenAI, Anthropic or Google key. It’s encrypted at rest and never shown again.
3. Create a ModelGate API key under API keys. Copy it — it’s shown once.
4. Point your SDK at ModelGate by changing the base URL (below). Your existing code keeps working.

Authentication

Every request carries your ModelGate API key, in either of two headers — use whichever your client makes easier. OpenAI SDKs send their api_key as a Bearer token, so passing your ModelGate key there is enough. The key identifies your project; provider keys are looked up server-side, so no provider secret ever leaves your dashboard.

Authorization: Bearer mg_…
# or
x-api-key: mg_…

If both are present, x-api-key wins. A Bearer token that doesn’t start with mg_ is not treated as a ModelGate key.

Keys can be revoked any time from the dashboard; a revoked key stops working immediately.

Making a request

The gateway speaks the OpenAI Chat Completions format at POST https://gw.modelgatehq.com/v1/chat/completions. Send the same body you’d send to OpenAI — model and messages — and ModelGate forwards it, using your stored key, to your project’s default provider (set in Settings). To pick a different one for a single call, add "provider": OPENAI, ANTHROPIC, GEMINI or AZURE_OPENAI.

curl

curl https://gw.modelgatehq.com/v1/chat/completions \
  -H "content-type: application/json" \
  -H "x-api-key: $MODELGATE_API_KEY" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "Ping"}]
  }'

OpenAI SDK (Python)

from openai import OpenAI

client = OpenAI(
    base_url="https://gw.modelgatehq.com/v1",
    api_key="YOUR_MODELGATE_KEY",     # sent as Authorization: Bearer
)

resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Ping"}],
)
print(resp.choices[0].message.content)

OpenAI SDK (Node)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://gw.modelgatehq.com/v1",
  apiKey: process.env.MODELGATE_API_KEY, // sent as Authorization: Bearer
});

const resp = await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "Ping" }],
});

The response comes back in the same OpenAI shape regardless of which provider served it, so anything that parses a Chat Completions response keeps working.

Streaming

Set stream: true and the gateway forwards the response as OpenAI-format server-sent events (chat.completion.chunk), whichever provider serves it — so an OpenAI SDK’s streaming iterator works unchanged. Cost, token counts and the waste score are computed on the assembled response when the stream finishes, so a streamed call is metered exactly like a buffered one.

stream = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Count to five"}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")

One limit: if a project has outbound secret-leak or PII enforcement turned on, streaming is refused with 409 — redaction can’t be applied to a response before it has been fully received. Use a non-streaming request, or set those guardrails to DETECT. Inbound prompt-injection enforcement is unaffected and still applies to streamed requests.

Supported models

Use the provider’s own model name. Prices below are per million tokens, for reference.

Pricing is stored in a versioned rate table and used to cost every request exactly — no estimates, no sampling. Rates change; the dashboard always reflects the table in force at the time of each call.

Optimization

ModelGate watches your traffic for two kinds of waste it can act on automatically, and flags calls that plain code could replace. Each is configured per project in Settings, and each has a safe middle setting that only recommends before it ever changes a request.

Exact-response caching

Identical, deterministic requests (same model, same messages, temperature 0) can be served from a cache instead of paying the provider again.

• OFF — never cache.
• RECOMMENDATION_ONLY — measure how much caching would save, change nothing.
• AUTO — serve repeats from cache. A cache hit costs $0; the saving is what the original call cost.

Model routing

When a small, simple call is running on a premium model, ModelGate can route it to the same provider’s value model. A call qualifies when its prompt is about 400 tokens or less, it has no tools and its temperature is 0.3 or lower. Features or endpoints named legal, medical, financial or security are never routed.

Any other model is left as requested. Azure OpenAI calls are never routed, because the model there is a deployment name you chose.

• OFF / RECOMMENDATION_ONLY — leave requests untouched; flag candidates.
• SAFE_AUTO / AGGRESSIVE_AUTO — route qualifying calls automatically. Both modes currently apply the same rules.

Code replacements

Some calls don’t need a model at all: the answer is a fixed label, a piece of the input, the input’s own values rearranged as JSON, a number computed from the input, or a simple fact about an image. Others need a model, but not an LLM: sentiment or intent in many categories, names of people and companies, a translation, what’s in a photo. ModelGate finds these tasks and writes code — or code that runs a small open model — to replace the call. It needs prompt storage and response capture on, because it compares what your app sent with what the model answered.

• Scan (free, every plan) — on the Tasks page. A task with at least 50 recorded input/output pairs is flagged when, for example, it only ever gives ten or fewer distinct answers, or 90% of its answers are copied verbatim from the input, or the answer is the input in another language. Up to 50 distinct short answers count as a classifier. No model call, no cost. The Tasks page also shows coverage: how much of your spend was analysed, and why the rest couldn’t be (storage off, fewer than 50 recorded calls, or the task needs an LLM).
• Generate (trial or paid plan) — first tries our own tested recipes: a regular expression, keyword rules, a JSON field mapping, a formula, or for images a pixel-statistics check, a dominant-colour check, sharpness (blur), saturation, image size or shape, QR/barcode decoding or local OCR. If none fits a text task, it asks the task’s own provider, with your stored key, for a recipe or a small JavaScript function — at most three calls.
• Open models — if code doesn’t verify, ModelGate tries small open-weight models on CPU: CLIP for image labels (zero-shot, or with a classifier trained on your recorded answers), a sentence-embedding classifier, DETR for counting objects, a named-entity model, Opus-MT for translation, and the most-downloaded matches from Hugging Face. A model is only considered if its licence is MIT, Apache, BSD or CC-BY, its weights are ONNX and under 400 MB; the code pins the exact model version that was tested. These need 50 recorded calls: up to 100 train the fit, at least 30 are held back.
• Verify — 20 recorded calls are used to build the code (more for open models); the rest are held back. A replacement is shown only if it matches at least 95% of at least 30 held-back answers it never saw. Model-written functions run in an isolated sandbox with no file system, network or timers, under memory and time limits. Translations are compared by meaning (multilingual sentence-embedding similarity of at least 0.9), not word for word.

A verified result appears as a REPLACE_WITH_CODE recommendation with the code, the match rate and up to five masked examples it got wrong. Recipes come as TypeScript (the exact logic that was verified) and a Python port, which we don’t execute — test it before shipping. Model-written functions come as JavaScript. Open-model replacements name the model, its licence (CC-BY needs attribution) and download size; they run on your servers with no API key or per-call cost, and their Python port loads the upstream PyTorch weights, which we don’t execute. Examples sent for generation go only to the provider the task already uses, with secrets masked; image tasks and open models make no LLM call. Images passed by URL are fetched once for verification — public HTTPS addresses only — and not stored.

const res = await openai.chat.completions.create({
  model: "gpt-4o",
  messages: [
    { role: "system", content: "Reply with one word: urgent or normal." },
    { role: "user", content: ticket.body },
  ],
});
const priority = res.choices[0].message.content;

export function replacement(input: string): string {
  const text = input.toLowerCase();
  const rules = [
    {
      "label": "urgent",
      "any": ["outage", "is down", "can't log in", "charged twice", "data loss"],
      "none": ["not urgent"],
    },
  ];
  for (const r of rules) {
    if (r.any.some((k) => text.includes(k)) && !r.none.some((k) => text.includes(k))) return r.label;
  }
  return "normal";
}

// where the LLM call was:
const priority = replacement(ticket.body);

The savings from automatic caching and downgrades are what we call realized savings — measured at the gateway, shown on your dashboard, and the basis for the success fee.

Connect GitHub

Optional. Connect a repository on the Code page of the dashboard and ModelGate can find the AI calls in your code, route them through the gateway for you, and open its suggestions as pull requests. One repository per project, and one project per repository.

What ModelGate can and can’t do

• Reads only the repositories you choose when you install the ModelGate GitHub App.
• Scans in memory and keeps only file paths, line numbers, the SDK and the model / max-token values at each call — never your source.
• Writes only to new modelgate/… branches, and only when you click to open a pull request. It never pushes to your default branch, never merges, and never runs your code.
• Permissions: contents and pull requests on the selected repositories. Revoke anytime in GitHub → Settings → Applications.

Route everything through the gateway

The scan finds where each SDK client is created and whether it already points at ModelGate. For OpenAI SDK clients (JavaScript/TypeScript and Python), Route all through ModelGate opens one pull request that changes those lines and creates an app key limited to running requests, shown once:

// before
const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
// after — your options are kept; the gateway settings win
const client = new OpenAI({ ...({ apiKey: process.env.OPENAI_API_KEY }), baseURL: process.env.MODELGATE_BASE_URL ?? "https://gw.modelgatehq.com/v1", apiKey: process.env.MODELGATE_API_KEY });

Before merging, add your OpenAI key in Providers and set MODELGATE_API_KEY in the app’s environment. After you deploy, the Code page shows Connected as soon as the first request reaches the gateway. Clients for other SDKs (Anthropic, Gemini) are listed with instructions — the gateway speaks the OpenAI format, so they switch to an OpenAI-compatible client as shown in Making a request.

Suggestions at the line, fixes as pull requests

Calls in the code are matched to the tasks the gateway sees (system prompt, feature tag, model), so each suggestion links to the file and line it’s about. For a cheaper model, a lower output limit or a verified code replacement, Open pull request makes the edit on a new branch for you to review.

Changes you make from suggestions

When you make a change we suggested — whether through our pull request or by hand — the gateway compares the task before and after and bills 30% of the saving for 3 months, then stops. Model, prompt and output-limit changes are measured; a call replaced with code no longer reaches the gateway, so its saving is estimated from the volume and cost per call before the change. Tracking is on by default; the account owner can turn it off per project, and any change can be marked not ours. See pricing.

Security guardrails

The gateway can inspect traffic in both directions and act on it. Each category is a per-project setting in Settings with three levels — the same OFF → DETECT → ENFORCE ladder as the optimizations. Detect logs incidents to the Security page without changing any request; Enforce blocks or redacts. Incidents are stored with a masked snippet only — the raw secret or PII value is never persisted.

• Prompt-injection shield (inbound) — scores untrusted input for jailbreak and instruction-override attempts; ENFORCE refuses a high-risk request with 403 blocked_by_guardrails before it reaches a provider.
• Secret-leak prevention (outbound) — signature + entropy detection for API keys, tokens, private keys and connection strings; ENFORCE redacts them from the response.
• PII protection (outbound) — emails, Luhn-valid card numbers, SSNs and more; ENFORCE masks them before the response is returned.

DETECT is available on every plan. ENFORCE (active blocking/redaction) is a paid capability, like the automatic optimizations.

Reliability

The gateway also scores whether your bot is stable — does it answer the same task the same way, keep its format, and avoid contradicting itself. Deterministic detectors (format breaks, degenerate output, refusals) run inline and free on every plan and feed a consistency score with a daily trend. Judge-based detectors (contradiction, instruction drift, grounding) read a whole conversation and are part of the Reliability add-on. Every setting lives in Settings under Reliability, and incidents appear on the Reliability page with masked evidence.

Conversation stitching

To reason about a whole thread, the gateway needs to know which turns belong together. Pass a stable conversation id and it’s exact; omit it and turns are stitched heuristically by prompt overlap within a time window.

x-modelgate-conversation-id: conv_8f31a2   # optional; otherwise stitched heuristically

Response capture (opt-in)

The judge detectors and the evidence drill-down need the answer text. Response storage is off by default, enabled per project under Store response bodies, and retention-capped — a purge job clears stored responses older than your window (default 14 days). The deterministic consistency score needs none of it; it works from response hashes.

The regression gate

An active Consistency Test fires several paraphrased variants of one prompt through the same model and returns a varianceScore (0–100, higher = more consistent). Call it from CI to fail a deploy when a change makes a critical prompt less stable. Active tests spend your provider tokens, so they’re metered.

curl -s -X POST https://gw.modelgatehq.com/v1/dashboard/reliability/test \
  -H "x-api-key: $MODELGATE_API_KEY" \
  -H "content-type: application/json" \
  -d '{"promptTemplate": "Is a refund available after 30 days?",
       "model": "gpt-4o-mini", "provider": "OPENAI", "variantCount": 6}'
# → {"testId":"…","status":"COMPLETED","varianceScore":83.5,"variantCount":6}

Deterministic detection, response capture and the consistency score are free on every plan. Judge detection, the confusion map and active tests are the Reliability add-on ($49/mo + metered runs). See how reliability works and pricing.

Controls and defaults

Logging, pricing and scoring ship on. Every optimization and every guardrail ships off, as a per-project setting you raise when you trust it — with a middle setting that only recommends.

Spend controls

Managed at the gate, before you’re billed by a provider. Set them per project in Settings:

• Requests per minute — a rate ceiling per API key.
• Max prompt / output tokens — reject oversized calls.
• Monthly spend cap (USD) — once the month’s provider spend hits the cap, further requests are refused with 402 until the cap is raised or the month rolls over.

Errors

Billing

Every account starts in free mode — no card: requests are logged, priced and audited, and recommendations are on, but nothing changes your traffic. Adding a card turns the automatic savings on and starts a 14-day free trial; the card is charged only when the trial converts on day 14 (cancel before then and you pay nothing). Pro is $99/month (1 project included, $29 per extra project) plus 30% of the savings the gateway realizes for you; the agency plan is $199/month including 5 clients, $29 per extra client. If nothing is saved, there’s nothing to pay beyond the base.

The $99 base is a PayPal subscription. The 30% success fee is billed once a month as a separate PayPal invoice for 30% of that month’s realized savings. Full detail is on your dashboard’s Billing page. Changes you make from our suggestions (a cheaper model, a shorter prompt, a lower output limit, a call replaced with code) are billed at 30% of their savings for 3 months after the change — on by default, off per project in Code. See also the Terms and Privacy Policy.

Questions

Email support@modelgatehq.com and we’ll help you get set up.