How it works
One base URL puts the gateway between your app and OpenAI, Anthropic, Gemini or Azure. From then on it does three things: prices every call, finds and cuts the waste, and screens traffic for safety and reliability. Everything that changes a request is off until you turn it on.
I · See every cost
This sample audit runs in your browser with the same waste rules and rate table as the gateway. Pick a scenario or edit the numbers.
A retrieval pipeline stuffing the whole context window to get a short answer.
System: You answer only from the supplied documents. User: [12 retrieved passages] — Which clause covers early termination?
Computed in your browser from ModelGate’s published rate table and the same rule set the gateway applies. Token counts are yours to change — a real request’s counts come from the provider.
ModelGate authenticates your key, applies your rate, token and spend ceilings, and forwards the call using the provider credential you stored. Prompt bodies are not persisted unless you ask for it.
Token counts come from the provider; rates come from a versioned table. Fixed-point arithmetic, no floating-point drift — and if a model has no published rate, the call is logged without a cost rather than with a guess.
Six rules produce a 0–100 score and named flags, and a worker turns them into recommendations carrying a computed saving, a confidence level and a risk rating.
II · Cut it
The checks are deterministic and published. When one matches your traffic, the finding lands on your dashboard with the flag that raised it.
Repeated prompts
A support bot or internal helpdesk fires near-identical prompts thousands of times a week — same system prompt, same FAQs — and pays the provider in full for each one. ModelGate hashes the normalized prompt, sees the repeat, and can serve it from an exact cache.
Premium model, trivial task
Sentiment tags, field extraction, short classifications — work a value-tier model handles — quietly run on a premium model. ModelGate flags the mismatch, and with routing switched on it moves small, simple, low-temperature calls to the same provider's value model — gpt-4o to gpt-4o-mini, claude-sonnet-5 to claude-haiku-4-5, gemini-2.5-pro to gemini-2.5-flash.
Context beyond usefulness
RAG pipelines and long chats keep stuffing history into every request well past the point it changes the answer. ModelGate records the input-to-output ratio on every call, so the features where context stopped earning its cost are visible rather than assumed.
No model needed
Pulling an email address out of a message, sorting tickets into a handful of labels, reshaping JSON, reading a QR code, checking whether a scan is blank — some calls have an exact answer a few lines of code can give. ModelGate finds them in your recorded traffic, writes the code, and only shows it once it matches the model's own answers on calls it was never shown.
When a task’s answers follow a rule, ModelGate writes the rule as code and checks it against your model’s own recorded answers — it’s only suggested once it matches at least 95% of 30+ calls it never saw. The code below is exactly what the generator emits for these examples; the results are what it returns. The “before” calls are illustrative.
Two labels, picked from a handful of phrases. A keyword rule, not a model.
const res = await openai.chat.completions.create({
model: "gpt-4o",
messages: [
{ role: "system", content: "Reply with one word: urgent or normal." },
{ role: "user", content: ticket.body },
],
});
const priority = res.choices[0].message.content;export function replacement(input: string): string {
const text = input.toLowerCase();
const rules = [
{
"label": "urgent",
"any": ["outage", "is down", "can't log in", "charged twice", "data loss"],
"none": ["not urgent"],
},
];
for (const r of rules) {
if (r.any.some((k) => text.includes(k)) && !r.none.some((k) => text.includes(k))) return r.label;
}
return "normal";
}
// where the LLM call was:
const priority = replacement(ticket.body);In your code
ModelGate sees your traffic; your repository shows where it comes from. Connect it and the setup and the fixes become pull requests you review and merge. Your source is read in memory and never stored — only file paths, line numbers and model names.
Route
The scan finds where each SDK client is created and whether it already goes through ModelGate. One click opens a pull request that changes those lines, and creates an app key to paste into your environment.
Point
Calls in the code are matched to the tasks the gateway sees — by system prompt, feature tag and model — so a suggestion reads “src/support/reply.ts:42”, not just “downgrade this”.
Fix
A cheaper model, a lower output limit, or the verified code that replaces a call — opened on a new branch. ModelGate never pushes to your main branch and never merges.
Measure
When you make a change we suggested, the gateway measures what it saves before and after. It's billed at 30% for 3 months, then stops — on by default, off any time, and any change can be marked not ours.
III · Keep it safe and reliable
Each category is off until you turn it on: detect records incidents without touching a request, enforce blocks or redacts. These are pattern-based detectors, not guarantees — here is exactly what they match.
Inbound
Ten weighted signals — instruction override, system-prompt exfiltration, role hijack, jailbreak personas, safety bypass, chat-template injection, zero-width characters and more — score the untrusted part of every request from 0 to 100. It is a heuristic: it catches the common families, not every phrasing.
Outbound
Seventeen credential signatures — provider keys, AWS and Google keys, GitHub and Slack tokens, Stripe keys, PEM private keys, JWTs, bearer tokens, database URLs with passwords — plus a Shannon-entropy pass for tokens no signature knows.
Outbound
Email addresses, Luhn-valid card numbers, structurally valid US Social Security numbers, phone numbers, IP addresses and IBANs. Overlapping matches resolve to the most severe, so a card number is never filed away as a phone number.
At rest
Your provider credentials are encrypted with AES-256-GCM under a master key held outside the database, and only a first-four-last-four hint is ever shown back. ModelGate API keys are stored as salted HMAC-SHA256 digests and compared in constant time.
Reliability works the same way: a consistency score plus format, looping and refusal checks on live traffic, free on every plan. See the reliability demo →Security architecture →Every control and its default →
Every request is logged, priced and audited from the first call. 14 days free when you add a card.