LLM gateway · Cost · Security · Reliability
ModelGate sits between your application and OpenAI, Anthropic, Gemini and Azure. Every request is logged, priced against a versioned rate table, and scored for waste. The same gateway screens traffic for prompt injection, secrets and personal data — and scores whether your bot is reliable: stable, in format, and consistent. Three axes, one integration. No estimates, no sampling.
| gpt-4o | 6,000 requests | $114.00 |
| claude-sonnet-5 | 2,000 requests | $42.00 |
| gemini-2.5-pro | 1,500 requests | $13.13 |
| gpt-4o-mini | 12,000 requests | $3.42 |
Sample audit
A retrieval pipeline stuffing the whole context window to get a short answer.
System: You answer only from the supplied documents. User: [12 retrieved passages] — Which clause covers early termination?
Computed in your browser from ModelGate’s published rate table and the same rule set the gateway applies. Token counts are yours to change — a real request’s counts come from the provider.
Integration
No new client library and no request rewriting. The gateway speaks the OpenAI chat-completions dialect and forwards to whichever provider owns the model you named.
client = OpenAI(
base_url="https://api.openai.com/v1",
api_key=OPENAI_KEY,
)
# cost: unknown waste: unknownclient = OpenAI(
base_url="https://gw.modelgatehq.com/v1",
api_key="unused", # ModelGate reads x-api-key
default_headers={"x-api-key": MODELGATE_KEY},
)
# every call logged, priced, scoredModelGate authenticates your key, applies your rate, token and spend ceilings, and forwards the call using the provider credential you stored. Prompt bodies are not persisted unless you ask for it.
Token counts come from the provider; rates come from a versioned table. Fixed-point arithmetic, no floating-point drift — and if a model has no published rate, the call is logged without a cost rather than with a guess.
Six rules produce a 0–100 score and named flags, and a worker turns them into recommendations carrying a computed saving, a confidence level and a risk rating.
One thing worth knowing before you switch: authentication is the x-api-key header rather than Authorization: Bearer. stream: true works and returns OpenAI-format SSE — the one exception is a project running outbound secret/PII enforce, where a stream would be refused because redaction can’t be applied mid-stream.
What the audit finds
The waste rules are deterministic and published. If one of these matches your setup, the finding lands on your dashboard with the flag that raised it.
Repeated prompts
A support bot or internal helpdesk fires near-identical prompts thousands of times a week — same system prompt, same FAQs — and pays the provider in full for each one. ModelGate hashes the normalized prompt, sees the repeat, and can serve it from an exact cache.
Premium model, trivial task
Sentiment tags, field extraction, short classifications — work a value-tier model handles — quietly run on a premium model. ModelGate flags the mismatch, and with routing switched on it moves small, simple, low-temperature GPT-4-family calls to gpt-4o-mini.
Context beyond usefulness
RAG pipelines and long chats keep stuffing history into every request well past the point it changes the answer. ModelGate records the input-to-output ratio on every call, so the features where context stopped earning its cost are visible rather than assumed.
Security by design
One place to screen traffic across every model and provider, with no application changes. Each category is off until you turn it on: detect records incidents without touching a request, enforce blocks or redacts. These are pattern-based detectors, not guarantees — we describe exactly what they match so you can judge the fit.
Inbound
Ten weighted signals — instruction override, system-prompt exfiltration, role hijack, jailbreak personas, safety bypass, chat-template injection, zero-width characters and more — score the untrusted part of every request from 0 to 100. It is a heuristic: it catches the common families, not every phrasing.
Outbound
Seventeen credential signatures — provider keys, AWS and Google keys, GitHub and Slack tokens, Stripe keys, PEM private keys, JWTs, bearer tokens, database URLs with passwords — plus a Shannon-entropy pass for tokens no signature knows.
Outbound
Email addresses, Luhn-valid card numbers, structurally valid US Social Security numbers, phone numbers, IP addresses and IBANs. Overlapping matches resolve to the most severe, so a card number is never filed away as a phone number.
At rest
Your provider credentials are encrypted with AES-256-GCM under a master key held outside the database, and only a first-four-last-four hint is ever shown back. ModelGate API keys are stored as salted HMAC-SHA256 digests and compared in constant time.
Reliability by design
The gateway sees every answer, so it can score whether your bot is stable — same task, same answer; in format; not contradicting itself as a conversation grows — and show what’s confusing it. Deterministic checks are free and always leading with the evidence; judge-based checks and active tests are an add-on.
The same labelling task, asked four ways. A reliable bot gives one answer regardless of phrasing.
Classify the sentiment. Reply as JSON {label}.Computed in your browser with the same shape of logic the gateway applies — a consistency score plus deterministic detectors, no model call. The score is an estimate; the flagged answers are the evidence.
Controls
Logging, pricing and scoring ship on. Every optimization and every guardrail ships off, as a per-project setting you raise when you trust it — with a middle setting that only recommends.
Open source
An open-source, local-first LLM gateway for cost visibility, usage analytics, and basic heuristic security signals. Point your OpenAI or Anthropic client at it and every call is logged, priced and scored on your own machine — no ModelGate account, no telemetry. It’s the same idea as ModelGate Cloud, small enough to read end to end.
git clone https://github.com/razdgann/modelgate-oss.git cd modelgate-oss cp .env.example .env docker compose up --build
ModelGate OSS
ModelGate Cloud
ModelGate OSS v0.1.0 is an early release and is not yet recommended for critical production workloads without your own validation.
Raz DaganFounder, ModelGate
System engineer with a background in physics, mathematics and AI. Building ModelGate to make LLM infrastructure more transparent, efficient and secure.
Three months free, no card. The ledger fills as your traffic arrives, and nothing changes about how your application calls a model until you decide it should.