Change one URL. Keep your code.
Today each team keeps its own provider keys and client setup. A model deprecation breaks a feature on a day nobody chose, and nobody owns the fix. Change the base URL, and every team uses one endpoint instead. Your libraries, streaming, and tool calls keep working. If it is not for you, change the URL back. That is the whole exit.
from openai import OpenAI
# Before
client = OpenAI(api_key=PROVIDER_KEY)
# After: same client, same calls
client = OpenAI(base_url="https://gateway.example.com/v1", api_key=GATEWAY_KEY)
Less setup for every team
- No code changes. The Gateway speaks each supported client's API as is. Your test suite runs against it unchanged.
- One URL for everything. Every team, model, and approved provider sits behind it.
- Names that outlive vendors. Give every team three names:
corporatesafety/premium,standard, andeconomy. Or change nothing: keep requesting the model names already in your code, and the Gateway accepts them and answers under the name you asked for. Power users can list the full model catalog instead. Behind every name, a provider switch is a recorded policy change, not a deploy on your side. - No access tickets. Engineers join through their sign-in group and mint their own keys inside the budget ceilings you set, so 200 people start without 200 requests.
- Outages handled. When a provider fails, the Gateway retries, falls back, or refuses with a reason. It records which path it took.
Objections
- "We will be locked in to your gateway." The Gateway speaks each client's native dialect, so your code never learns ours. Leaving is the same base-URL change, reversed. Your policies and your request history export, so nothing stays behind.
- "We already run LiteLLM, or a cloud provider's own gateway." Keep it for routing. That is a data plane: it forwards and retries. This adds the decision layer underneath it: screening, pricing, and a policy record a plain router has no place for.
- "Our cloud provider already has a gateway." A fit when all your traffic stays in one cloud and that cloud's own gateway meets your policy. Come back when a second provider joins the stack.
- "Setup sounds like a project." Four steps, no rewrite: change the URL, set your rules, let it decide, let it start finding savings. You start on one workload before you touch a second team.
- "Another hop will slow my feature." Screening runs in the request path. Pricing and grading run alongside it, not inside it. Overhead is a published target, under 60 ms added, 40 ms to first streamed token, and your first workload measures the real number on your traffic.
Run your own tests against it
Point your client test suite at the Gateway, unchanged.
- Compare outputs and stream events with direct provider calls.
- Kill a provider mid-test. Read which path the Gateway took, and why.
- Ask for the model name your code already uses. The response answers under that name, and the decision record shows what served it and under which policy version.
Put one team first in line.
We admit customers in cohorts, in waitlist order. Don't miss the next one. When your slot comes up, you start on one real workload, on your plan from day one. You keep a report on every request. If we are not earning our fee, leaving is the same URL change, reversed.