best 10 llm routers and ai gateways

openrouter is the closest thing this category has to a default. one openai-format endpoint, one key, and a catalogue that reaches essentially every commercial model plus a large free tier of open-weight ones. claude sonnet 4.5 lists at $3 per million input tokens — anthropic's own price, passed through unchanged. the faq is explicit that it passes provider pricing through 'without any markup on inference pricing'.

the fee that matters sits one step earlier. funding your credit balance by card costs 5.5% with an $0.80 minimum, so $100 of usable credit costs $105.50 and that $3 model effectively becomes $3.17. it is still cheaper than requesty's honest 5% on spend, and it disappears entirely if you bring your own provider keys — but byok itself starts charging 5% once you pass the monthly allowance.

what it does not have is a governance story. rate caps on the free models are documented, and byok works, but fallback behaviour, caching, spend limits and observability are not described anywhere we could reach within the pricing and faq pages. if you need to prove to someone that a prompt was filtered before it left your network, this is not the layer that does it.

there is no self-hosted option and no published model count — the faq says only 'all the major llm models on the market'. for a product whose entire value proposition is breadth of catalogue, declining to state its size is an odd omission.

portkey's licence is plain mit, read from the licence file at github.com/portkey-ai/gateway, with no enterprise directory and no carve-out. that puts it in a different position from litellm and bifrost, both of which withhold governance features from their open builds. you can run the whole thing yourself, and enterprise adds private cloud deployment rather than unlocking features.

the billing unit is the interesting part. portkey charges per recorded log — 10,000 a month free, 100,000 on the $49 production tier, $9 per additional 100,000 — and takes no cut of model spend. that means token-heavy workloads never scale the gateway bill: a million input tokens of claude sonnet 4.5 costs $3 to anthropic and nothing extra to portkey. the inverse is also true, and it is the catch. a chatty agent making many small calls burns logs fast, and log-based pricing is harder to forecast than a percentage.

the feature set is the most complete in the category and is documented tier by tier: fallbacks, retries and timeouts on every plan, simple caching from the free tier and semantic caching from production, virtual keys with budgets, and guardrails that go from community to llm and partner-provided as you move up. observability covers logs, traces, feedback and alerts.

the free tier's three-day log retention is the real limitation. it is enough to confirm an integration works and not enough to investigate anything that happened last week, which pushes any serious evaluation onto the paid plan quickly.

helicone is apache 2.0 at the root with no second licence file anywhere in the tree, including the web directory — the only one of the three self-hostable options here where we found no open-core split at all. saml sso and on-prem support are sold as enterprise services rather than gated behind a licence key, which is a meaningfully different arrangement from litellm's per-seat enterprise licence.

the token markup is genuinely zero, and the docs say so while naming openrouter's 5.5% by way of comparison. byok is supported. helicone makes its money on requests and storage instead: the hobby tier covers 10,000 requests a month, pro is $79 for an organisation with unlimited seats, and team is $799 for five organisations with three-month retention.

what you get for that is a gateway and an observability product in the same system — failover across providers, caching, custom rate limits, cost and latency tracking, sessions and user attribution, and prompt management with versioning deployable through the gateway itself. for teams currently running a proxy plus a separate tracing tool, collapsing those is the argument.

the pricing page will not tell you what anything costs past the included allowance. the per-request and per-gigabyte overage rates appear only inside an interactive calculator, so the bill at any volume above 10,000 requests cannot be worked out from published figures. that is a strange choice for a product otherwise this transparent.

vercel's pricing page states it directly: 'ai gateway charges no markup and no platform fee on tokens', and the docs extend that to byok. a million input tokens of claude sonnet 4.5 costs anthropic's list rate, deducted from a prepaid credit balance, with nothing added. the gateway works from a free hobby account, which makes it the cheapest serious starting point here alongside cloudflare.

the routing story is solid — one key across hundreds of models, automatic retry and failover to other providers, load balancing, and compatibility with openai chat completions, openai responses and anthropic messages formats, so most existing code points at it unchanged. embeddings, image and video generation ride the same credit balance.

the controls that make it deployable in a regulated environment are where the metering starts. team-wide provider allowlist bills at $0.10 per 1,000 successful requests, team-wide zero data retention at the same rate, custom reporting at $0.075 per 1,000 tag writes plus $5 per 1,000 reporting queries, and trace drains bill on a separate meter. none of these are expensive individually; together they mean the zero-markup headline describes the token path only.

the free tier includes monthly credits on a subset of models, but the amount is not stated on either the pricing page or the docs pricing page — so the one figure a hobby user most wants is the one missing.

the docs are unambiguous that inference is passed through 'with no markup — you pay the same per-token rates as you would directly with the provider'. the gateway itself is free on every cloudflare plan, including the free one, and $5 a month on workers paid buys ten million persistent logs per gateway rather than 100,000 across all of them. for a team already inside cloudflare, that is close to free capability.

what you get is genuinely useful: caching, rate limiting, request retry and fallback, per-request logging, and analytics broken out by request, token and cost. guardrails exist and bill on tokens scanned. logpush is available on paid plans with ten million requests a month included and $0.05 per million after.

the constraint is catalogue. the openai-compatible endpoint reaches fourteen providers — anthropic, openai, groq, mistral, cohere, perplexity, workers ai, google ai studio, vertex, xai, deepseek, cerebras, baseten and parallel. that covers the models most teams actually use, but it is an order of magnitude narrower than openrouter or vercel, and there is no self-hosted path.

one charge to watch: funding calls through cloudflare's unified billing credits adds 5% at purchase, the same shape of fee openrouter charges. bring your own provider keys and it does not apply.

litellm's readme promises 'a single, unified interface to call 100+ llm providers ... using the openai format', and that breadth is the reason it is everywhere. self-hosted, it adds no per-token cost — your keys, your compute, provider bills direct at list. virtual keys, per-project spend tracking, load balancing, retry and fallback routing, and the admin dashboard all work on the free mit core.

the licence is where it gets complicated, and litellm is at least honest about it. the root licence file is mit but explicitly states that everything under the 'enterprise/' directory is governed by a separate licence, and that file is a proprietary berriai enterprise licence requiring 'a valid berriai enterprise license for the correct number of user seats' for production use. development and testing are permitted unlicensed.

what sits behind that line is precisely what an organisation adopts a gateway to get: sso and scim, oidc and jwt auth, audit logs with retention, rbac, automated key rotation, secret-manager integrations for aws kms and vault, ip-based acls, multi-tenant orgs and teams, tag-based budgets, soft-budget alerts, programmatic spend reports, log export, and team-scoped guardrails. a small team proxying calls will never hit the line; a company rolling it out across engineering hits it immediately.

and there is no price. the enterprise page offers a quote and a 30-day trial key rather than a figure, so you cannot budget for it without a sales conversation — the one thing self-hosting was supposed to avoid.

requesty charges 5% on base model cost and puts a worked example on its own pricing page: 'a model costing $10 per 1m tokens from openai costs $10.50 through requesty'. so claude sonnet 4.5 lands at $3.15 per million input tokens against $3.00 direct. no subscription, no seat fees, no minimum spend. in a category where everyone else's real cost is buried in platform fees, log counts or credit-purchase charges, that is genuinely the easiest number to reason about.

it is also, on identical traffic, more expensive than openrouter with byok, cloudflare or vercel. the trade is that routing, caching and eu data residency are included on every plan, and fallbacks, spend limits and advanced observability arrive at the first paid tier rather than the third.

eu hosting on every plan is a real differentiator here — most of this category is us-only or silent on the question, and requesty states it on the pricing page rather than in a compliance pdf.

the ceiling is guardrails. pii detection, guardrails and sso with full rbac are enterprise-only at custom pricing, so the governance story stops exactly where litellm's paid tier begins. there is no self-hosted option either, and the vendor cannot keep its own catalogue size straight — the pricing page says 600+ models and the docs quickstart says 300+.

bifrost is a go binary with a published overhead of roughly 20 microseconds per request, and the licence file at github.com/maximhq/bifrost is unmodified apache 2.0 with no enterprise directory and no second licence anywhere in the repository. on those two facts alone it looks like the most permissive option in the category.

the readme then says enterprise deployments 'unlock advanced capabilities including adaptive load balancing, clustering, guardrails, mcp gateway, and other features', plus custom plugins and oidc user provisioning — documented at docs.getbifrost.ai/enterprise and absent from the apache-licensed build. this is the harder open-core pattern to spot: nothing in the licence restricts you, the code simply is not there. litellm's explicit carve-out is more honest about the same arrangement.

what does ship is respectable: automatic fallbacks and retries across providers and models, load balancing across keys, semantic caching, hierarchical budgets with virtual keys and per-team and per-customer limits, opentelemetry observability with a dashboard, and drop-in replacement for the openai, anthropic and google genai sdks.

the vendor cannot agree with itself on scale. the repo says it 'unifies access to 23+ providers' and the product page advertises '8+ providers and 1000+ ai models'. we have used the repo figure and left the model count unverified, which is an uncomfortable place to be for a spec that a buyer would reasonably treat as settled.

truefoundry puts 1,600+ models behind one openai-compatible endpoint and will deploy both the control plane and the gateway plane inside your own vpc, or fully air-gapped, so prompts never leave your infrastructure. for teams whose blocker is data egress rather than cost, that is the feature that decides it.

billing meters requests, not tokens: 50,000 a month free on developer, one million for $499 on pro, one million with stricter data controls for $2,999 on pro plus, and custom above ten million. because the meter is requests, routing a million input tokens of claude sonnet 4.5 consumes allowance rather than scaling the bill, and you pay anthropic list for the tokens themselves.

we could not verify a no-markup claim, and have not published one. the pricing page prices requests and never mentions token markup in either direction, and the gateway docs make no markup or byok disclosure. the absence of a charge on the published rate card is not the same as a vendor stating it takes no cut, and this site does not treat it as such.

the feature set is genuinely enterprise-grade — latency-based and weighted load balancing, automatic fallback, per-user and per-service rate limiting with quota enforcement, pii filtering and toxicity guardrails, caching, and cost attribution across teams. the problem is the ladder. fifty thousand requests to $499 a month is a step most teams hit before they are ready for it, and self-hosting is locked to the custom-priced enterprise tier.

kong's argument is coherent: your model calls are api traffic, so they should sit behind the same quotas, access control, pii handling and l7 observability as everything else you run. the ai gateway adds llm governance, token usage management, semantic caching, cost tracking, mcp server generation and governance, and agent-to-agent traffic management on top of the existing kong konnect control plane.

it does not resell inference. you bring your own provider credentials and kong adds nothing per token, charging instead for gateway capacity — so a million input tokens of claude sonnet 4.5 costs anthropic's list price and $0 in kong markup.

what kong costs is the problem. the pricing page gives a 30-day free trial with full enterprise functionality, then bills konnect plus per gateway per month without publishing the base figure. the add-on rates are printed — $100 a month per model proxy with five included, $200 a month per additional million api requests — but a rate card with no base price cannot be modelled.

the second gap matters more for anyone planning to self-host. the kong gateway core is apache 2.0, verified from the raw licence file, but neither the plugin hub listing nor the ai-proxy plugin page shows a tier table, so which ai plugins are open source and which are enterprise-gated is undocumented. we have not published a claim either way — and for a product whose main appeal is running it yourself, that is the single most important thing left unstated.