LiteLLM alternatives for teams that outgrew the proxy

Teams rarely leave LiteLLM because it stopped working. They leave when four things the proxy treats as side concerns become the actual job: log retention, key rotation, failover under load, and per-team budgets somebody can audit. On one side sit self-hosted replacements that keep the gateway inside your own infrastructure, Bifrost and the open source Portkey gateway among them, both free to run as of 14 August 2026; on the other sit managed gateways, Portkey Cloud at $49 a month, TrueFoundry at $499, Cloudflare AI Gateway with a free core tier, which take the operational load away and hand you a bill instead. The comparator just below prices each one against your own request volume and model spend.

What actually breaks when a self-hosted proxy grows

LiteLLM is very good at the thing it was built for: one API surface in front of many providers, running on your own hardware, with nobody between you and the model. The pressure does not come from that core job. It comes from everything a growing team starts asking of the box that now sits on the critical path of every AI request in the company.

Observability stops at the log table

At ten thousand requests a month, a table of requests and responses is plenty. At ten million it is a storage problem with a retention policy attached, and the questions change shape: which prompt version regressed, which customer generated Tuesday’s spike, why the p99 doubled. Answering those needs traces and aggregation, not rows. This is usually the first thing that pushes a team to either bolt an observability layer on top of the proxy or move to a gateway that ships one.

Key rotation becomes somebody’s manual job

Provider keys live in the proxy configuration. Rotating them means touching that configuration, redeploying, and hoping nothing held a stale key. LiteLLM’s own Enterprise tier is where secret manager integration and key rotation sit, next to SSO, SCIM and OIDC or JWT authentication, which says plainly that the free build is not meant to carry that responsibility at scale. Every serious alternative draws the same line in roughly the same place.

Rate limits and failover under real load

A gateway earns its keep the day a provider returns 429s for twenty minutes. Retries, timeouts, fallbacks and load balancing across providers are what separate a router from a passthrough, and they are also where implementation quality shows. Maxim publishes a benchmark putting its own Bifrost gateway at 54 times better p99 latency and 9.5 times the throughput of LiteLLM at 500 requests per second on identical hardware. That is a vendor benchmark on the vendor’s own product, so read the ratio as a claim rather than a measurement, but the pressure point it names is real: a Python proxy under sustained concurrency does not behave like a Go one.

Team budgets nobody can audit

Virtual keys with a budget attached solve the first version of this problem. The second version is a finance question: spend by key, by user, by team and by tag, with an audit log on every request, exported somewhere the people who sign the invoice can actually read it. On LiteLLM that is Enterprise territory, and the same is true of almost every alternative below.

Self-hosted alternatives that stay in your infrastructure

If the reason you run a proxy at all is that requests must not leave your network, the managed half of this page is not for you. Three options keep the data plane exactly where it is.

Bifrost, the Go gateway benchmarked against LiteLLM

Bifrost, from Maxim, is Apache 2.0 and free forever in its open source build, shipped as a Docker image, a Kubernetes deployment or a Go binary. The free tier is unusually complete for this category, and the line it draws with the paid tier is easy to read. Checked 14 August 2026.

• Free, self-hosted: drop-in replacement on the same API surface, OpenTelemetry output, budget management with virtual keys, custom routing, automatic fallbacks, semantic caching, an MCP gateway and a prompt repository.
• Enterprise, priced on quote: guardrails, cluster mode, adaptive load balancing, SAML and OIDC single sign-on, vault integrations, log exports, audit logs and role-based access control.

Portkey open source gateway

Portkey splits itself in two, and the split is clean. The gateway is open source and self-hostable with no request limit: universal API, retries, timeouts, routing, guardrails, automatic fallbacks, load balancing and a basic dashboard, with community support. The analytics, prompt management and governance features live in the hosted product instead. If your objection to LiteLLM is throughput or configuration sprawl rather than observability, the open source gateway on its own is a lateral move that costs nothing.

Kong and Envoy, if you already run an API gateway

The most underrated answer to this question is that you may already own the solution. Kong’s AI Gateway adds LLM routing, semantic caching, token quotas by user and by model, PII sanitisation and consumption telemetry to the gateway your platform team already operates, and it now covers MCP and agent-to-agent traffic on the same control plane. Envoy has an equivalent AI extension for teams standardised on it. Kong publishes no price for the AI plugins, which sit inside Kong Konnect behind a free trial and then a sales conversation, so this route is only cheap when the licence is already paid for.

Managed gateways that take the proxy off your hands

The trade here is explicit. You stop operating a critical-path service, and in exchange your requests transit a vendor and your bill grows with your volume.

Portkey Cloud and TrueFoundry

Portkey Cloud prices on recorded logs. The Developer plan is free forever with 10,000 logs a month and three days of retention, which is a prototyping tier and not a production one. Production is $49 a month for 100,000 logs, then $9 per additional 100,000 up to three million, and it is the tier that adds alerts, role-based access control, semantic caching and unlimited prompt templates. Above three million logs the published grid stops and Enterprise begins.

TrueFoundry prices on requests and seats instead. Developer is free at 50,000 requests a month for up to three users, Pro is $499 for a million requests and ten users, Pro Plus is $2,999 for a million requests and twenty-five users, and extra volume runs at $499 per additional two million requests. Self-hosted, VPC and air-gapped deployment are Enterprise only, with roughly $600 to $1,000 a month of hosting on top. Prices read 14 August 2026.

Cloudflare AI Gateway

Cloudflare’s answer is the odd one out, because the core is free: dashboard analytics, caching and rate limiting cost nothing beyond an account. What is metered is storage and the extras.

Persistent logs cap at 100,000 across all gateways on the free tier and 10 million per gateway on a paid plan, Logpush is paid-plan only at 10 million a month then $0.05 per million, and unified billing adds a 5 percent fee on credit purchases while passing provider rates through unchanged. Guardrails bill as Workers AI inference. For a team that wants caching, rate limiting and a dashboard without operating anything, it is the cheapest serious entry point on this page. Read 14 August 2026.

OpenRouter, when routing matters more than governance

OpenRouter is a different animal: a router across hundreds of models with no markup on inference, funded by a 5.5 percent fee on credit purchases by card with a $0.80 minimum, or 5 percent by crypto. Bringing your own provider keys is free up to $25,000 of monthly inference and costs 5 percent above that allowance. Read 14 August 2026.

What it does not give you is the governance layer teams leave LiteLLM to obtain: no self-hosted build, no VPC, no audit trail on your own terms. It answers the model access question, not the platform question. We put the two side by side in our OpenRouter and LiteLLM comparison, and cover the rest of that field in the wider list of OpenRouter alternatives.

Keep LiteLLM and add the layer that is missing

Replacing a working gateway is a migration with a rollback plan attached. Often the honest fix is narrower: keep the proxy and buy only the piece it does not do.

Helicone for request-level observability

Helicone is open source and sits either as a proxy or as an async logger beside the one you already run. Hobby is free at 10,000 requests a month with one seat, one gigabyte of storage and seven days of retention.

Pro is $79 a month with unlimited seats, alerts, reports and one month of retention, and Team is $799 with three months of retention, SOC-2 and HIPAA. Volume above a plan is billed usage-based at a rate the pricing page does not publish, which is the one number to ask sales for before you commit. Read 14 August 2026.

Langfuse for traces and evaluation

Langfuse is not a gateway and routes nothing, which is exactly why it pairs well with a proxy you want to keep. Self-hosting is free with no usage or seat cap.

The cloud is $0 for 50,000 units a month on Hobby, $29 for 100,000 units on Core, $199 on Pro for three years of retention and $2,499 on Enterprise. Overage runs at $8 per additional 100,000 units, and that is the rate the comparator uses.

Langfuse says the unit price falls at high volume, but publishes the steps inside its own calculator rather than on the pricing page, so we hold $8 instead of inventing a discount. Units are events rather than requests, which pushes the estimate the other way: one traced call with several spans burns more than one unit. Read 14 August 2026.

What we priced, and what we could not price

Prices read on 14 August 2026

Every figure on this page, in the comparator, in the table below and in the sections above, was read from the vendor’s own public pricing page on 14 August 2026. Where a published grid stops, we say so rather than extrapolating past it. The token prices underneath all of this move on their own schedule, and we track those separately in our comparison of per million token prices for every major model.

What has no public price

Some numbers on this page do not exist publicly, and no amount of research produces them. Here is exactly what we could not put a figure on, and why.

• Quoted per deal, no list price at all: LiteLLM Enterprise, Bifrost Enterprise, Portkey Enterprise and Kong’s AI Gateway.
• Published but incomplete: Helicone’s usage-based overage above a plan, and Langfuse’s graduated unit price at high volume.
• Not priced here on purpose: the Cloudflare paid plan you need past 100,000 persistent logs, and Envoy AI Gateway, which we mention as an option without putting a number on it.

If a comparison table anywhere hands you a hard monthly figure for any of those, ask where it came from.

Frequently asked questions

Is LiteLLM still a reasonable default?

For most teams, yes. It is free, it speaks to everything, and the failure mode of adopting it early is small. The question is not whether to start there, it is whether you have hit one of the four walls above, because each one has a cheaper answer than a full migration.

Do you have to replace the gateway to get SSO and audit logs?

No, and switching rarely helps on its own, because SSO, SCIM, audit logs and role-based access control are gated behind the paid tier on LiteLLM, Bifrost, Portkey and TrueFoundry alike. You are choosing which vendor to buy governance from, not whether to buy it.

Does a managed gateway add latency?

It adds a network hop, and how much that costs depends on where the gateway runs relative to you and to the provider. Cloudflare’s edge is the strongest case for the managed side on that specific point. If latency is your binding constraint, measure it on your own traffic before switching, because a vendor benchmark will not answer it for you.

Can you run two gateways during a migration?

Yes, and it is the safest way to do it. Point a small share of traffic at the new gateway, keep the old one as the fallback, and compare cost and error rates on the same workload for a couple of weeks. Most alternatives here advertise a drop-in replacement for the same request format, which makes the split cheap to set up and cheap to undo.

How often are these prices re-checked?

Whenever a vendor changes a grid, and on a scheduled pass otherwise. Each figure carries the date it was read so you can judge the freshness yourself instead of trusting a generic update stamp.