Distribute model traffic
Balance requests across configured model deployments using the router or proxy.
Self-hosted gateway
LiteLLM
Run a proxy or use a Python router to distribute requests across model deployments, with load balancing, retries, cooldowns and fallbacks.
Sources reviewed . Documentation review; not a performance benchmark or runtime test.
What can you do with LiteLLM?
Handle transient failures
Configure retries, cooldowns and fallback deployments for failed requests.
Operate the gateway
Plan credentials and shared usage tracking when deploying multiple proxy instances.
Before you integrate
Plan deployment, provider credentials and shared usage tracking when running the proxy across multiple instances.
Test against your own workload
Compare successful responses, latency and the total bill, including retries and fallbacks. Verify streaming, tool calls and structured output on the routes you plan to use. These are evaluation steps, not claims that every feature is supported.
Check model-provider incidents →Pricing and model access
Model rates, platform fees, credits and eligibility can change. Check the publisher’s current terms before choosing a route. Our calculator estimates model token costs; it does not quote a negotiated LiteLLM bill.
Check LiteLLM documentation ↗Package download trends
Downloads include automated installs and repeat downloads. They do not measure unique users, hosted requests or market share.
litellm
PyPI, excluding mirrors · Python SDK and gateway
88,946,984 downloads in 28 days
2026-09-29 to 2026-10-26
+5.8% versus the previous 28 days
Source and comparison period
Previous: 84,059,002 downloads, 2026-09-01 to 2026-09-28.
Retrieved 2026-10-27. This is a saved snapshot, not a live count.
Sources and corrections
Suggest an update →Compare related routing products
Selected by product type and shared routing capabilities.
Bifrost →
Run an OpenAI-compatible gateway with provider failover, load balancing and a web interface for configuration.
TensorZero →
An open-source gateway with routing, observability and evaluation. Its official repository is archived; assess maintenance before adopting it.
Requesty →
Send model requests through an OpenAI-compatible endpoint with fallback, load-balancing and latency-based routing policies.