Skip to content

Compare router adoption and package download rankings →

LLM routers & AI gateways.

Estimate your monthly LLM API costs →

Choose the model, control the provider, and plan for failures. Explore the infrastructure between your application and its model APIs.

11 editorial selections · Sources reviewed · Our methodology

Check API and coding-tool incidents →

Explore routing options

These products have different scopes. This is a sourced starting point, not a performance ranking.

11 of 11 routers and gateways

OpenRouter logoHosted gateway

OpenRouter

Access models through one API and control which inference providers serve a request. Provider preferences and fallbacks help manage availability.

Provider selectionFallbacks

Before you integrateCheck provider-level data policies and model parameter support when setting routing preferences.

LiteLLM logoSelf-hosted gateway

LiteLLM

Run a proxy or use a Python router to distribute requests across model deployments, with load balancing, retries, cooldowns and fallbacks.

Load balancingFallbacksPython

Before you integratePlan deployment, provider credentials and shared usage tracking when running the proxy across multiple instances.

Portkey logoAI gateway

Portkey

Use a unified API with conditional routing, provider and model fallbacks, caching and guardrails.

Conditional routingFallbacksGuardrails

Before you integrateReview the configuration and deployment options that apply to your team's governance requirements.

Vercel AI Gateway logoHosted gateway

Vercel AI Gateway

Route requests across providers and fallback models with visibility into routing attempts, latency, token usage and cost.

Provider selectionFallbacksObservability

Before you integrateSet provider preferences explicitly and inspect response metadata to confirm which provider served the request.

Cloudflare AI Gateway logoHosted gateway

Cloudflare AI Gateway

Manage model requests with logging, caching and rate limits, plus fallback providers for errors or configured timeouts.

FallbacksObservabilityCaching

Before you integrateDefine error and timeout behavior, and verify the authentication requirements of the endpoint you use.

Model-selection framework

RouteLLM

Serve and evaluate learned routers that choose between a stronger and a weaker model using a configurable cost-quality threshold.

Model selectionEvaluationPython

Before you integrateCalibrate thresholds on representative queries and evaluate the model pair you actually intend to deploy.

Merge Gateway logoHosted gateway

Merge Gateway

Use one inference API with model routing, budgets and data controls. Choose ordered failover, intelligent selection or a custom benchmark-weighted routing policy.

Model selectionFallbacksSpend controls

Before you integrateUse an inference key rather than a management key. Check model eligibility against vendor, region, credential and data-retention restrictions.

Helicone AI Gateway logoHosted gateway

Helicone AI Gateway

Route model requests through an OpenAI-compatible endpoint with provider fallbacks and request observability.

FallbacksObservability

Before you integrateCheck provider routing, credential handling and logging settings for the models you plan to use.

Bifrost logoSelf-hosted gateway

Bifrost

Run an OpenAI-compatible gateway with provider failover, load balancing and a web interface for configuration.

Load balancingFallbacksCaching

Before you integratePlan provider credentials and gateway hosting. Check which governance capabilities require an enterprise deployment.

TensorZero logoSelf-hosted gateway

TensorZero

An open-source gateway with routing, observability and evaluation. Its official repository is archived; assess maintenance before adopting it.

FallbacksEvaluationPython

Before you integrateThe official repository was archived on June 12, 2026. Review maintenance and dependency risks before deploying. A published Python package does not establish ongoing support.

Requesty logoHosted gateway

Requesty

Send model requests through an OpenAI-compatible endpoint with fallback, load-balancing and latency-based routing policies.

FallbacksLoad balancingObservability

Before you integrateUse a Requesty key and configure the gateway base URL. Confirm current platform fees and data-handling terms directly with the publisher.

RequestYour application

Prompt, context and requirements

RoutingChoose a route

Model policy, provider preference or fallback

InferenceModel provider API

The selected provider runs the model

LLM routing, explained.

What is an LLM router?

An LLM router selects the model or inference provider that receives a request. Routing may follow explicit rules, availability, cost or latency preferences, or a learned model-selection policy. The supported policies depend on the product.

How is model routing different from provider routing?

Model routing chooses which model should answer a request. Provider routing chooses where a selected model runs. A fallback can switch providers or models after a failure; that is different from choosing a model based on the task before the first call.

Is an AI gateway the same as an LLM router?

A gateway provides a common entry point for model API requests and may add authentication, logging, caching and usage controls. Routing is one capability a gateway can provide. A model-selection framework may instead run inside your application without being a full gateway.

Does routing guarantee lower cost or better answers?

No. Evaluate quality, total request cost and latency on your own workload. Include retries and fallback calls in the measurements, and check tool calling, structured output and data-handling requirements before switching models or providers.

How does an LLM router relate to MCP Connect?

An LLM router directs inference requests to models or providers. MCP Connect connects MCP servers and routes tool calls. An agent can use both: a model reasons about a task, then calls tools through MCP.

Search API Tracker

Curated starting points