LLM Routing
The right model for
every request. Automatically.
Not every request needs GPT-4o. PromptUnit classifies each call across 10 task dimensions and routes it to the cheapest model that still clears your quality bar.
LLM routing is the practice of directing each AI request to the model best suited for it, balancing cost and quality on a per-request basis rather than defaulting every call to the same flagship model. In production, most AI workloads are a mix of simple and complex tasks. Simple tasks (summarization, classification, structured extraction) do not require the same model capability as complex ones (multi-step reasoning, code generation, research synthesis). Routing separates them automatically.
PromptUnit's routing engine classifies each incoming request across 10 signal dimensions: task type, prompt complexity, context length, output format requirements, presence of tool calls, domain specificity, and more. From those signals, it selects the cheapest model in your provider set that clears your quality threshold for that kind of request. Your application code doesn't change.
The routing engine adjusts to your traffic. Each answer is scored automatically for its form (not empty, not a refusal, not cut off, the format asked for), and repeated requests are counted. Every night it looks at your last 7 days: kinds of requests whose answers score well and are rarely repeated get a slightly lower bar, so cheaper models qualify more often, and kinds that score poorly or get repeated get a higher one. Each model's scores are also corrected by how it did on your traffic over 30 days. The full cycle, with the numbers, is in how your savings grow over time.
Cross-provider routing extends this further. When you connect keys for more than one provider (OpenAI, Anthropic, Google, Groq, DeepSeek), PromptUnit looks for the cheapest qualifying model across all of them for each request, not just within the provider your app calls. Your integration code stays the same.
Task classification
Detects summarization, classification, extraction, reasoning, code generation, and more across 10 signal dimensions.
Quality threshold
You set the quality setting (0–100). Every routing decision is bounded by it.
Cross-provider routing
Searches for the cheapest qualifying model across all connected providers, not just within one.
Adaptive learning
Every night the quality bar for each kind of request moves up or down from your own results, and each model's score is corrected by how it did on your traffic.
Every decision visible
Your dashboard shows each request from the first one: the model you asked for, the model that answered, and what it saved.
Override control
Tag any request with a feature flag to keep it on a specific model, regardless of routing logic.
Routing in practice
Your code sends the same request it always has. PromptUnit intercepts it, classifies the task, selects the optimal model, and returns a standard OpenAI-format response. Your application never knows a routing decision was made.
// Your code, unchanged
const response = await openai.chat.completions.create({
model: "gpt-4o",
messages: [{ role: "user", content: "Summarize this article..." }],
});
// PromptUnit routes it to gpt-4o-mini, 94% cheaper
// Response arrives in standard OpenAI format, your app never notices
Who this is for
LLM routing is the right solution for teams that want to reduce AI inference costs without building and maintaining routing logic themselves. Writing a routing layer in-house requires defining task types, maintaining model benchmarks, handling provider changes, and managing quality regressions, typically 2–4 weeks of engineering effort, with ongoing maintenance cost.
PromptUnit's routing engine is pre-built, self-improving, and connected to a live benchmark database that tracks model quality across providers. Teams that integrate PromptUnit get production-grade routing on day one, with accuracy that improves as their own traffic data accumulates.
For the underlying strategy, static rules vs content-based classification vs ML-based adaptive routing, and how to validate quality before trusting any of them in production, read LLM Model Routing: The Complete Guide. For routing across providers specifically, see Cross-Provider LLM Routing.
Start routing in 5 minutes
Free to start. Savings from your first request. Pay only from savings.
Get Started Free