Concept
Model Router
System that dispatches each request to the most-appropriate model from a pool.
Definition
Model routers — RouteLLM, Martian, custom heuristics — direct cheap queries to small models and hard ones to flagships. They optimise cost-quality tradeoffs at scale and are increasingly bundled into LLM gateways like Portkey and OpenRouter.
Common use cases
- Cost optimisation
- Quality SLAs
- Provider hedging