Concept
Token Budget
Application-set ceiling on tokens consumed per request or per session for cost control.
Definition
Token budgets limit how many tokens a user or feature can spend on LLM calls in a given window. They are enforced via max_tokens, account-level quotas, or LLM-gateway middleware. Without them costs scale unpredictably with user behaviour.
Common use cases
- Cost guards
- Fair-use limits
- Multi-tenant serving