Technique
Prompt Caching
Reusing computation from previously-seen prompt prefixes across requests.
Definition
Prompt caching stores the KV-cache state of recurring prompt prefixes — long system messages, RAG contexts — and reuses it on subsequent requests, slashing both latency and cost. Anthropic, OpenAI and Google offer it as a first-class API feature.
Common use cases
- Long system prompts
- RAG
- Cost reduction