Technique
Context Distillation
Fine-tuning a model so that previously long system prompts are baked into its weights.
Definition
Context distillation trains a model on (prompt, response) pairs generated by the same model with an elaborate system prompt, so the behaviour persists without the prompt at inference time. It cuts cost and latency for fixed personas or policies.
Common use cases
- Persona compression
- Latency reduction
- Cost optimisation