Technique
Prompt Tuning
Learning a small set of continuous 'soft prompt' embeddings while keeping the model frozen.
Definition
Prompt tuning learns a handful of virtual token embeddings prepended to the input. The base model is unchanged. At scale (10B+ parameters) it matches full fine-tuning on many tasks while training <0.1% of parameters.
Common use cases
- Multi-task serving
- Cheap adaptation
- Few-shot specialisation