Safety & Alignment
aka CAI
Constitutional AI
Anthropic's alignment method using model-generated critiques against a written constitution.
Definition
Constitutional AI trains a model to revise its own outputs against an explicit set of principles ('constitution'). The revised samples form preference data for RL, replacing most of the human labelling in RLHF. It powers Claude's distinctive alignment style.
Common use cases
- Self-supervised alignment
- Safer chat
- Bias reduction