Concept
Synthetic Data
Training data generated by another model rather than collected from humans or the web.
Definition
Synthetic data — generated by an LLM, rendered, simulated, or augmented — supplements scarce real data. Phi was trained heavily on synthetic 'textbook-quality' examples; many fine-tunes use synthetic instruction data. Quality control and contamination are central concerns.
Common use cases
- Edge-case coverage
- Data augmentation
- Capability targeting