Technique
Model Merging
Combining the weights of multiple fine-tuned models into a single composite model.
Definition
Model merging (SLERP, task arithmetic, TIES, DARE) averages or blends fine-tuned checkpoints to combine capabilities without re-training. It has become a staple of the open community for producing strong general-purpose Llama or Mistral variants.
Common use cases
- Capability composition
- Open community models
- Cost-efficient specialisation