Technique
aka Low-Rank Adaptation
LoRA
Parameter-efficient fine-tuning that injects small trainable low-rank matrices alongside frozen weights.
Definition
LoRA freezes the base model and adds low-rank update matrices (rank 4–64) to each layer. Training touches <1% of parameters, fitting on a single GPU; multiple LoRAs can be swapped at inference time. It is the dominant open-source fine-tuning method.
Common use cases
- Domain adaptation
- Style transfer
- Character personas