🔮The Codex
LoRA (Low-Rank Adaptation)
An efficient method for customizing AI models without retraining the entire model.
📖 Apprentice Explanation
LoRA is a shortcut for customizing AI models. Instead of retraining the whole model (expensive and slow), LoRA adds small, trainable pieces that adapt the model for specific tasks.
🧙 Archmage Notes
LoRA decomposes weight update matrices into low-rank factors, reducing trainable parameters by 10-100x. QLoRA combines LoRA with 4-bit quantization for even greater efficiency. Rank selection (typically 8-64) balances quality and efficiency.