Fine-tuning and LoRA
Teaching a pretrained model a new job by training a small add-on instead of every weight.
Read first:Pretraining and scaling lawsBackpropagation
Step 1 of 6· Miniature 16 × 16 matrices with illustrative values; the B·A product, parameter counts and GPT-3 accuracies are real (Hu et al., 2021)
A pretrained model knows a lot about language but not your task, tone or format. Fine-tuning continues its training on a smaller set of examples of the behaviour you want. Full fine-tuning updates every weight, which for a large model needs a lot of memory and a full-size copy for each task. LoRA freezes the original weights and learns a small correction for chosen weight matrices, written as the product of two thin matrices. Only the thin matrices are trained, sometimes as little as 0.01% of the model, and the correction can be added back into the weights so the model runs exactly as fast as before.
Why it matters for your product
Fine-tuning is how a general model learns a house style, a strict output format or a narrow task. LoRA makes that affordable: training needs far less GPU memory, each adapter is megabytes rather than a full model, and one base model can serve many adapters, one per task or customer. QLoRA goes further by keeping the frozen base in 4-bit precision. Try prompting and retrieval first, since they are cheaper to change, and fine-tune when you need consistent behaviour they cannot deliver.
For engineersShow the maths
h = W₀·x + B·A·x, B ∈ ℝ^(d×r), A ∈ ℝ^(r×k), r ≪ min(d, k)
The layer's output is the frozen matrix's output plus a correction that squeezes the input through a narrow r-dimensional bottleneck and expands it again.
trainable = r · (d + k) instead of d · k
LoRA trains one tall, thin matrix and one short, wide matrix instead of the whole grid.
Worked example: For d = k = 12,288 and r = 4: 4 × 24,576 = 98,304 trainable numbers instead of 150,994,944, about 0.065%.
W_merged = W₀ + (α / r) · B·A
After training, the scaled correction is added into the frozen matrix once, so serving costs exactly what the original model cost.
Worked example: Storing 100 task versions of GPT-3 as adapters takes about 350 GB + 100 × 35 MB ≈ 354 GB, instead of 100 × 350 GB ≈ 35 TB.
On demanding targets such as programming and maths, LoRA at typical ranks has been shown to learn less than full fine-tuning, though it also forgets less of what the model already knew. Once an adapter is merged into the weights, requests for different tasks can no longer share one batch as easily.
The words you will hear
Fine-tuning
Primary sources
- 01LoRA: Low-Rank Adaptation of Large Language ModelsHu et al. · 2021
- 02QLoRA: Efficient Finetuning of Quantized LLMsDettmers et al. · 2023
- 03LoRA Learns Less and Forgets LessBiderman et al. · 2024
Connected ideas
Speed and cost
Quantisation
Storing a model's numbers with fewer bits, and keeping the answers nearly the same.
ExploreTraining and alignment
RLHF: learning from human feedback
How people's choices between answers become a reward that steers a model.
ExploreTraining and alignment
Distillation
Training a small model to imitate a large one, including what the large one nearly said.
ExploreRetrieval and search
Retrieval-augmented generation (RAG)
How a model answers from your documents, and shows you exactly where each claim came from.
ExploreWant this working on your data?
We design and build the systems these ideas power: retrieval, agents, voice and the models behind them. Start with a free discovery call.