The Ultimate Guide to LLM Fine-Tuning: Full Fine-Tuning, PEFT, LoRA, QLoRA, DoRA & QDoRA
Build an intuitive understanding of LLM fine-tuning from model weights and matrix decomposition to the mathematics behind Full Fine-Tuning, PEFT, LoRA, QLoRA, DoRA, and QDoRACover image generated by the author with ChatGPT.Overview of the fine-tuning methods covered. Image by the author, made with…
Annons
Annons
Build an intuitive understanding of LLM fine-tuning from model weights and matrix decomposition to the mathematics behind Full Fine-Tuning, PEFT, LoRA, QLoRA, DoRA, and QDoRACover image generated by the author with ChatGPT.Overview of the fine-tuning methods covered. Image by the author, made with Napkin AI.A freshly trained large language model is a bit like a brilliant new hire who has read most of the internet but has never once seen your company. It writes well, it reasons, and it knows a little about almost everything. It also has no idea how your support team talks to customers, what your product actually does, or which answers will get you into trouble. Fine-tuning is the onboarding.Once you start reading about it, though, the vocabulary piles up fast. Full fine-tuning, PEFT, LoRA, QLoRA, DoRA, QDoRA. They get thrown around as if they were completely different animals. They are not. Almost all of them are variations on a single idea: how much of the model do you touch, and how cleverly do you touch it?This article builds the whole picture up from first principles. We will start with how a model produces a single word, and end with a method that fine-tunes a model on a laptop GPU. Hold on to that one question: how much of the model are we changing, and the rest falls into place.First, how does a model even answer you?Before we change a model, it helps to be honest about what the model is doing when it answers. It feels like a conversation. Underneath, it is three steps repeated very fast.First, it chops your sentence into pieces (tokenization). A token is usually a word or a fragment of a word, so the word “tokenization” itself might split into “token”, “ization”. Then it reads those tokens in context, picking up grammar, tone, and meaning. Finally, it does the only thing it was ever trained to do: it predicts the most likely next token, as a set of probabilities.An LLM answering you is really three steps on a loop: split the text into tokens, understand the context, then predict the next word by probability.That last part is the whole game. The model is not looking anything up. It is guessing the next token, then the next, then the next. Everything we do in fine-tuning is really about nudging those probabilities so the guesses come out the way we want.Where the model’s knowledge comes fromThose probabilities have to come from somewhere. They come from pre-training.In plain terms, pre-training is where the model reads a gigantic pile of text (web pages, books, code, wikis, and so on) and plays one long game of “guess the next word” over all of it, billions of times. Slowly, its internal numbers (parameters, also called weights) settle into values that make good guesses. That is vanilla pre-training.There is usually a second step. A raw pre-trained model is good at continuing text, but not naturally good at following instructions. So we show it many examples of instruction-and-response pairs, and it learns to actually answer questions instead of just rambling on. This step is instruction tuning.Image by the author.Pre-training teaches the model language by predicting the next word over a huge corpus. Instruction tuning then teaches it to follow instructions instead of just continuing the text.The result is the “base” model you download. Capable, general, and completely unaware of you.Why a pre-trained model is not enoughThat general model has four blind spots that show up the moment you try to use it for real work.Its knowledge is generic, a one-size-fits-all view of the world with none of your specifics. It is static, frozen at whenever training stopped (the knowledge cutoff), so it has not heard about anything since.It is domain blind; it does not know your business, your products, or your internal language. And it can hallucinate, saying something confidently wrong in a tone that sounds exactly like the truth.The four standard weaknesses of an off-the-shelf pre-trained model. Image by the author.None of these are bugs. They are just what “trained on everything, tuned for nobody” looks like. The interesting question is how you fix them.Three ways to improve a model, and where fine-tuning fitsThere are three common tools for closing that gap, and they are not really rivals. They are different jobs.The first is prompt engineering: you simply write better instructions, sometimes with a few worked examples inside the prompt (zero-shot, one-shot or few-shot, depending on how many examples you give). It is the cheapest thing to try, and you should always try it first.The second is Retrieval-Augmented Generation (RAG): you keep your knowledge in a separate searchable store, fetch the relevant pieces at question time, and paste them into the prompt so the model answers from fresh, specific facts. This is the right tool when the problem is a lack of knowledge.The third is fine-tuning: you actually change the model’s weights so the new behavior is baked in. This is the right tool when the problem is behavior, tone, or format — the things a prompt keeps failing to pin down.A useful rule of thumb: prompt first, reach for RAG when the model is missing facts, and fine-tune when you need it to consistently behave a certain way.So what is fine-tuning, really?Strip away the jargon, and fine-tuning is simple. You take a model that has already been pre-trained on the whole world, and you continue training it, this time on a much smaller dataset that is specific to your organization or domain. The model keeps everything it learned before and adjusts to your data.Fine-tuning in one line: start from a base model pre-trained on a giant dataset, keep training it on your own domain data, and you get a model tuned to your needs.That is it. The debates that follow (full fine-tuning versus LoRA versus the rest) are all about one thing: which parts of the model you allow to change, and how you store the change.What fine-tuning changes actuallyWhen people say “fine-tuning,” they can mean two quite different goals, and it helps to name them.Sometimes you want to change behavior: how the model talks, its tone, its format, the way it follows your instructions. Sometimes you want to gain knowledge, to absorb facts and vocabulary that you did not have. And often you want both at once.Fine-tuning is usually aimed at one of two things: changing behavior or adding knowledge, and frequently both together. Image by the author.Being clear about which one you are after saves a lot of wasted training runs. If you only need fresh facts, fine-tuning is often the hard way to get them, and RAG is easier. If you need consistent behavior, fine-tuning is exactly the tool.When should you fine-tune, and when should you not?There is no need to overthink the decision. A few signals tell you fine-tuning is worth it.The classic signals that it is time to fine-tune.Fine-tune when prompts keep falling short, no matter how carefully you write them, when you have a stable body of knowledge that is not changing every day (a static knowledge base), and when you actually have enough good training examples to learn from.It also earns its keep when you need a custom tone and style, when the base model hallucinates badly in your area, or when you work in a niche domain with its own language.The flip side matters just as much. If a good prompt already does the job, do not fine-tune. If your facts change daily, reach for retrieval instead. Fine-tuning is a commitment, and it is worth making only when the cheaper options have run out.Method 1: full fine-tuning (touch everything)The most direct kind of fine-tuning is full fine-tuning, sometimes called instruction fine-tuning when the goal is instruction-following. The idea is exactly what it sounds like: you continue training from the existing checkpoint and update every single parameter in the model. Nothing is frozen.This is the right choice when your task is very different from anything the base model has seen, because you are giving the model complete freedom to change.The problem is cost, and it is easy to underestimate. Picture a model with tens of billions of parameters. Full fine-tuning means updating all of them, and then storing a full, separate copy of the model for every task or customer you tune for. If one copy is dozens of gigabytes, three customers are three of those copies before you have served a single request.On top of the storage, full fine-tuning has a handful of well-known downsides.The recurring costs of full fine-tuning. Image by the author.It is not memory efficient, because you need enough hardware to hold and update the entire model. It risks catastrophic forgetting, where teaching the model your new task quietly erases some of what it already knew.It carries a high computational cost; it can overfit if your dataset is small, and heavy changes can hurt performance on everything downstream. All of which raises an obvious question: do we really need to touch every parameter?The smarter idea: PEFT (turn only a few knobs)The answer, most of the time, is no. That is the insight behind PEFT, Parameter-Efficient Fine-Tuning.Instead of updating the whole network, you freeze the vast majority of it and train only a small subset of parameters. Same model, but you are turning a few carefully chosen knobs instead of rebuilding the machine.Full fine-tuning updates every parameter. PEFT freezes almost everything and updates only a small subset.This buys you three things at once. It overcomes the main drawback of adjusting all the parameters; it only touches a small slice of the model, and because most of the original weights stay frozen, it is far less prone to catastrophic forgetting.PEFT is a family, not a single method. The common members are adapters, prompt tuning, prefix tuning, and then the line we will spend the rest of this article on: LoRA, QLoRA, DoRA, and QDoRA. They build on each other neatly, so we will take them in order.LoRA: do not rewrite the weights; track the changeLoRA (Low-Rank Adaptation) starts from a lovely little reframing. Instead of directly editing the pre-trained weights, it leaves them completely alone and learns the change separately.Think of the original weight matrix W as locked. LoRA introduces a second matrix, the weight update (written ΔW), that sits alongside W. At runtime, the model uses both the frozen W plus the learned ΔW. You never overwrite the original; you just track what should be added to it.Image by the author.LoRA freezes the pre-trained weights W and learns a separate update on top. The update is built from two small matrices, A and B, with B starting at zero, so training begins as a clean no-op.Now for the trick that makes it cheap. That update ΔW is not stored as a full, dense matrix. It is factored into two much thinner matrices, usually called A and B, whose shared inner dimension (the rank, r) is tiny. Multiply the tall skinny one by the short wide one, and you get something the same shape as ΔW, but you only ever had to store the two thin pieces.Image by the author.A full update would be an n by n matrix. LoRA approximates it as a tall (n by r) times a wide (r by n) matrix. With a small rank r, that is a fraction of the numbers.This is why LoRA adapters are so small, often just a few megabytes against a base model of many gigabytes. And because the adapter is a separate little file, you can keep one frozen base model and swap in a different adapter for each task.Image by the author.One frozen base model, many small adapters. Load adapter A for one task, adapter B for another, all sharing the same heavy weights underneath.Image by the author.The practical payoff: tiny per-task adapters riding on one shared base give you high accuracy, easy maintenance, low cost, and low latency.If you want the one-line version of the math, the model output goes from using W alone to using W plus BA. The frozen weights do the heavy lifting, and the small trained pair BA carries your customization.QLoRA: shrink the model first, then LoRALoRA already saves you from storing a full copy per task. But you still have to load the full base model into memory to train against it, and for large models, that alone needs a serious GPU. QLoRA (Quantized LoRA) attacks that problem with a simple recipe: quantize first, then LoRA.Quantizing just means storing each number with fewer bits. A typical model keeps its weights in 16-bit precision. QLoRA squeezes the frozen base down to 4-bit, roughly a quarter of the size, and then trains ordinary LoRA adapters on top.Image by the author.Step one: compress the frozen base weights from 16-bit down to 4-bit. A 7B model drops from about 14 GB to about 3.5 GB. The weights are only ever read during training, never updated, so squeezing them is safe.The clever part is how it computes without giving the savings back. The 4-bit weights stay compressed in memory the whole time. When the model actually needs a layer for the math, it briefly expands just that one layer back to 16-bit (dequantize), does the calculation, and throws the expanded copy away. Only one layer is ever in full precision at a time.Image by the author.Step two: store everything small, compute big for just one layer at a time. Expand a layer to 16-bit, do the math, discard it. Peak memory ends up around 6 GB, not 20.Image by the author.The payoff. Plain LoRA still carries a 14 GB base and wants a powerful GPU. QLoRA carries a 3.5 GB base and runs on a laptop-class GPU. The quality lost to 4-bit is small, and the adapter learns to correct for it.That last point matters. You might expect 4-bit to wreck quality. In practice, the loss is tiny, partly because the trainable adapter adjusts to compensate for it.A quick detour: magnitude and directionTo understand the next two methods, DoRA and QDoRA, we need one small idea about weights. It is worth slowing down for, because everything after it depends on it.Every weight in a model is really a vector, and a vector has two separate properties. It has a magnitude, how long it is, which you can read as its strength or confidence. And it has a direction, where it points, which you can read as what it represents.Image by the author.Two independent properties of a weight vector. Same direction but a different length means the same feature, just stronger or weaker (left). Same length but a different direction means a genuinely different feature (right).So changing a weight’s length turns a feature up or down. Changing its direction swaps it for a different feature. These are two different kinds of edits, and that difference is the whole reason DoRA exists.Why LoRA struggles at sharp turns, and how DoRA fixes itHere is where LoRA shows a weakness. Full fine-tuning can change a weight’s direction and its magnitude independently, because it is free to move each parameter wherever it likes. If it wants to rotate a weight to point somewhere new while keeping its length, it just does.Image by the author.Full fine-tuning (top) can rotate a weight to a new direction and keep its length, moving direction, and magnitude independently. LoRA (bottom) has only one additive update, so a big rotation drags the magnitude along with it.LoRA has only one lever: a single additive update (the BA we met earlier) that has to account for both the change in direction and the change in magnitude at once. The two are coupled.If you want a large change in direction, the only way LoRA can express it is with a large update, and that inflates the magnitude as a side effect, even when you did not want the magnitude to move.Image by the author.The coupling problem. Turning W toward a new direction with one additive vector forces that vector to be long, so the magnitude gets disturbed, too. DoRA’s fix is to separate the two jobs: rotate versus resize.DoRA (Weight-Decomposed Low-Rank Adaptation) fixes this by splitting the weight into its two properties before touching it. It pulls out the magnitude as its own trainable number (m) and the direction as its own component, then uses LoRA only to adjust the direction, and normalizes so the update genuinely changes direction without smuggling in a size change.Finally, it recombines them, which the deck sums up as W′ = m × normalized(V + B·A).Image by the author.DoRA decomposes each weight into magnitude and direction, adapts only the direction with LoRA (and re-normalizes so it stays a pure rotation), then multiplies the learned magnitude back in. Direction and magnitude are trained separately, the way full fine-tuning gets for free.The result is a method that behaves much more like full fine-tuning in quality, while still training only a small set of parameters like LoRA. You get cleaner direction changes because they are no longer fighting the magnitude.QDoRA: DoRA on a 4-bit budgetOnce you have DoRA, QDoRA almost writes itself. Remember that QLoRA was just “quantize the base, then run LoRA on top”. QDoRA is the same move with the better adapter: quantize the base model to 4-bit and keep it frozen, then train a DoRA adapter (the magnitude plus the LoRA direction update) on top of it.QDoRA is QLoRA with DoRA swapped in for plain LoRA. You keep the 4-bit memory savings of QLoRA and gain DoRA’s higher accuracy.So you get the memory savings of 4-bit quantization and the accuracy of DoRA at the same time. It is the newest of the group and still maturing, but the recipe is a clean combination of two ideas you already understand.Putting it all togetherStep back, and the whole family lines up on a single spectrum, trading memory against accuracy and maturity.In practice, here is the short version. Full fine-tuning gives the best raw quality but is heavy and rarely necessary. LoRA is the sensible default: fast, cheap, and small. QLoRA is what you use when you want to fine-tune a large model on modest hardware.DoRA is worth reaching for when you need LoRA’s efficiency but more of full fine-tuning’s quality, and QDoRA combines DoRA’s quality with 4-bit memory savings when you are tight on both.A tiny bit of codeNone of this is as scary in code as it sounds. With the PEFT library, turning a base model into a LoRA-ready one is a few lines. You load the model, describe the adapter in a config (its rank r, which layers to attach to, and so on), and wrap the model.from peft import LoraConfig, get_peft_modelfrom transformers import AutoModelForCausalLM, AutoTokenizer# 1. Load the base modelmodel = AutoModelForCausalLM.from_pretrained("google/gemma-2b-it")tokenizer = AutoTokenizer.from_pretrained("google/gemma-2b-it")# 2. Describe the LoRA adapterlora_config = LoraConfig( r=16, # rank of the update lora_alpha=32, # scaling target_modules=["q_proj", "v_proj"], # which layers get an adapter lora_dropout=0.05, bias="none", task_type="CAUSAL_LM",)# 3. Wrap the model, everything else stays frozenmodel = get_peft_model(model, lora_config)That r=16 is the rank we talked about, and target_modules is you choosing which parts of the model get an adapter. Everything else in the model stays frozen. Move to QLoRA, and you would load the base model in 4-bit before this step. Move to DoRA, and you would flip one flag in the same config.Best practices and the pitfalls to avoidWhatever method you pick, a few habits decide whether the run is worth anything.Data quality and quantity come first, by a wide margin. A smaller set of clean, well-labelled examples beats a mountain of messy ones. After that, hyperparameter tuning (things like the learning rate and the LoRA rank) is what turns a mediocre run into a good one.And you want regular evaluation throughout, not just at the end, so you can watch the model improving or going off the rails while there is still time to react.There are four classic ways for a fine-tune to go wrong, and they are all avoidable once you know their names.Overfitting: the model memorizes your training examples instead of learning the pattern, so it looks great on your data and poor on anything new.Underfitting: the opposite. You did not train enough, or your signal was too weak, and the model never really learned the task.Catastrophic forgetting: learning the new task erases old abilities, the risk we keep coming back to (and the reason PEFT methods, which freeze most of the model, are safer here).Data leakage: information from your test set sneaks into training, giving you scores that look wonderful and mean nothing.The through-lineThat is the whole landscape. Notice that every method we covered is an answer to the same question we started with: how much of the model do you change, and how cleverly do you store the change?Full fine-tuning changes everything. LoRA changes a tiny added-on slice. QLoRA shrinks the base before doing it. DoRA is smarter about what that slice is allowed to move, and QDoRA does all of it on a 4-bit budget.Once you see them as points on that one spectrum, rather than a pile of unrelated acronyms, choosing between them stops being guesswork.This story is published under the Generative AI publication. Connect with us on LinkedIn and follow Zeniteq to stay in the loop with the latest AI stories. Let’s shape the future of AI together!The Ultimate Guide to LLM Fine-Tuning: Full Fine-Tuning, PEFT, LoRA, QLoRA, DoRA & QDoRA was originally published in Generative AI on Medium, where people are continuing the conversation by highlighting and responding to this story.