In Part One, we talked strategy why choosing the right base model is the foundation of every fine‑tuning project. Think of it as picking the plot of land before you build a skyscraper. Get the foundation wrong, and no amount of fancy architecture will save you.

Now it’s time to roll up our sleeves. This is where theory meets practice and where the fine‑tuning toolbox comes out. Spoiler: there’s more than one wrench in this kit, and some are shinier than others.

Fine‑tuning isn’t just about pointing an LLM at your data and hoping for the best. It’s about knowing which technique fits your hardware, your dataset, and your patience level. Some methods are heavyweight champions (Full Fine‑Tuning), while others are featherweight ninjas (Prompt‑Tuning, Prefix‑Tuning).

Let’s break them down.

---

Major Fine‑Tuning Techniques for LLMs

1. Full Fine‑Tuning
2. LoRA (Low‑Rank Adaptation)
3. QLoRA (Quantized LoRA)
4. Prefix‑Tuning
5. Prompt‑Tuning
6. Adapters
7. P‑Tuning v2
8. Instruction Fine‑Tuning (Supervised Fine‑Tuning)
9. RLHF (Reinforcement Learning with Human Feedback)

Lets Dive Deeper

---

1. Full Fine‑Tuning

The “all‑you‑can‑train” buffet. You update every single parameter of the model.

Full fine‑tuning of LLMs means updating all parameters of a pre‑trained model on a new dataset, making it the most powerful but also the most resource‑intensive adaptation method. It offers maximum flexibility and accuracy for domain‑specific tasks, but requires huge compute, careful optimization, and strong data curation.

• Definition: Training a pre‑trained LLM (e.g., LLaMA‑3, Gemma‑2B) on a task‑specific dataset while updating every parameter in the network.
• Contrast: Unlike parameter‑efficient fine‑tuning (PEFT) methods (LoRA, adapters, prefix‑tuning), which only adjust a small subset of weights, full fine‑tuning modifies the entire model.
• Goal: Achieve the highest possible alignment with the target domain or task (e.g., legal text, medical records, mining equipment specs).

Technical Workflow

1. Model Initialization
   • Start from a pre‑trained checkpoint (billions of parameters).
   • Freeze nothing every layer is trainable.
2. Dataset Preparation
   • Curate a large, high‑quality, domain‑specific dataset.
   • Ensure balance, deduplication, and provenance tracking (critical for industry datasets).
3. Training Setup
   • Learning rate: Much lower than pre‑training (to avoid catastrophic forgetting).
   • Batch size: Large, often requiring gradient accumulation.
   • Optimizer: AdamW or similar, tuned for stability.
   • Regularization: Dropout, weight decay, gradient clipping.
4. Compute Requirements
   • GPUs/TPUs with high memory (A100/H100 clusters).
   • Distributed training frameworks (DeepSpeed, Megatron‑LM).
   • Checkpointing and mixed‑precision (FP16/BF16) for efficiency.
5. Evaluation
   • Use validation sets for perplexity, accuracy, F1, BLEU, or task‑specific metrics.
   • Monitor overfitting and catastrophic forgetting.

---

2. LoRA (Low‑Rank Adaptation)

LoRA (Low‑Rank Adaptation) fine‑tuning adds small trainable low‑rank matrices to frozen LLM weights, enabling efficient domain adaptation with far fewer parameters. It preserves the base model’s general knowledge, drastically reduces compute cost, and allows modular adapters for different tasks, making it a practical and scalable alternative to full fine‑tuning.

• Definition: A parameter‑efficient fine‑tuning method where instead of updating all weights, you insert small trainable matrices (low‑rank adapters) into the model’s layers.
• Goal: Achieve task/domain adaptation with far fewer trainable parameters, while keeping the original pre‑trained weights frozen.
• Key Insight: Large weight matrices in LLMs are often highly redundant. LoRA approximates their updates using low‑rank decomposition.

Technical Workflow

1. Freeze Base Model
   • The original LLM weights remain untouched.
   • This preserves general knowledge and reduces risk of catastrophic forgetting.
2. Insert LoRA Modules
   • For each target weight matrix $W\in {\mathbb{R}}^{d\times k}$, LoRA adds two small matrices:
     • $A\in {\mathbb{R}}^{d\times r}$
     • $B\in {\mathbb{R}}^{r\times k}$ where $r$ (rank) is much smaller than $d$ or $k$.
   • The effective weight update is $W^{\prime }=W+\mathrm{\Delta }W$, with $\mathrm{\Delta }W=A\cdot B$.
3. Train Only LoRA Parameters
   • During fine‑tuning, only $A$ and $B$ are updated.
   • The number of trainable parameters is reduced by orders of magnitude.
4. Deployment
   • At inference, you can merge $W+\mathrm{\Delta }W$ into a single matrix or keep adapters separate.
   • Multiple LoRA adapters can be swapped in/out for different tasks without retraining the base model.

---

3. QLoRA (Quantized LoRA)

QLoRA (Quantized LoRA) fine‑tuning combines low‑rank adapters with model quantization, allowing large LLMs to be trained efficiently on consumer‑grade hardware. By compressing the base model into lower‑precision formats (like 4‑bit) and only updating lightweight LoRA modules, it drastically reduces memory usage and compute cost while maintaining strong performance, making it ideal for fine‑tuning very large models without massive infrastructure.

• Definition: QLoRA (Quantized LoRA) is a parameter‑efficient fine‑tuning method that combines low‑rank adapters with model quantization.
• Goal: Make fine‑tuning very large LLMs feasible on consumer‑grade hardware by drastically reducing memory usage.
• Key Insight: Compress the base model into lower‑precision formats (like 4‑bit) while training only lightweight LoRA modules, preserving performance at a fraction of the cost.

Technical Workflow

1. Quantization
   • Convert the base model weights into 4‑bit precision.
   • Reduces memory footprint and enables training on smaller GPUs.
2. LoRA Adapter Insertion
   • Add low‑rank trainable matrices into selected layers.
   • These adapters are the only parameters updated.
3. Training
   • Fine‑tune adapters while keeping quantized base frozen.
   • Efficient training with minimal compute and memory.
4. Deployment
   • Adapters can be swapped for different tasks.
   • Quantized base remains reusable across domains.

---

4. Prefix‑Tuning

Prefix‑Tuning fine‑tunes LLMs by learning a small set of continuous prefix vectors that are prepended to the model’s input at each layer. Instead of updating the full network, only these prefix parameters are trained, making it highly parameter‑efficient. It enables strong task adaptation with minimal compute and memory cost, while keeping the base model frozen and reusable across domains.

• Definition: Prefix‑Tuning fine‑tunes LLMs by learning continuous prefix vectors that are prepended to the input at each transformer layer.
• Goal: Adapt the model to specific tasks while keeping the base weights frozen.
• Key Insight: Instead of retraining the full model, Prefix‑Tuning trains only small prefix parameters that steer the model’s internal representations.

Technical Workflow

1. Freeze Base Model
   • The pre‑trained LLM remains unchanged.
   • Preserves general knowledge and reduces compute cost.
2. Prefix Vector Injection
   • Trainable prefix embeddings are added at each transformer layer.
   • These vectors act as “soft context” guiding the model’s attention.
3. Training
   • Only prefix parameters are updated.
   • Requires far fewer parameters than full fine‑tuning.
4. Deployment
   • Prefixes can be swapped for different tasks.
   • Lightweight and efficient for multi‑domain adaptation.

---

5. Prompt‑Tuning

Prompt‑Tuning fine‑tunes LLMs by learning a small set of task‑specific prompt embeddings that guide the model’s behavior without changing its core weights. These continuous prompts are optimized during training and prepended to inputs, making adaptation lightweight and efficient. It’s highly parameter‑efficient, preserves the base model, and works well for specialized tasks with minimal compute cost.

• Definition: Prompt‑Tuning fine‑tunes LLMs by learning small sets of continuous prompt embeddings that guide the model’s behavior.
• Goal: Adapt the model to specific tasks without changing its core weights.
• Key Insight: Instead of retraining billions of parameters, the model learns optimized “soft prompts” that are prepended to inputs, steering outputs toward the desired task.

Technical Workflow

1. Freeze Base Model
   • The pre‑trained LLM remains unchanged.
   • Preserves general knowledge and reduces compute needs.
2. Train Prompt Embeddings
   • Continuous vectors are prepended to the input sequence.
   • These embeddings are the only trainable parameters.
3. Optimization
   • Fine‑tune embeddings on task‑specific data.
   • Lightweight training with minimal compute.
4. Deployment
   • Prompts can be swapped for different tasks.
   • Extremely efficient for multi‑task adaptation.

---

6. Adapters

Adapters fine‑tuning inserts small trainable layers between the frozen layers of an LLM, allowing task‑specific adaptation without updating the full model. Only the adapter parameters are trained, making the process lightweight and efficient. This approach preserves the base model’s knowledge, supports modular deployment, and enables quick switching between domains by loading different adapters.

• Definition: Adapters are small trainable layers inserted between the frozen layers of a pre‑trained LLM.
• Goal: Enable task‑specific adaptation without updating the full model.
• Key Insight: Instead of retraining billions of parameters, adapters add lightweight modules that capture domain knowledge while preserving the base model’s general capabilities.

Technical Workflow

1. Freeze Base Model
   • Original LLM weights remain unchanged.
   • Ensures general knowledge is preserved.
2. Insert Adapter Layers
   • Small neural modules are added between transformer blocks.
   • These layers are the only trainable parameters.
3. Train Adapters
   • Fine‑tune adapters on task‑specific data.
   • Base model remains untouched, reducing compute cost.
4. Deployment
   • Adapters can be swapped in/out for different tasks.
   • Multiple adapters can coexist for multi‑domain usage.

---

7. P‑Tuning v2

P‑Tuning v2 fine‑tunes LLMs by optimizing continuous prompt embeddings across all transformer layers, making it more expressive and stable than earlier prompt‑tuning methods. It keeps the base model frozen, trains only lightweight parameters, and scales efficiently to very large models. This approach achieves strong task performance with minimal compute cost, offering a balance between efficiency and adaptability.

• Definition: P‑Tuning v2 is a parameter‑efficient fine‑tuning method that optimizes continuous prompt embeddings across all transformer layers, not just the input layer.
• Goal: Provide stronger expressiveness and stability than earlier prompt‑tuning approaches, scaling efficiently to very large LLMs.
• Key Insight: By injecting trainable prompt vectors deeper into the model, P‑Tuning v2 achieves richer task adaptation while keeping the base model frozen.

Technical Workflow

1. Freeze Base Model
   • The original LLM weights remain unchanged.
   • Preserves general knowledge and reduces risk of forgetting.
2. Prompt Embedding Injection
   • Trainable continuous prompt vectors are added at multiple transformer layers.
   • These embeddings act like “soft instructions” guiding the model’s internal representations.
3. Training
   • Only prompt embeddings are updated.
   • Requires far fewer parameters than full fine‑tuning.
   • Scales well to very large models (hundreds of billions of parameters).
4. Deployment
   • Prompts can be swapped for different tasks.
   • Lightweight and efficient, suitable for multi‑domain adaptation.

---

8. Instruction Fine‑Tuning (Supervised Fine‑Tuning)

Instruction Fine‑Tuning (Supervised Fine‑Tuning) adapts LLMs by training them on curated instruction–response pairs, teaching the model to follow human‑like prompts more reliably. Unlike parameter‑efficient methods, it updates a significant portion of the model’s parameters, aligning outputs with desired behaviors. This approach improves usability, safety, and task performance, but requires high‑quality labeled data and careful optimization to avoid bias or overfitting.

• Definition: Instruction Fine‑Tuning (also called Supervised Fine‑Tuning, or SFT) adapts LLMs by training them on curated datasets of instruction–response pairs.
• Goal: Teach the model to follow human‑like prompts more reliably and produce useful, aligned outputs.
• Key Insight: Instead of just predicting the next token, the model learns to respond in ways that match human‑written answers to instructions.

Technical Workflow

1. Dataset Preparation
   • Collect high‑quality instruction–response pairs (e.g., “Explain Newton’s laws” → “Newton’s laws state…”).
   • Ensure diversity, balance, and provenance to avoid bias.
2. Model Training
   • Start from a pre‑trained LLM checkpoint.
   • Fine‑tune with supervised learning, updating large portions of parameters.
   • Use smaller learning rates to preserve general knowledge.
3. Evaluation
   • Validate on held‑out instruction datasets.
   • Metrics: helpfulness, accuracy, alignment, and reduced hallucinations.

---

9. RLHF (Reinforcement Learning with Human Feedback)

RLHF fine‑tunes LLMs by combining human preference data with reinforcement learning. First, the model is trained on supervised instruction–response pairs, then a reward model is built from human feedback ranking outputs. Finally, reinforcement learning (often PPO) optimizes the LLM to maximize alignment with human‑preferred responses. This process improves helpfulness, safety, and alignment, but requires extensive human annotation and careful reward modeling.

• Definition: RLHF is a fine‑tuning approach that aligns LLMs with human preferences by combining supervised learning and reinforcement learning.
• Goal: Make models more helpful, safe, and aligned with human values by teaching them to prefer outputs that humans rate highly.
• Key Insight: Instead of just predicting the next token, the model learns to optimize for human‑defined reward signals.

Technical Workflow

1. Supervised Fine‑Tuning (SFT)
   • Train the base LLM on curated instruction–response pairs.
   • This gives the model a baseline ability to follow prompts.
2. Reward Model Training
   • Collect human feedback: annotators rank multiple model outputs for the same prompt.
   • Train a reward model to predict these rankings.
3. Reinforcement Learning
   • Use reinforcement learning (often PPO — Proximal Policy Optimization).
   • The LLM generates responses, the reward model scores them, and PPO updates the LLM to maximize reward.
4. Evaluation
   • Human evaluation remains central.
   • Metrics include helpfulness, harmlessness, and alignment with instructions.

---

Comparison Table

Technique	Params Trained	Hardware Needs	Best Use Case
Full Fine‑Tuning	100%	Very high	Domain adaptation, small models
LoRA	~1–5%	Moderate	Efficient task adaptation
QLoRA	~1–5% + quant.	Low	Large models on consumer GPUs
Prefix‑Tuning	Few million	Very low	Text generation tasks
Prompt‑Tuning	Few million	Very low	Classification, lightweight tasks
Adapters	~1–10%	Moderate	Modular multi‑task setups
P‑Tuning v2	Few million	Low	Better prompt‑tuning
Instruction FT	Variable	Moderate	Aligning with instructions
RLHF	Variable	High	Human‑aligned responses

Trade‑offs

• Efficiency vs. accuracy: The more parameter‑efficient methods (prompt‑tuning, prefix‑tuning) save compute but may underperform compared to LoRA/QLoRA.
• Scalability: QLoRA is currently the most practical for fine‑tuning very large models on limited hardware.
• Alignment: Instruction fine‑tuning and RLHF are essential for safety and usability but require curated datasets and human feedback.

---

Choosing the Right Wrench

Fine‑tuning isn’t a one‑size‑fits‑all game. It’s a toolbox, and each method—whether heavyweight like Full Fine‑Tuning, efficient like LoRA/QLoRA, or featherweight like Prompt‑Tuning and Prefix‑Tuning—has its place. Adapters and P‑Tuning v2 give you modularity and scalability, while Instruction Fine‑Tuning and RLHF push models toward human‑aligned behavior.

The real art lies in matching the technique to your hardware, dataset, and goals. Do you need maximum accuracy for a mission‑critical domain? Reach for Full Fine‑Tuning. Want efficiency and modularity? LoRA or Adapters are your friends. Working with massive models on modest GPUs? QLoRA is your ticket. And if alignment and usability are your north star, Instruction Fine‑Tuning and RLHF are essential.

Think of it this way: the skyscraper you’re building (your fine‑tuned model) will only stand tall if you choose the right tools for the job.

Now it’s your turn to roll up your sleeves:

• Experiment with different fine‑tuning methods on small tasks to see their trade‑offs firsthand.
• Benchmark results not just on accuracy, but also on efficiency, scalability, and usability.
• Share your findings with the community—every experiment adds to the collective knowledge.
• Decide strategically: don’t just ask “Can I fine‑tune this?”—ask “Which fine‑tuning method makes the most sense for my context?”

The toolbox is in your hands. The foundation is set. The skyscraper is waiting. So, which wrench will you pick up first?