How to Fine-Tune an LLM — LoRA, QLoRA, and Full Fine-Tuning (2026)

This how to fine-tune an LLM guide covers the complete pipeline — from dataset preparation to production deployment. We compare LoRA, QLoRA, and full fine-tuning with working Python code, cost analysis across GPU tiers, and the critical distinction between behavioral change and knowledge injection.

Fine-tuning changes model behavior — output format, domain vocabulary, reasoning style — but it does not reliably inject new factual knowledge.

This is the single most important concept to understand before fine-tuning any model:

Fine-tuning changes how a model behaves. It does not reliably add new factual knowledge.

If you want your model to answer in a specific format, adopt a domain vocabulary, follow a particular reasoning style, or refuse certain requests — fine-tuning is the right tool.

If you want your model to know about your company’s internal documents, recent events, or proprietary data — use RAG instead. Fine-tuning might memorize some facts from training data, but it cannot be relied upon for factual accuracy the way a retrieval system can.

For a detailed comparison of when to use each approach, see Fine-Tuning vs RAG.

Fine-tune when you need to change the model’s default behavior in ways that prompting alone cannot achieve:

• Output format: JSON with specific field names, medical report structure, legal document formatting
• Domain vocabulary: Using industry-specific terms correctly and consistently
• Reasoning style: Chain-of-thought for math, step-by-step for diagnostics, concise for chat
• Tone and persona: Customer service voice, technical documentation style, brand voice
• Task specialization: Classification, extraction, summarization in a specific domain
• Refusal behavior: What the model should and should not answer

Do not fine-tune when:

• You need the model to know current information (use RAG)
• Prompt engineering achieves the same result (cheaper, faster, no training needed)
• You have fewer than 50 high-quality training examples
• The behavior you want changes frequently (fine-tuning is slow to update)
• You need the model to cite sources (use RAG with attribution)

Fine-tuning a model for production follows a four-stage pipeline from dataset curation through evaluation to deployment.

LoRA trains less than 1% of model parameters using low-rank adapter matrices, making it the default choice for 90% of fine-tuning tasks.

A pre-trained model has weight matrices W of size (d × d) — for a 7B model, these matrices can be 4096 × 4096. Full fine-tuning updates every value in these matrices.

LoRA decomposes the weight update into two small matrices: A (d × r) and B (r × d), where r is the “rank” (typically 8-64). The effective update is W + A×B. Instead of updating 16 million values (4096²), you update 2 × 4096 × 16 = 131,072 values — a 99.2% reduction.

The insight: weight updates during fine-tuning tend to be low-rank — they can be well-approximated by this decomposition without significant quality loss.

High-quality instruction-response pairs in JSONL format are the foundation of every fine-tuning run — quality consistently outweighs quantity.

Fine-tuning datasets use instruction-response pairs in JSONL format:

{"instruction": "Summarize this medical report in 3 bullet points", "input": "Patient presented with...", "output": "• Diagnosis: Type 2 diabetes\n• Treatment: Metformin 500mg\n• Follow-up: 3 months"}
{"instruction": "Extract the key findings from this research abstract", "input": "We investigated...", "output": "1. Finding A (p<0.001)\n2. Finding B (effect size 0.72)"}

For chat-style fine-tuning, use the conversation format:

{"messages": [{"role": "system", "content": "You are a medical assistant..."}, {"role": "user", "content": "Summarize this report"}, {"role": "assistant", "content": "• Diagnosis: ..."}]}

100 carefully curated, expert-written examples consistently outperform 10,000 noisy, auto-generated examples. Quality guidelines:

• Every example should demonstrate the exact behavior you want
• Remove duplicates and near-duplicates
• Include edge cases and boundary conditions
• Have domain experts review the dataset
• Split 90/10 train/validation — never train on your validation set

The Hugging Face PEFT and TRL libraries handle LoRA training with minimal configuration — the script below covers the complete flow from model loading to adapter saving.

import torch
from datasets import load_dataset
from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
    TrainingArguments,
)
from peft import LoraConfig, get_peft_model, TaskType
from trl import SFTTrainer

# ── Configuration ──────────────────────────────────
MODEL_NAME = "meta-llama/Llama-3.1-8B-Instruct"
DATASET_PATH = "data/training.jsonl"
OUTPUT_DIR = "output/llama-3.1-8b-lora"
LORA_RANK = 16         # Higher = more capacity, more memory
LORA_ALPHA = 32        # Scaling factor (usually 2x rank)
LORA_DROPOUT = 0.05    # Regularization
LEARNING_RATE = 2e-4   # Standard for LoRA
EPOCHS = 3
BATCH_SIZE = 4
MAX_SEQ_LENGTH = 2048

# ── Load model and tokenizer ──────────────────────
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
tokenizer.pad_token = tokenizer.eos_token

model = AutoModelForCausalLM.from_pretrained(
    MODEL_NAME,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

# ── Configure LoRA ─────────────────────────────────
lora_config = LoraConfig(
    task_type=TaskType.CAUSAL_LM,
    r=LORA_RANK,
    lora_alpha=LORA_ALPHA,
    lora_dropout=LORA_DROPOUT,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],  # Attention layers
)

model = get_peft_model(model, lora_config)
model.print_trainable_parameters()
# Output: trainable params: 6,553,600 || all params: 8,030,261,248 || trainable%: 0.0816

# ── Load dataset ───────────────────────────────────
dataset = load_dataset("json", data_files=DATASET_PATH, split="train")
dataset = dataset.train_test_split(test_size=0.1)

# ── Training arguments ─────────────────────────────
training_args = TrainingArguments(
    output_dir=OUTPUT_DIR,
    num_train_epochs=EPOCHS,
    per_device_train_batch_size=BATCH_SIZE,
    learning_rate=LEARNING_RATE,
    warmup_ratio=0.1,
    logging_steps=10,
    eval_strategy="steps",
    eval_steps=50,
    save_strategy="steps",
    save_steps=100,
    bf16=True,
    gradient_accumulation_steps=4,
    report_to="none",
)

# ── Train ──────────────────────────────────────────
trainer = SFTTrainer(
    model=model,
    args=training_args,
    train_dataset=dataset["train"],
    eval_dataset=dataset["test"],
    max_seq_length=MAX_SEQ_LENGTH,
)

trainer.train()
trainer.save_model(OUTPUT_DIR)
print(f"LoRA adapter saved to {OUTPUT_DIR}")

To use QLoRA, add 4-bit quantization when loading the model:

from transformers import BitsAndBytesConfig

quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,  # Nested quantization for extra savings
)

model = AutoModelForCausalLM.from_pretrained(
    MODEL_NAME,
    quantization_config=quantization_config,
    device_map="auto",
)

This reduces memory usage by ~4x, enabling fine-tuning of 70B models on a single A100 80GB.

Never ship a fine-tuned model without measuring its task accuracy, format compliance, and catastrophic forgetting against the base model.

Never ship a fine-tuned model without measuring its quality against the base model:

from transformers import pipeline

# Load base and fine-tuned models
base_pipe = pipeline("text-generation", model=MODEL_NAME, torch_dtype=torch.bfloat16)
tuned_pipe = pipeline("text-generation", model=OUTPUT_DIR, torch_dtype=torch.bfloat16)

# Run evaluation prompts
eval_prompts = [
    "Summarize this medical report: Patient presented with...",
    "Extract entities from: The FDA approved...",
    "Classify this support ticket: My account is locked...",
]

for prompt in eval_prompts:
    base_output = base_pipe(prompt, max_new_tokens=200)[0]["generated_text"]
    tuned_output = tuned_pipe(prompt, max_new_tokens=200)[0]["generated_text"]
    print(f"Prompt: {prompt[:50]}...")
    print(f"Base:  {base_output[:100]}...")
    print(f"Tuned: {tuned_output[:100]}...")
    print("---")

Always test your fine-tuned model on general-purpose tasks it was not fine-tuned for. If a medical fine-tune degrades the model’s ability to write code or answer history questions, you have catastrophic forgetting.

Mitigation: lower the LoRA rank, reduce epochs, or mix general-purpose data into your training set (10-20% of examples).

Merge LoRA adapters into the base model before production deployment to eliminate per-request adapter overhead and simplify serving.

For production deployment, merge LoRA adapters into the base model to eliminate adapter overhead:

from peft import PeftModel
from transformers import AutoModelForCausalLM

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(MODEL_NAME, torch_dtype=torch.bfloat16)

# Load and merge LoRA adapter
model = PeftModel.from_pretrained(base_model, OUTPUT_DIR)
merged_model = model.merge_and_unload()

# Save the merged model
merged_model.save_pretrained("output/llama-3.1-8b-merged")
tokenizer.save_pretrained("output/llama-3.1-8b-merged")

LoRA fine-tuning on a 7-8B model costs $2-4 on a single A100 — roughly 10-100x less than full fine-tuning at comparable quality.

Cloud APIs eliminate GPU provisioning but charge per token:

Cost rule of thumb: For datasets under 10K examples, cloud APIs are often cheaper than provisioning GPUs when you factor in setup time.

The most common fine-tuning failures — overfitting, catastrophic forgetting, and using the wrong tool entirely — are all avoidable with the right process.

Overfitting on small datasets: The model memorizes training examples instead of learning generalizable behavior. Signs: low training loss but high validation loss, model outputs training examples verbatim.

Catastrophic forgetting: Fine-tuning on narrow data degrades general capabilities. The model becomes an expert at your task but loses the ability to do anything else.

Wrong tool for the job: Fine-tuning to inject knowledge that should be retrieved via RAG. The model may memorize facts from training data, but it cannot be updated without retraining.

Hyperparameter sensitivity: LoRA rank too high causes overfitting. Learning rate too high causes instability. Epochs too many causes memorization. Start with conservative defaults and tune one parameter at a time.

Before investing in fine-tuning, try these approaches first (in order):

1. Better prompting — Few-shot examples in the system prompt often achieve 80% of fine-tuning quality
2. System prompt optimization — Iterate on instructions, format examples, and constraints
3. RAG — If the problem is knowledge access, not behavior change, RAG is the right tool
4. Fine-tuning — Only if steps 1-3 demonstrably fail to achieve the required quality

Fine-tuning interviews test trade-off reasoning, not API recall — senior candidates are expected to know when not to fine-tune.

Fine-tuning questions test whether you understand the trade-offs, not whether you can recite the API. Senior candidates are expected to know when not to fine-tune.

Q: “When would you fine-tune a model vs use RAG?”

❌ Weak: “Fine-tuning is for when you want the model to know your data. RAG is for when you want to search documents.”

✅ Strong: “Fine-tuning changes behavior — output format, domain vocabulary, reasoning style. RAG provides knowledge — current data, proprietary documents, citable sources. They solve different problems. If a customer wants their support bot to always respond in JSON with specific fields, that is a fine-tuning problem. If they want the bot to reference their product documentation, that is a RAG problem. In practice, many production systems use both: fine-tune for behavioral consistency, RAG for knowledge retrieval.”

Q: “What is LoRA and why is it preferred over full fine-tuning?”

❌ Weak: “LoRA is faster and uses less memory.”

✅ Strong: “LoRA decomposes the weight update into two low-rank matrices. Instead of updating all 8 billion parameters, you train approximately 6.5 million — a 99.9% reduction. This works because empirically, the weight updates during fine-tuning are low-rank — they can be well-approximated by this decomposition. The practical impact: you can fine-tune an 8B model on a single A100 in 2 hours for $4 instead of 8 hours on 2 GPUs for $48 with full fine-tuning. And because you are modifying fewer parameters, catastrophic forgetting is significantly reduced.”

• Explain how LoRA works at a mathematical level
• What is catastrophic forgetting and how do you mitigate it?
• Design a fine-tuning pipeline for a medical domain application
• How do you evaluate whether fine-tuning improved model quality?
• Compare cloud fine-tuning APIs vs self-hosted training
• When would you use full fine-tuning instead of LoRA?

Production fine-tuning follows one of three patterns depending on team infrastructure, dataset size, and update frequency requirements.

Pattern 1: Managed Cloud Fine-Tuning

Training data (S3/GCS) → Cloud API (Bedrock/Vertex) → Managed endpoint

Best for: teams without GPU infrastructure, fast iteration, small-medium datasets.

Pattern 2: Self-Hosted Training + Cloud Serving

Training data → GPU instance (A100) → LoRA adapter → Merge → vLLM on inference GPU

Best for: teams with ML engineering capacity, cost optimization at scale.

Pattern 3: Continuous Fine-Tuning

Production logs → Data pipeline → Weekly retrain → A/B test → Gradual rollout

Best for: systems where user feedback continuously improves the model.

Fine-tuned models can degrade over time as the data distribution shifts. Monitor:

• Task accuracy on a held-out evaluation set (weekly)
• User feedback signals (thumbs up/down, regeneration rate)
• Latency percentiles (p50, p95, p99) to detect inference issues
• Token usage to catch prompt injection or unexpected input patterns

Use LoRA by default, add RAG when you need factual knowledge, and only escalate to full fine-tuning when you have a large dataset and a demonstrable quality gap.

• Hugging Face PEFT — LoRA, QLoRA, and adapter methods
• Hugging Face TRL — SFTTrainer, DPO, RLHF training
• QLoRA Paper — Original QLoRA research
• LoRA Paper — Original LoRA research
• vLLM — High-throughput inference serving

• Fine-Tuning vs RAG — Detailed decision framework for when to use each approach
• RAG Architecture — How retrieval-augmented generation works
• LLM Evaluation — Measuring model quality systematically
• Vector Database Comparison — Choosing the right database for your RAG pipeline
• GenAI Interview Questions — Practice questions covering fine-tuning, RAG, and system design

Last updated: March 2026. The fine-tuning ecosystem is evolving rapidly; verify current library versions and cloud API pricing against official documentation.