祝融说。 Chapter 8: Customizing Your Own LLM: Efficient Fine-Tuning in Practice

Chapter 8: Customizing Your Own LLM: Efficient Fine-Tuning in Practice

2026.08.10

In the previous chapter, we took in the grand landscape of large language models (LLMs). We know that open-source LLMs like Llama and Qwen are "general-purpose brains" pretrained on massive amounts of general text. They are knowledgeable and powerful. Through careful prompt engineering, we can already guide them to produce satisfactory results on many tasks.

However, in many real business scenarios, the challenges we face are more specific and profound:

We need a customer service bot that can understand and use company-internal terminology and jargon.
We need an assistant that can imitate a specific writer's style for creative writing.
We need a code generator that can accurately translate natural language queries into a specific database query language (like SQL).

In these scenarios, relying solely on prompt engineering may fall short. A general-purpose LLM might understand "what a database is," but it does not know the specific structure of your company's intricate "user behavior table." This is where we need a more powerful technique to transform this "general-purpose brain" into the "domain expert" we need. That technique is fine-tuning.

Fine-tuning, as the name suggests, involves making "small" adjustments to the parameters of a pretrained large model using a smaller dataset specific to a particular task or domain. This process is like providing targeted on-the-job training to a knowledgeable generalist, enabling them to quickly acquire the specialized knowledge and skills required for a specific role.

However, traditional full fine-tuning — updating all of a model's parameters — is an extremely expensive "gamble" for LLMs with billions or even hundreds of billions of parameters. It not only demands massive GPU memory (a rough estimate: the weights alone of a 70B model in FP16 take about 140GB, and with optimizer states and gradients, full fine-tuning typically requires multiple GPUs and well beyond that figure — the exact requirement depends on precision, parallelism strategy, and implementation) but also produces a completely new model copy the same size as the original, which is a huge burden for storage and deployment.

Fortunately, researchers have proposed a range of Parameter-Efficient Fine-Tuning (PEFT) techniques that have completely changed the game. The core idea of PEFT is: during fine-tuning, freeze most of the original LLM's parameters and only introduce or modify a small fraction (typically less than 1%) of new parameters. This allows us to achieve results comparable to or even better than full fine-tuning at a fraction of the resource cost (even on a single consumer-grade GPU).

In this chapter, we will deeply explore this exciting field. We will learn:

The Principles of Fine-Tuning: We will clarify why we fine-tune and when to choose fine-tuning over relying on prompts or RAG.
Overview of PEFT Techniques: We will survey mainstream PEFT methods such as Prefix-Tuning, P-Tuning, and Prompt Tuning, understanding their core ideas.
LoRA and QLoRA: We will focus on dissecting the currently most popular and practical PEFT method — LoRA (Low-Rank Adaptation) — and learn about its variant QLoRA that further reduces resource consumption. We will delve into its mathematical principles and provide clear code implementations.
Data Engineering: High-quality data is the key to successful fine-tuning. We will learn how to construct instruction-tuning datasets in specific formats, the "spiritual nourishment" fed to the model for learning.
Hands-On Project: Through a complete hands-on project — fine-tuning a 7B open-source LLM on a single GPU using LoRA to enable it to answer domain-specific Q&A — we will integrate all the knowledge points in this chapter.

Mastering efficient fine-tuning techniques gives you the "philosopher's stone" to transform a general-purpose AI into a specialized AI. You will be able to create unique, highly competitive custom LLMs for your business scenarios at a controllable cost. Now, let us begin this exciting journey of "taming" the giant.

8.1 The Principles of Fine-Tuning: Why and When to Fine-Tune

8.1.1 What Is Fine-Tuning?

Fine-tuning is the process of continuing to train a model that has already been pretrained on large-scale data (a pretrained model) using a smaller, more targeted dataset to adapt the model to a specific task or domain.

This process can be likened to human learning:

Pretraining: Like a person receiving general education from primary school through university, learning vast amounts of general knowledge in language, math, history, science, etc. This stage shapes their worldview and basic cognitive abilities. An LLM's pretraining is done on internet-scale text.
Fine-tuning: Like this person, after graduating from university, joining a law firm. They need to learn legal terminology, case analysis methods, courtroom argument techniques, and other specialized knowledge. This "on-the-job training" is fine-tuning. They use their powerful general language and logical abilities to quickly absorb specialized knowledge in the legal field and become a lawyer.

During fine-tuning, the model's weights are not randomly initialized but are inherited from the pretraining stage. We continue training these weights at a smaller learning rate using task-specific data (e.g., legal Q&A pairs), making small adjustments so they better fit the new task distribution.

8.1.2 Why Fine-Tune? — The Benefits of Fine-Tuning

Since we already have prompt engineering and RAG (Retrieval-Augmented Generation), why do we need such a "heavy" operation as fine-tuning? Fine-tuning brings unique and irreplaceable value:

Injecting Domain Knowledge: This is one of fine-tuning's most core values. While RAG can provide the model with external knowledge, fine-tuning internalizes knowledge into the model's parameters. This allows the model to use technical terms more naturally and fluently, and understand subtle relationships within the domain. For example, by fine-tuning on medical literature, a model can learn to think and express itself like a doctor.
Learning Specific Styles or Formats: If you want the model's output to have a specific style (like Shakespearean, official company tone) or follow a complex output format (like generating specific JSON or XML structures), fine-tuning is more reliable and efficient than repeatedly describing it in prompts. The model "learns" the pattern from the data.
Acquiring New Capabilities: Some capabilities are difficult to teach through prompts. For example, translating natural language into a company-proprietary DSL that has never appeared on the internet. By providing large amounts of paired (natural language, DSL) data for fine-tuning, the model can learn this new "translation" ability.
Improving Reliability and Consistency: For tasks that need to be performed at scale and repeatedly, relying on complex few-shot prompts can lead to unstable outputs. A fine-tuned model's behavior is more predictable and consistent, reducing reliance on extremely long, elaborate prompts.
Optimizing Inference Cost: A smaller, fine-tuned model (e.g., 7B) may outperform a larger general-purpose model (e.g., 70B) that requires complex few-shot prompts on a specific task. In production, using a smaller model means lower inference latency and cost. Additionally, after fine-tuning, your prompts can become shorter and simpler, also reducing token consumption per API call.

8.1.3 When to Fine-Tune? — A Technology Selection Decision

Fine-tuning is not a panacea; it comes with costs (data, computation, time). Before deciding whether to fine-tune, you should first try more lightweight methods. A typical technology selection path is as follows:

Step 1: Prompt Engineering (Zero Cost)

For simple, one-off tasks, or when you first start exploring a new scenario, first try to solve the problem through carefully designed prompts (including Zero-shot, Few-shot, CoT, etc.).

Applicable Scenarios: General QA, simple summarization, text polishing, creative generation.

If the results meet the requirements, stop here.

Step 2: Retrieval-Augmented Generation (RAG) (Medium Cost)

If the model's main problem is "insufficient knowledge" or "hallucination" (e.g., answering questions about internal company products or the latest news), then RAG is the first choice.

Applicable Scenarios: Building Q&A systems based on private knowledge bases, report generation requiring source citations, tasks requiring real-time information.

Cost: Requires building and maintaining a knowledge base and retrieval system.

If RAG solves the problem, fine-tuning is usually not needed. RAG and fine-tuning are not mutually exclusive and can be used together (first fine-tune the model to adapt to domain language style, then provide real-time knowledge through RAG).

Step 3: Parameter-Efficient Fine-Tuning (PEFT) (Medium-High Cost)

When you have tried the first two steps but the model still cannot meet the following requirements, you should consider fine-tuning:

Needs deep adaptation to domain terminology and language style.
Needs to learn a new, complex output format or capability.
Has extremely high requirements for performance and reliability on specific tasks.
Needs to optimize inference cost by shortening prompts or using smaller models.

Cost: Requires collecting and annotating high-quality fine-tuning datasets, and investing a certain amount of computational resources.

Step 4: Full Fine-Tuning (Extremely High Cost)

With today's mature PEFT techniques, the demand for full fine-tuning has significantly decreased. It is typically only considered in the following cases:

You have massive, high-quality domain data.
Your task is drastically different from the model's original pretraining task.
You are pursuing ultimate performance, regardless of cost.
In most cases, PEFT is a more cost-effective choice than full fine-tuning.

Decision Flowchart:

graph TD
    A[Start: Define Task Requirements] --> B{Try Prompt Engineering};
    B -- Results Satisfactory? --> C[Done];
    B -- Results Unsatisfactory --> D{Problem is insufficient knowledge/hallucination?};
    D -- Yes --> E[Implement RAG];
    E -- Results Satisfactory? --> C;
    E -- Results Unsatisfactory/Need Combination --> F{Need to learn style/format/new capability?};
    D -- No --> F;
    F -- Yes --> G[Implement PEFT Fine-Tuning];
    G -- Results Satisfactory? --> C;
    G -- Results Unsatisfactory/Pursuing Ultimate Performance --> H[Consider Full Fine-Tuning];

8.2 Overview of Parameter-Efficient Fine-Tuning (PEFT)

The core idea of PEFT is: during fine-tuning, keep the main parameters Φ of the pretrained model frozen, and only adjust a small set of additionally added or selectively unfrozen parameters Δθ. Since |Δθ| << |Φ|, this greatly reduces computational and storage overhead.

Based on where and how parameters are modified, PEFT methods can be divided into three main categories:

8.2.1 Adapter-Based Methods: "Inserting" New Modules into the Model

Representative Method: Adapter Tuning

Idea: Inside each Transformer layer, insert two small, bottleneck-structured feedforward neural network modules (called Adapters). During fine-tuning, only the parameters of these newly inserted Adapter modules are trained, while the original Self-Attention and Feed-Forward layers are frozen.

Advantages: Simple to implement, stable performance.

Disadvantages: The inserted Adapter modules increase inference latency.

8.2.2 Prompt-Based Methods: "Adding" Learnable Prompts at the Input

These methods do not modify the model structure but instead work on the input layer.

Representative Methods: Prefix-Tuning, P-Tuning, Prompt Tuning

Idea:

Prompt Tuning: Prepend some learnable, continuous "virtual tokens" (Soft Prompts) to the input word embedding sequence. During fine-tuning, only the embeddings of these virtual tokens are updated, while all other parts of the model (including the word embedding layer) remain frozen.
Prefix-Tuning / P-Tuning: Similar to Prompt Tuning, but these add learnable "prefixes" to every layer of the Transformer, not just the input layer, giving the model greater adjustment freedom.

Advantages: No change to model structure, no increase in inference latency.

Disadvantages: Performance is sometimes less stable than Adapter methods, and the learnable prompt length is a difficult hyperparameter to tune.

8.2.3 Reparameterization-Based Methods: "Low-Rank" Modifications to Weight Matrices

This category is currently the most mainstream and effective PEFT paradigm.

Representative Method: LoRA (Low-Rank Adaptation)

Idea: It is based on the assumption that the change in model weights during fine-tuning has a "low rank." That is, the large weight update matrix ΔW can be approximated by the product of two smaller, low-rank matrices A and B: ΔW ≈ B A.

Implementation: During fine-tuning, LoRA freezes the original weight matrix W and connects a "bypass" consisting of matrices A and B in parallel. We only train the parameters of A and B. During inference, the trained B*A can be merged with the original W (W' = W + B*A), thus introducing zero additional inference latency.

Advantages: Performance is very close to full fine-tuning, with no extra overhead during inference. Flexible implementation, can be applied to any linear layer.

Disadvantages: Not optimal for all tasks and rank choices (e.g., for knowledge-intensive injection it may fall short of full fine-tuning), but it is now the mainstream de facto standard for PEFT.

We will dive into the principles and implementation of LoRA in the next section.

8.3 LoRA and QLoRA: Principles and Code Implementation

8.3.1 Core Principles of LoRA

Suppose we have a pretrained weight matrix W₀ R^(dk) (e.g., the linear layer weights for Q, K, V projections in a Transformer). In full fine-tuning, we update this matrix to get W₀ + ΔW. LoRA's core insight is that this update matrix ΔW has a very low "intrinsic rank," meaning it can be represented with less information.

LoRA's approach is to use two low-rank matrices B R^(dr) and A R^(rk) to represent ΔW, where the rank r is a hyperparameter much smaller than d and k (e.g., r=8, 16, 64).

Change in Forward Propagation:

Original path: h = W₀ x
LoRA path: h = W₀ x + B A x

To further reduce computation, LoRA also introduces a scaling factor α: h = W₀ x + (α/r) B A x

Training Process:

Freeze the original weights W₀.
Randomly initialize matrix A (e.g., Gaussian distribution) and matrix B (initialized to 0).
During fine-tuning, only update the parameters of A and B.

Comparison of Trainable Parameters:

Full fine-tuning: d k
LoRA: r d + r k = r (d + k)

Since r << d and r << k, LoRA drastically reduces the number of trainable parameters. For example, for a 4096x4096 matrix, full fine-tuning needs to update about 16.7M parameters. Using LoRA with r=8, only 8 (4096 + 4096) ≈ 65k parameters need updating — a reduction of over 250x!

Inference Process (Weight Merging):

After training, we can merge the LoRA bypass back into the main model, achieving zero additional inference latency.

Compute the trained ΔW = (α/r) B A.
Compute the new weight matrix W' = W₀ + ΔW.
During deployment, use this merged weight matrix W' directly, completely discarding A and B.

8.3.2 QLoRA: Fine-Tuning Giant Models on Consumer-Grade GPUs

LoRA has already greatly lowered the barrier to fine-tuning, but for very large models (like 70B), loading the model itself requires a huge amount of VRAM. QLoRA (Quantized LoRA) pushes resource consumption to the extreme.

QLoRA combines two techniques:

4-bit NormalFloat (NF4) Quantization: This is QLoRA's core innovation. It quantizes the frozen weights of the pretrained model (i.e., W₀) from standard 16-bit floating point (FP16) or 32-bit floating point (FP32) into a new 4-bit data type (NF4). This reduces the memory required to load the model by about 4x. For example, a 7B model requires about 14GB VRAM in FP16, but only about 4-5GB after 4-bit quantization.
Double Quantization: To further save memory, QLoRA performs a second quantization on the "quantization constants" themselves produced during quantization.
Paged Optimizers: Leverages NVIDIA Unified Memory to prevent out-of-memory (OOM) errors when processing long sequences that cause gradient checkpointing memory spikes.

QLoRA Training Process:

Load the pretrained model's weights in 4-bit NF4 format onto the GPU.
Insert LoRA adapters (matrices A and B).
During training, only LoRA's parameters (A and B) are computed and updated in high precision (e.g., FP16).
When forward and backward propagation require the original weights W₀, the system dynamically dequantizes the 4-bit weights to FP16, performs the computation, and immediately discards them afterward. The memory always retains only the 4-bit weights.

In this way, QLoRA maintains almost the same performance as 16-bit LoRA fine-tuning while drastically reducing peak VRAM usage. The original paper (QLoRA, arXiv:2305.14314) reports fine-tuning a 65B model on a single 48GB GPU; on consumer-grade cards, the more realistic operating range is fine-tuning 7B–13B-class models on a single 24GB GPU (like an RTX 3090/4090), and follow-up independent reproductions suggest the actual VRAM requirement for 65B-class models may exceed the paper's reported figure.

8.3.3 Implementing LoRA/QLoRA with the `peft` Library

Hugging Face's peft library greatly simplifies the implementation of PEFT methods.

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import get_peft_model, LoraConfig, TaskType

# Model name
model_name = "meta-llama/Llama-2-7b-hf"

# --- QLoRA Configuration ---
# 1. BitsAndBytesConfig for 4-bit quantization
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16 # Computation dtype
)

# 2. Load the quantized model
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    quantization_config=bnb_config,
    device_map="auto" # Automatically distribute the model across available devices
)

# Load the tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_name)
tokenizer.pad_token = tokenizer.eos_token # Set pad token

# --- LoRA Configuration ---
# 3. LoraConfig
peft_config = LoraConfig(
    task_type=TaskType.CAUSAL_LM, # Task type: causal language model
    r=8,                          # LoRA rank
    lora_alpha=32,                # LoRA alpha parameter
    lora_dropout=0.1,             # Dropout rate for LoRA layers
    # Specify the module names to apply LoRA to, typically Q and V projection layers
    target_modules=["q_proj", "v_proj"], 
    bias="none"
)

# 4. Use get_peft_model to apply LoRA to the model
peft_model = get_peft_model(model, peft_config)

# Print trainable parameters
peft_model.print_trainable_parameters()
# Output: trainable params: 4,194,304 || all params: 6,742,609,920 || trainable%: 0.0622...

With just a few lines of code, we have transformed a massive LLM into a lightweight, fine-tunable PEFT model. Next, we can train this peft_model just like a regular PyTorch model, using transformers.Trainer or a custom training loop.

8.4 Data Engineering: Building High-Quality Instruction-Tuning Datasets

"Garbage in, garbage out." This saying is fully realized in LLM fine-tuning. The quality of the dataset is the single most critical factor determining the success or failure of fine-tuning.

Instruction-tuning is the most mainstream fine-tuning paradigm today. Its core idea is to transform various tasks into an "instruction-response" format, teaching the model to follow human instructions.

8.4.1 Dataset Format

A typical instruction-tuning dataset is usually a JSONL file (one JSON object per line), where each JSON object contains the following fields: instruction: A clear description of the task. input (optional): The context or input for the task. output: The desired standard answer for the model to generate.

Examples:

Q&A task without input

{"instruction": "What is the capital of China?", "input": "", "output": "The capital of China is Beijing."}

Classification task with input

{"instruction": "Please determine whether the sentiment of the following sentence is positive, negative, or neutral.", "input": "The food at this restaurant tastes great, but the service is too slow.", "output": "Neutral"}

Code generation task

{"instruction": "Write a Python function to calculate the nth term of the Fibonacci sequence.", "input": "n = 10", "output": "def fibonacci(n):\n  if n <= 1:\n    return n\n  else:\n    return fibonacci(n-1) + fibonacci(n-2)"}

8.4.2 Prompt Templating

Before feeding the data to the model, we need to combine these structured fields into a single text string. This process is called prompt templating. A good template clearly shows the task structure to the model.

A commonly used template (Alpaca format):

Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
{instruction}

### Input:
{input}

### Response:
{output}

If input is empty, the ### Input: part can be omitted. During training, the model's goal is to generate the Response part based on the Instruction and Input.

8.4.3 Principles for Building High-Quality Datasets

Quality is far more important than quantity: A few hundred high-quality, carefully designed data points can be far more effective than tens of thousands of low-quality, noisy data points.
Diversity: Instruction Diversity: The same task can be asked in many different ways. For example, "Summarize this article," "Write an abstract for this article," "What is the core viewpoint of this article?" Input Diversity: Cover a variety of possible input scenarios and edge cases. Response Diversity: For open-ended questions, the answers should also be diverse.
Clear and Unambiguous: Instructions should be clear and precise, avoiding vague descriptions.
Correctness: The output must be absolutely correct.
Conciseness: While maintaining clarity, instructions and input should be as concise as possible.

8.4.4 Data Sources

Using Existing Open-Source Datasets:

Alpaca: 52k instruction data released by Stanford, generated by GPT-3.5. The pioneering work in instruction tuning.
Dolly: 15k high-quality instruction data crowdsourced by Databricks employees.
Open-Orca: A large dataset containing millions of instruction data generated by stronger models (e.g., GPT-4).

Generating Your Own Data:

Human Annotation: Highest cost, but best quality. Domain experts write instructions and answers firsthand.
Generating with Strong Models (e.g., GPT-4): Design some high-quality "seed prompts," then have GPT-4 generate large numbers of Q&A pairs, classifications, summaries, and other data around your domain. This is currently the most common and cost-effective method. However, the generated data must undergo strict human screening and revision.

Extracting from Existing Documents: Convert company FAQs, product manuals, API documentation, etc., into Q&A pair format.

8.5 Hands-On Project: Fine-Tuning a 7B Model for Domain-Specific Q&A with LoRA

Project Objective: We will use a custom small dataset about "basic AI knowledge" to fine-tune a 7B open-source model (such as Llama 3 8B or Qwen 7B) using QLoRA, enabling it to answer related AI concept questions accurately.

Tech Stack:

transformers
peft
accelerate (for distributed training and device management)
bitsandbytes (for QLoRA)
datasets (for loading and processing data)

Step 1: Prepare the Dataset (ai_knowledge_qa.jsonl)

{"instruction": "What is a neural network?", "input": "", "output": "A neural network is a computational model that mimics the structure and function of a biological brain, consisting of a large number of artificial nodes called neurons organized in a hierarchical structure. It learns and recognizes patterns by training on large amounts of data."}
{"instruction": "Please explain what backpropagation is.", "input": "", "output": "Backpropagation is a core algorithm for training models in neural networks. It calculates the gradient of the loss function with respect to each parameter of the network, and then uses optimization algorithms like gradient descent to update the weights, thereby minimizing prediction error."}
{"instruction": "What is the core idea of the Transformer model?", "input": "", "output": "The core idea of the Transformer model is the Self-Attention mechanism. It completely abandons the traditional recurrent (RNN) and convolutional (CNN) structures, allowing the model, when processing one word in a sequence, to directly calculate and attend to the importance of all other words in the sequence, thereby efficiently capturing long-distance dependencies."}
... (Prepare 100-500 similar data points)

Step 2: Write the Fine-Tuning Script (finetune.py)

import torch
from datasets import load_dataset
from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
    BitsAndBytesConfig,
    TrainingArguments,
    Trainer,
)
from peft import LoraConfig, get_peft_model

# --- 1. Load Model and Tokenizer (QLoRA Configuration) ---
model_name = "meta-llama/Llama-3-8B" # Or another 7B/8B model

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    quantization_config=bnb_config,
    device_map="auto",
)
model.config.use_cache = False # Disable cache during training

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "right"

# --- 2. LoRA Configuration ---
peft_config = LoraConfig(
    r=16,
    lora_alpha=32,
    lora_dropout=0.05,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
    task_type="CAUSAL_LM",
)

peft_model = get_peft_model(model, peft_config)
peft_model.print_trainable_parameters()

# --- 3. Load and Preprocess the Dataset ---
dataset = load_dataset("json", data_files="ai_knowledge_qa.jsonl", split="train")

def format_prompt(example):
    # Alpaca prompt template
    prompt = f"""Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
{example['instruction']}

### Response:
{example['output']}"""
    return {"text": prompt}

dataset = dataset.map(format_prompt)

# --- 4. Set Training Arguments ---
training_arguments = TrainingArguments(
    output_dir="./results",
    num_train_epochs=3,
    per_device_train_batch_size=4,
    gradient_accumulation_steps=1,
    optim="paged_adamw_32bit",
    learning_rate=2e-4,
    weight_decay=0.001,
    fp16=False,
    bf16=True, # Strongly recommended if your GPU supports bf16
    max_grad_norm=0.3,
    max_steps=-1,
    warmup_ratio=0.03,
    group_by_length=True,
    lr_scheduler_type="constant",
    logging_steps=25,
)

# --- 5. Initialize Trainer and Start Training ---
trainer = Trainer(
    model=peft_model,
    train_dataset=dataset,
    args=training_arguments,
    # DataCollatorForLanguageModeling automatically handles padding and masking
    data_collator=lambda data: {'input_ids': tokenizer([x['text'] for x in data], return_tensors='pt', padding=True, truncation=True, max_length=512).input_ids,
                                'labels': tokenizer([x['text'] for x in data], return_tensors='pt', padding=True, truncation=True, max_length=512).input_ids},
)

trainer.train()

# --- 6. Save the Fine-Tuned Model ---
# Only save the LoRA adapter weights, which are very small
trainer.model.save_pretrained("./results/final_checkpoint")
tokenizer.save_pretrained("./results/final_checkpoint")

Step 3: Run and Inference

Run training: python finetune.py

Inference: Write an inference.py script, load the base model and the fine-tuned LoRA weights, then perform Q&A.

from peft import PeftModel
# ... Load base model and tokenizer (same as above) ...

# Load LoRA weights and merge
model = PeftModel.from_pretrained(model, "./results/final_checkpoint")
model = model.merge_and_unload() # Merge weights for fast inference

# Perform inference
instruction = "Please explain what LoRA is."
prompt = f"""Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction

{instruction}

### Response

"""

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(inputs, max_new_tokens=200)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)

print(response)

After running the inference script, you will find that the model can answer questions about LoRA in a very professional and accurate manner — this is the new knowledge and expression it learned from our custom dataset.

Chapter Summary

In this chapter, we deeply mastered the key technology for customizing large models — parameter-efficient fine-tuning.

We first clarified the value and timing of fine-tuning, establishing a clear technology selection path from prompt engineering, to RAG, to PEFT.

Then, we systematically learned the core ideas of PEFT, and focused on dissecting the currently most powerful and popular method, LoRA, and its low-resource variant QLoRA, understanding their mathematical principles and engineering implementation. We understood how to "tame" billion-parameter giant models on consumer-grade hardware through low-rank decomposition and 4-bit quantization.

We also emphasized the decisive role of data engineering in fine-tuning, learning how to build high-quality instruction-tuning datasets and transform them into prompt formats that the model can learn from.

Finally, through an end-to-end hands-on project, we put all the theoretical knowledge into practice, using QLoRA to fine-tune a 7B-scale LLM with our own hands, turning it into a Q&A expert in the AI domain. This process gave you a complete experience of the entire workflow from data preparation, model configuration, and training execution to final inference.

After completing this chapter, you are no longer just a user of LLMs. You have become an AI engineer capable of "sculpting" and "shaping" large models according to business needs. This ability is key to building your core competitiveness in the age of large models. In the next chapter, we will explore another powerful paradigm that complements fine-tuning — Retrieval-Augmented Generation (RAG) — learning how to equip LLMs with an "external brain" that can connect to real-time, private knowledge bases.

writings