祝融说。

The journey through the main text has ended, but your expedition as an AI engineer has only just begun. In the explorations that lie ahead, you will need a reliable toolbox, a resource library you can consult at any time, and a guide to help you clear obstacles. This appendix is designed precisely for that purpose.

It is not a continuation of the main text, but a faithful companion for your future learning and work. Here, you will find:

Appendix A: Recommended AI Development Tools and Resources

This section is your "arsenal." I have carefully curated and organized top-tier tools and resources spanning the entire AI development lifecycle -- from data processing, model training, and experiment management, to performance optimization and deployment services. They represent the best of the community and the de facto industry standards. Mastering them will multiply your development efficiency.

Appendix B: Local GPU Development Environment Setup Guide

This section is your "logistics base manual." A stable and efficient local development environment is the foundation for all AI experimentation. This guide will walk you step by step through the entire process -- from installing NVIDIA drivers and the CUDA toolkit, to configuring Conda, PyTorch, and Jupyter -- helping you avoid the most common pitfalls that beginners encounter, and laying a solid logistical foundation for your AI expedition.

Appendix C: Core Glossary

This section is your "portable dictionary." The AI field is filled with abbreviations and specialized terminology. This glossary brings together the most core and frequently used terms in this book and the current AI landscape, providing a concise and clear explanation for each. When you encounter an unfamiliar concept, you can consult it at any time -- it will be your tool for clearing cognitive obstacles.

Please bookmark this content and revisit it often. It will be like an experienced veteran, providing you with the most timely and reliable support when you need it. Now, let us open this toolbox together and make thorough preparations for the journey ahead.

Appendix A: Recommended AI Development Tools and Resources

This appendix aims to provide a curated, high-quality list of AI development tools and resources, covering the full lifecycle from data to deployment.

A.1 Core Frameworks and Libraries

PyTorch: https://pytorch.org/

• Description: The de facto standard framework for deep learning research and application today. Renowned for its flexibility, ease of use, and strong community support. The core teaching framework of this book.
• Ecosystem: torchvision (computer vision), torchaudio (audio processing), torchtext (natural language processing, gradually being replaced by Hugging Face).

TensorFlow: https://www.tensorflow.org/

• Description: Another mainstream deep learning framework developed by Google. Known for its powerful production deployment capabilities (TensorFlow Serving) and cross-platform support (TensorFlow Lite). Although its popularity in research has declined somewhat, it still has extensive applications in industry.

JAX: https://github.com/google/jax

• Description: A high-performance numerical computing library from Google. It combines the autograd (automatic differentiation) and XLA (Accelerated Linear Algebra) compiler, and is increasingly popular in the research community (especially within Google) for its extreme performance and functional programming paradigm.

Scikit-learn: https://scikit-learn.org/

• Description: An essential library for traditional machine learning in Python. It provides concise implementations of nearly all classic machine learning algorithms (classification, regression, clustering, dimensionality reduction), along with tools for data preprocessing, model selection, and evaluation.

A.2 Large Language Model (LLM) Ecosystem

Hugging Face Transformers: https://huggingface.co/docs/transformers/

• Description: The "Swiss Army knife" for accessing and using pretrained models. Provides a unified interface for tens of thousands of Transformer-based models (BERT, GPT, Llama, etc.). An absolute core component of modern NLP and LLM development.

Hugging Face PEFT: https://github.com/huggingface/peft

• Description: A library for Parameter-Efficient Fine-Tuning. Easily implements LoRA, QLoRA, Prefix-Tuning, and other techniques, greatly lowering the resource barrier for LLM fine-tuning.

LangChain: https://www.langchain.com/

• Description: A powerful LLM application development framework. It abstracts the various stages of interacting with LLMs (data connection, model invocation, prompt management, memory, agent) into standard components, allowing you to quickly build complex LLM applications like assembling building blocks, such as RAG and Agent.

LlamaIndex: https://www.llamaindex.ai/

• Description: A data-centric LLM application development framework. Initially focused on simplifying RAG construction, it has since expanded to areas like Agent. Its abstraction level is higher, typically enabling functionality with less code.

A.3 High-Performance Inference and Deployment

vLLM: https://github.com/vllm-project/vllm

• Description: An extremely fast and easy-to-use LLM inference and serving engine. Through PagedAttention and continuous batching, it can multiply LLM inference throughput several times over. One of the preferred frameworks for deploying LLMs in production environments.

TensorRT-LLM: https://github.com/NVIDIA/TensorRT-LLM

• Description: NVIDIA's official LLM inference optimization library. Achieves extreme inference performance on NVIDIA GPUs through deep kernel fusion and hardware-specific optimizations. Top-tier performance, but with a higher barrier to entry compared to vLLM.

llama.cpp: https://github.com/ggerganov/llama.cpp

• Description: An LLM inference project implemented in pure C/C++, designed for efficient CPU execution. Supports the GGUF format, making it possible to run LLMs on personal computers, MacBooks, and even mobile devices.

Ollama: https://ollama.ai/

• Description: A minimalist local LLM runtime tool. With a single command (e.g., ollama run llama3), you can download and run mainstream open-source LLMs locally and provide an API service. Ideal for local development and rapid experimentation.

A.4 Experiment Management and Observability

Weights & Biases (W&B): https://wandb.ai/

• Description: A powerful machine learning experiment tracking and visualization platform. Automatically records all metrics during training (loss, accuracy), hyperparameters, gradients, and model weights, generating beautiful visual charts. A powerful tool for improving experiment efficiency and standardization.

MLflow: https://mlflow.org/

• Description: An open-source machine learning lifecycle management platform. Includes experiment tracking (MLflow Tracking), model packaging (MLflow Models), model registry (Model Registry), and other components.

Prometheus: https://prometheus.io/

• Description: The standard in cloud-native monitoring. Used for collecting and storing time-series data (such as GPU utilization, service QPS, latency, etc.).

Grafana: https://grafana.com/

• Description: An open-source metrics analysis and visualization platform. Typically used alongside Prometheus to display monitoring data through rich dashboards.

A.5 Data Processing and Vector Databases

Pandas: https://pandas.pydata.org/

• Description: The de facto standard for data analysis in Python. Provides the powerful DataFrame data structure for efficiently processing and analyzing structured data.

NumPy: https://numpy.org/

• Description: The foundation of scientific computing in Python. Provides multidimensional array objects and a vast library of mathematical functions, serving as the underlying dependency for nearly all AI frameworks.

Hugging Face Datasets: https://huggingface.co/docs/datasets/

• Description: Provides convenient access and efficient processing of thousands of commonly used datasets. Supports memory mapping, enabling the handling of datasets far exceeding memory capacity.

FAISS: https://github.com/facebookresearch/faiss

• Description: A high-performance vector similarity search library developed by Facebook AI. Powerful and extremely fast, it is the core engine for building vector retrieval systems.

ChromaDB: https://www.trychroma.com/

• Description: An open-source vector database designed for AI applications. Its API is simple and easy to learn, making it ideal for RAG application prototyping and small-to-medium-scale deployments.

Neo4j: https://neo4j.com/

• Description: The most popular graph database. Used for storing and querying highly interconnected data such as knowledge graphs. Its query language, Cypher, is very intuitive.

A.6 Learning and Community Resources

arXiv: https://arxiv.org/

• Description: The first stop for the latest AI research papers. Especially the cs.CL (Computation and Language), cs.LG (Machine Learning), and cs.CV (Computer Vision) categories.

Papers with Code: https://paperswithcode.com/

• Description: A website that connects academic papers with their open-source code implementations. Makes it easy to find state-of-the-art (SOTA) models and implementations for various tasks.

Hugging Face Hub: https://huggingface.co/

• Description: The world's largest community for sharing AI models, datasets, and applications. The "GitHub" for AI developers.

Kaggle: https://www.kaggle.com/

• Description: A data science and machine learning competition platform. Provides a wealth of real-world datasets and high-quality notebooks, making it an excellent place to learn and practice data science skills.

Distill.pub: https://distill.pub/

• Description: An academic journal dedicated to explaining machine learning research in a clear, intuitive, and interactive way. Although no longer updated, its archived articles are all classics.

Appendix B: Local GPU Development Environment Setup Guide

This guide uses Ubuntu 22.04 and NVIDIA GPUs as an example to walk through building a stable, isolated local deep learning development environment from scratch.

B.1 Step 1: Install the NVIDIA Driver

This is the most critical and error-prone step. Using Ubuntu's official repository for installation is recommended to ensure stability and compatibility.

1. Check your GPU model:

   lspci | grep -i nvidia
   

   Note your GPU model, e.g., NVIDIA Corporation GA102 [GeForce RTX 3090].

2. Find the recommended driver version:

   ubuntu-drivers devices
   

   The system will list available drivers and mark the recommended version.

3. Automatically install the recommended driver:

   sudo ubuntu-drivers autoinstall
   

   This command will automatically install the most suitable driver version for your hardware.

4. Reboot the system:

   sudo reboot
   

5. Verify the driver installation: After rebooting, run the following command in your terminal. If you can see your GPU information listed, the driver was installed successfully.

   nvidia-smi
   

   The nvidia-smi command will become your best friend -- it displays real-time information about your GPU model, driver version, CUDA version, temperature, power consumption, memory usage, and running processes.

B.2 Step 2: Install the CUDA Toolkit

CUDA is NVIDIA's parallel computing platform and programming model. Frameworks like PyTorch rely on it to leverage the GPU.

Important: You do NOT need to manually install a CUDA version that exactly matches your driver! Modern NVIDIA drivers are backward-compatible. The CUDA version shown by nvidia-smi is the maximum version your driver supports. PyTorch comes bundled with its own CUDA runtime libraries. Therefore, you generally do not need to install the CUDA toolkit globally on your system, unless you need to compile custom CUDA extensions.

If you do need to install it (e.g., to compile vLLM or TensorRT-LLM), follow these steps:

1. Visit the NVIDIA CUDA Toolkit Archive: https://developer.nvidia.com/cuda-toolkit-archive

2. Select a version: Choose a version compatible with the framework you plan to use (e.g., CUDA 12.1).

3. Select your platform: Choose your operating system (Linux -> x86_64 -> Ubuntu -> 22.04).

4. Select installation method: deb (local) is recommended.

5. Follow the official instructions: The website will provide a series of wget and dpkg commands to execute in order.

6. Configure environment variables: After installation, add the CUDA path to your ~/.bashrc or ~/.zshrc file:

   export PATH=/usr/local/cuda-12.1/bin${PATH:+:${PATH}}
   export LD_LIBRARY_PATH=/usr/local/cuda-12.1/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
   

   Then run source ~/.bashrc to apply the changes.

7. Verify the installation:

   nvcc --version
   

   If you can see the CUDA compiler version information, the installation was successful.

B.3 Step 3: Install Conda for Environment Management

To avoid Python package conflicts between different projects, using Conda to create and manage isolated virtual environments is highly recommended.

1. Download Miniconda: Miniconda is a lightweight version of Conda.

   wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
   

2. Install Miniconda:

   bash Miniconda3-latest-Linux-x86_64.sh
   

   Follow the prompts, agree to the license, and agree to run conda init at the end of installation.

3. Restart your terminal: Close and reopen your terminal. You should see (base) appearing before the command prompt, indicating that Conda is active.

4. Create a new virtual environment:

   # Create an environment named 'llm-dev' with Python 3.10
   conda create -n llm-dev python=3.10
   

5. Activate the environment:

   conda activate llm-dev
   

   Now, all your pip install operations will only affect this isolated environment, without polluting the system or other projects.

B.4 Step 4: Install PyTorch

1. Visit the PyTorch website: https://pytorch.org/get-started/locally/

2. Select your configuration: On the website, choose your setup:

   PyTorch Build: Stable Your OS: Linux Package: Pip Language: Python Compute Platform: CUDA 12.1 (choose a CUDA version supported by your driver)

3. Copy and run the installation command: The website will generate an installation command, similar to:

   pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
   

   Run this command in your activated Conda environment.

4. Verify PyTorch installation:

   import torch
   
   # Check PyTorch version
   print(torch.__version__)
   
   # Check if CUDA is available
   print(torch.cuda.is_available()) # Should return True
   
   # Check GPU count and name
   if torch.cuda.is_available():
       print(torch.cuda.device_count())
       print(torch.cuda.get_device_name(0))
   

B.5 Step 5: Install Jupyter Lab and Other Common Libraries

# In the activated llm-dev environment
pip install jupyterlab
pip install pandas scikit-learn matplotlib
pip install transformers datasets accelerate bitsandbytes

Start Jupyter Lab:

jupyter lab

This will open a powerful, interactive development interface in your browser. At this point, a complete, isolated, and powerful local AI development environment has been set up.

Appendix C: Core Glossary

• A
  • Agent: A system with an LLM at its core, capable of autonomously using tools to complete complex tasks through a "think-act-observe" loop.
  • Attention: A neural network mechanism that allows a model to dynamically focus on the most relevant parts of the input when processing sequences. It is the core of the Transformer.
  • AWQ (Activation-aware Weight Quantization): A post-training quantization technique that improves model performance under low-bit quantization by protecting weights associated with salient activations during quantization.
• B
  • Batch Size: The number of samples processed simultaneously in one iteration of training or inference.
  • BERT (Bidirectional Encoder Representations from Transformers): A pretrained language model based on the Transformer Encoder. It learns bidirectional context representations through the Masked Language Model (MLM) task, excelling at Natural Language Understanding (NLU) tasks.
  • Bias: 1. In machine learning, the systematic difference between a model's predictions and the true values. 2. In AI ethics, stereotypes or unfair treatment of specific groups by a model due to its training data.
• C
  • Chain-of-Thought (CoT): A prompting technique that guides an LLM to write out its reasoning process step by step before giving a final answer, improving its performance on complex reasoning tasks.
  • Chunking: In RAG, the process of splitting long documents into smaller, meaningful text segments.
  • Continuous Batching: An advanced inference batching strategy that allows completed requests to be dynamically removed from a batch and new requests to be added during inference, maximizing GPU utilization.
  • Cypher: The declarative graph query language for the Neo4j graph database.
• D
  • Decoder-only Architecture: A model architecture that uses only the decoder portion of the Transformer, such as the GPT series. Excels at text generation tasks.
  • Deployment: 1. In machine learning, the process of integrating a trained model into a production environment to serve external requests. 2. In K8s, a resource object that manages Pod replicas and updates.
• E
  • Embedding: A representation that maps discrete symbols (such as words or text chunks) into a low-dimensional, dense, continuous vector space.
  • Encoder-only Architecture: A model architecture that uses only the encoder portion of the Transformer, such as BERT. Excels at text understanding tasks.
  • Epoch: In model training, one complete pass through the entire training dataset.
• F
  • Fine-tuning: The process of continuing to train a pretrained model on a smaller, task-specific dataset to adapt it to a new task.
  • Few-shot Learning: The ability to provide an LLM with a small number (1-5) of task examples in a prompt to guide it in completing similar tasks.
• G
  • GGUF (Georgi Gerganov Universal Format): A model file format designed for llama.cpp, used for running LLMs efficiently on CPU.
  • GPT (Generative Pre-trained Transformer): A series of generative pretrained language models based on the Transformer Decoder, developed by OpenAI.
  • GPTQ (Generative Pre-trained Transformer Quantization): A popular post-training quantization technique that reduces model precision loss through column-wise quantization and error compensation.
• H
  • Hallucination: The phenomenon where an LLM generates information that appears plausible but is actually false or inconsistent with facts.
• I
  • In-context Learning (ICL): The ability of an LLM to perform a new task by learning from examples provided in the prompt, without updating model weights. Few-shot learning is one form of this.
• K
  • Knowledge Graph (KG): A structured knowledge base that represents entities and their relationships in a graph structure.
  • Kubernetes (K8s): An open-source container orchestration system for automating the deployment, scaling, and management of containerized applications.
• L
  • Latency: The time taken from sending a request to receiving a complete response. In LLMs, it is often divided into TTFT (Time to First Token) and TPOT (Time Per Output Token).
  • LLM (Large Language Model): A deep learning language model with a very large number of parameters (typically billions or more), such as GPT-3, Llama, etc.
  • LoRA (Low-Rank Adaptation): A parameter-efficient fine-tuning technique that approximates weight updates by introducing and training only two low-rank matrices, significantly reducing the number of trainable parameters.
• M
  • MaaS (Model as a Service): A service model that platformizes model deployment and management, providing model inference capabilities through APIs.
  • MIG (Multi-Instance GPU): A hardware-level virtualization technology supported on NVIDIA A100 and later GPUs, allowing a single physical GPU to be partitioned into multiple fully isolated GPU instances.
  • Multi-modality: The ability of an AI system to simultaneously understand and process multiple different types of information (such as text, images, audio).
• P
  • PagedAttention: An efficient KV cache management mechanism proposed by the vLLM framework, inspired by operating system paging, which greatly reduces memory waste.
  • PEFT (Parameter-Efficient Fine-Tuning): A category of fine-tuning methods characterized by updating only a small fraction of a model's parameters during fine-tuning.
  • Prompt Engineering: The art and science of designing and optimizing prompts input to an LLM to guide it toward producing desired outputs.
• Q
  • QLoRA (Quantized LoRA): A further optimized version of LoRA that loads the frozen base model weights in 4-bit quantization, dramatically reducing the memory required for fine-tuning.
  • Quantization: The process of converting model weights from high-precision floating-point numbers to low-precision integers (such as INT8, INT4) to reduce memory footprint and accelerate computation.
• R
  • RAG (Retrieval-Augmented Generation): An architecture that retrieves relevant information from an external knowledge base before an LLM generates an answer, providing it as context to improve answer accuracy and timeliness.
  • ReAct (Reasoning and Acting): An Agent framework that enables an LLM to reason and call tools through a "think-act-observe" loop.
• S
  • Self-Attention: The core mechanism of the Transformer, allowing each element in an input sequence to compute and attend to the importance of all other elements in the sequence.
  • System Design: In software engineering, the process of defining a system's architecture, components, modules, interfaces, and data to meet specific requirements.
• T
  • Throughput: The number of requests (RPS) or total tokens (TPS) a system can handle per unit time.
  • Token: The basic unit of text processing in NLP. It can be a word, a subword, or a character.
  • Transformer: A deep learning model architecture based entirely on attention mechanisms, which has become the foundation of modern NLP and the broader AI field.
• V
  • vLLM: An open-source high-performance LLM inference and serving framework, renowned for its PagedAttention technology.
• Z
  • Zero-shot Learning: The ability of a model to perform a task without having seen any examples of that task.