In the previous chapters, we have delved deeply into how to build, fine-tune, and deploy powerful large language models (LLMs). We know that through pretraining on massive amounts of text, LLMs learn rich world knowledge and powerful language abilities. They resemble well-read, omniscient "generalists" capable of conversing on any topic.
However, we also clearly recognize the inherent flaws of LLMs:
- Knowledge is implicit and unstructured: An LLM's knowledge is stored within the "black box" of its billions of parameters. We cannot easily inspect, edit, or update it.
- Prone to hallucination: Its knowledge is statistical, not factual. When asked about information it is unsure of or does not know, it tends to "fabricate" seemingly plausible answers.
- Limited logical reasoning: Although LLMs demonstrate some reasoning ability, they still easily make mistakes when handling complex, multi-hop logical relationships.
At the same time, there exists another technical route in the AI field with a long history and a distinctly different philosophy -- the Knowledge Graph (KG). Knowledge graphs represent world knowledge in a structured, graph-based form. They consist of entities (such as "Leonardo DiCaprio," "Titanic") and the relationships connecting these entities (such as "starred in," "directed").
A Knowledge Graph is like a rigorous, precise "expert." Every piece of knowledge in it is explicit, verifiable, and explainable. Its strengths are precisely the weaknesses of LLMs:
Knowledge is explicit and structured: We can clearly see, query, and modify every fact in the graph.
Highly factual: The graph stores structured facts confirmed by humans, and queries within its scope are answered without fabricated content. Note, however, that "no hallucination" has boundaries: extraction errors during graph construction, schema design flaws, and data staleness can all harden wrong facts into the graph. What it counters is generative hallucination, not errors in the knowledge itself.
Powerful multi-hop reasoning: Graph databases natively support complex queries that follow relationship paths, such as "Find all movies directed by James Cameron and starring Leonardo DiCaprio."
When the "broadly knowledgeable" LLM meets the "rigorous" Knowledge Graph, a profound chemical reaction is taking place. The combination of these two technical paradigms is considered key to building the next generation of more powerful, more reliable, and more explainable AI systems. The powerful natural language understanding capabilities of LLMs can be used to automatically construct Knowledge Graphs from unstructured text. Conversely, the precise, structured knowledge of Knowledge Graphs can enhance LLMs, providing them with factual grounding and improving the accuracy of their reasoning.
In this chapter, we will deeply explore this exciting cross-disciplinary field. We will learn:
- Knowledge Graph Basics: We will start from scratch, understanding fundamental concepts like entities, relationships, and triples, and learn about graph databases specifically designed for storing and querying graph data.
- Neo4j and Cypher: We will get hands-on with the most popular graph database in the industry -- Neo4j -- and learn its powerful and intuitive graph query language, Cypher.
- Automatically Building Knowledge Graphs from Text: We will leverage the powerful capabilities of LLMs to design a workflow that automatically extracts entities and relationships from unstructured text and injects them into a Knowledge Graph.
- KG-RAG: We will learn a more advanced RAG paradigm than traditional vector retrieval -- Knowledge Graph-enhanced Retrieval (KG-RAG). You will understand how to convert a user's natural language question into a structured query against a Knowledge Graph, obtaining more precise and explainable answers.
- Hands-On Project: Through a complete hands-on project -- building a small movie Knowledge Graph and combining it with an LLM for natural language querying -- we will integrate all the techniques in this chapter, creating a "movie knowledge expert" that can be queried in everyday language.
Mastering the integration of LLMs and Knowledge Graphs will place you at the forefront of AI application development. You will be able to build AI systems that are not only "eloquent" but also "reasoned and grounded," truly moving toward a more trustworthy and intelligent future.
12.1 Knowledge Graph Basics: Entities, Relationships, and Graph Databases
12.1.1 What Is a Knowledge Graph?
A Knowledge Graph (KG) is essentially a semantic network that uses a graph data structure to describe concepts, entities, and their interrelationships in the real world.
A Knowledge Graph is composed of the most basic unit -- the triple. A triple takes the form (Subject, Predicate, Object).
For example, for the fact "Leonardo starred in Titanic," we can represent it as:
- Subject (Head Entity): Leonardo DiCaprio
- Predicate (Relationship): starred in
- Object (Tail Entity): Titanic
When thousands of such triples come together, they weave into a vast, net-like Knowledge Graph.
A simple movie Knowledge Graph example
In this graph:
- Nodes (Entities): Represent objects in the real world, such as
Tom Hanks(actor),Forrest Gump(movie). Nodes can have labels indicating their type (e.g.,:Person,:Movie) and properties storing their own information (e.g.,name: "Tom Hanks",born: 1956). - Edges (Relationships): Represent connections between entities, such as
ACTED_IN. Relationships can also have properties (e.g.,roles: ["Forrest"]).
12.1.2 Why Do We Need a Graph Database?
You might ask, can't this information also be stored in a traditional relational database (like MySQL)? For example, creating an actors table, a movies table, and an acting_relations junction table.
For simple, fixed queries, relational databases are workable. But when we need to explore complex, multi-hop, and unknown-depth relationships between entities, the shortcomings of relational databases become apparent.
Consider a query: "Find actors who have worked with Tom Hanks, and then find which directors those actors have worked with."
In a relational database, this requires multiple, expensive JOIN operations. As the query depth increases, the number of JOINs grows exponentially, and query performance degrades sharply.
In a graph database, this query is very natural. It is like starting from the Tom Hanks node, following ACTED_IN relationships to find the movies he starred in, then from those movie nodes following the reverse ACTED_IN relationships to find other actors, then from those actors further... This process is called graph traversal. Graph databases are deeply optimized for this type of traversal, and their performance far exceeds that of relational databases.
The core advantage of graph databases: Index-free Adjacency. Each node directly holds pointers to its neighboring nodes. When traversal is needed, the database can follow these pointers directly, without needing to look up matching rows through an index like a relational database. This allows graph databases to handle deep association queries without significant performance degradation as the total data volume increases.
12.1.3 Types of Knowledge Graphs
General Knowledge Graph: Aims to cover the widest possible range of general domain knowledge. Famous examples include Google Knowledge Graph, Wikidata, DBpedia, and Freebase. They are massive in scale and broad in knowledge, but may lack depth or real-time updates.
Domain-specific Knowledge Graph: Focuses on a specific domain, such as finance, healthcare, e-commerce, or law. These are typically built by enterprises themselves and contain a large amount of private, specialized knowledge, making them important corporate knowledge assets. Our focus in this chapter is on how to build and apply domain-specific Knowledge Graphs.
12.2 Introduction to Neo4j and the Cypher Query Language
Neo4j is currently the most popular and mature graph database. It is a native graph database, designed and optimized entirely around the graph structure.
12.2.1 Installing and Starting Neo4j
The easiest way is to use Docker:
docker run \
--name neo4j-llm \
-p 7474:7474 -p 7687:7687 \
-d \
-e NEO4J_AUTH=neo4j/password \
neo4j:latest
Port 7474 is the HTTP port for the Neo4j Browser, a web interface for interactive querying and visualization.
Port 7687 is the Bolt protocol port, which applications use to connect to Neo4j via a driver.
After starting, access http://localhost:7474 in your browser, log in with the username neo4j and password password, and you will enter the Neo4j Browser.
12.2.2 Cypher: A Query Language Born for Graphs
Cypher is Neo4j's declarative graph query language. Its design philosophy is to "draw graphs using ASCII art," making its syntax very intuitive.
Core Syntactic Elements:
- Nodes: Represented by parentheses
().(n): An anonymous node of any type.(p:Person): A node with the labelPerson, referenced by the variablep.(m:Movie {title: 'Forrest Gump'}): A node with the labelMovieand atitleproperty of 'Forrest Gump'.
- Relationships: Represented by square brackets
[]and arrows-->or<--.-[r]-: An anonymous relationship in any direction.-[r:ACTED_IN]->: A relationship of typeACTED_IN, going from left to right, referenced by the variabler.-[r:DIRECTED {year: 1994}]->: A relationship with a property.
- Pattern: Combines nodes and relationships to describe the graph structure you want to find.
(p:Person)-[:ACTED_IN]->(m:Movie): Describes the pattern of a person acting in a movie.
Common Cypher Clauses:
CREATE: Creates nodes and relationships.CREATE (p:Person {name: 'Tom Hanks', born: 1956}) CREATE (m:Movie {title: 'Forrest Gump', released: 1994}) CREATE (p)-[:ACTED_IN {roles: ['Forrest']}]->(m)MATCH: Matches patterns in the graph. This is the most commonly used query clause.// Find all movies that Tom Hanks has acted in MATCH (p:Person {name: 'Tom Hanks'})-[:ACTED_IN]->(m:Movie) RETURN m.titleRETURN: Specifies the results to be returned by the query.WHERE: Adds filtering conditions.// Find movies starring Tom Hanks that were released after 1990 MATCH (p:Person {name: 'Tom Hanks'})-[:ACTED_IN]->(m:Movie) WHERE m.released > 1990 RETURN m.title, m.releasedMERGE: An intelligent version ofCREATE. If the pattern does not exist, it creates it; if it already exists, it matches it. This is commonly used to avoid creating duplicate nodes.MERGE (p:Person {name: 'Robert Zemeckis'}) MERGE (m:Movie {title: 'Forrest Gump'}) MERGE (p)-[:DIRECTED]->(m)DELETE: Deletes nodes and relationships.SET: Modifies the properties of a node or relationship.
Multi-hop query example:
// Find actors who have worked with Tom Hanks (excluding himself)
MATCH (tom:Person {name: 'Tom Hanks'})-[:ACTED_IN]->(m:Movie)<-[:ACTED_IN]-(coactor:Person)
WHERE tom <> coactor
RETURN DISTINCT coactor.name
The intuitive explanation of this query is: find Tom Hanks, follow ACTED_IN relationships to the movies he acted in, then from those movies follow the reverse ACTED_IN relationships to find other actors. Cypher's expressiveness and intuitiveness make it a powerful tool for working with graph data.
12.3 Automatically Building Knowledge Graphs from Text
Manually building a Knowledge Graph is time-consuming and labor-intensive. By leveraging the LLM's powerful natural language understanding and structured information extraction capabilities, we can achieve automatic Knowledge Graph construction (KG Auto-construction) from unstructured text.
Workflow:
- Define the Graph Schema: First, clarify which types of entities and relationships you want the graph to contain. For example, in the movie domain, entity types could be
:Movie,:Person,:Genre; relationship types could be:ACTED_IN,:DIRECTED,:BELONGS_TO_GENRE. - Design an Extraction Prompt: Design a powerful prompt that guides the LLM to extract triples conforming to our defined Schema from the given text.
- Text Processing and Information Extraction: Split the source documents (such as Wikipedia pages, news articles) into chunks, then send each text chunk along with the extraction prompt to the LLM.
- Structured Output Parsing: Require the LLM to return the extraction results in a structured format like JSON for easy programmatic parsing.
- Inject into the Graph Database: Use
MERGEstatements to write the parsed triples into Neo4j, building or updating the Knowledge Graph.
Example: Using an LLM to Extract Movie Information from Text
Input Text:
"Forrest Gump is a 1994 American comedy-drama film directed by Robert Zemeckis and written by Eric Roth. It is based on the 1986 novel of the same name by Winston Groom. The film stars Tom Hanks, Robin Wright, Gary Sinise, Mykelti Williamson and Sally Field."
Extraction Prompt:
You are an expert in knowledge graph construction. From the text provided, extract entities and relationships according to the following schema.
Return the result in a JSON format with two keys: "entities" and "relationships".
Schema:
- Entities:
- Person: {name: string}
- Movie: {title: string, released: integer}
- Genre: {name: string}
- Relationships:
- (Person)-[:ACTED_IN]->(Movie)
- (Person)-[:DIRECTED]->(Movie)
- (Movie)-[:BELONGS_TO_GENRE]->(Genre)
Text:
"""
Forrest Gump is a 1994 American comedy-drama film directed by Robert Zemeckis and written by Eric Roth. It is based on the 1986 novel of the same name by Winston Groom. The film stars Tom Hanks, Robin Wright, Gary Sinise, Mykelti Williamson and Sally Field.
"""
Expected JSON Output from LLM:
{
"entities": [
{"label": "Movie", "properties": {"title": "Forrest Gump", "released": 1994}},
{"label": "Person", "properties": {"name": "Robert Zemeckis"}},
{"label": "Person", "properties": {"name": "Tom Hanks"}},
{"label": "Person", "properties": {"name": "Robin Wright"}},
{"label": "Genre", "properties": {"name": "Comedy"}},
{"label": "Genre", "properties": {"name": "Drama"}}
],
"relationships": [
{"source": {"label": "Person", "name": "Robert Zemeckis"}, "type": "DIRECTED", "target": {"label": "Movie", "title": "Forrest Gump"}},
{"source": {"label": "Person", "name": "Tom Hanks"}, "type": "ACTED_IN", "target": {"label": "Movie", "title": "Forrest Gump"}},
{"source": {"label": "Person", "name": "Robin Wright"}, "type": "ACTED_IN", "target": {"label": "Movie", "title": "Forrest Gump"}},
{"source": {"label": "Movie", "title": "Forrest Gump"}, "type": "BELONGS_TO_GENRE", "target": {"label": "Genre", "name": "Comedy"}},
{"source": {"label": "Movie", "title": "Forrest Gump"}, "type": "BELONGS_TO_GENRE", "target": {"label": "Genre", "name": "Drama"}}
]
}
Python Code Snippet for Injecting into Neo4j:
from neo4j import GraphDatabase
driver = GraphDatabase.driver("bolt://localhost:7687", auth=("neo4j", "password"))
def add_to_graph(tx, data):
for entity in data['entities']:
tx.run(f"MERGE (n:{entity['label']} {{name: $name}})", name=entity['properties'].get('name') or entity['properties'].get('title'))
for rel in data['relationships']:
source_name = rel['source']['name']
target_name = rel['target']['title'] if rel['target']['label'] == 'Movie' else rel['target']['name']
tx.run(f"""
MATCH (a:{rel['source']['label']} {{name: $source_name}})
MATCH (b:{rel['target']['label']} {{name: $target_name}})
MERGE (a)-[:{rel['type']}]->(b)
""", source_name=source_name, target_name=target_name)
with driver.session() as session:
session.write_transaction(add_to_graph, llm_output_json)
driver.close()
By repeating this "extraction-injection" process, we can transform a large amount of unstructured documents into a well-structured, knowledge-rich domain-specific Knowledge Graph.
12.4 KG-RAG: Enhancing Retrieval Accuracy with Knowledge Graphs
In Chapter 9, we learned about vector-based RAG (Vector-RAG). Its advantages are simplicity of implementation and the ability to handle any text. However, it also has problems:
- Retrieval lacks precision: Similarity-based retrieval sometimes recalls text chunks that are not completely relevant or contain noise.
- Lack of interpretability: We do not know why these text chunks are considered "similar."
- Difficulty answering aggregation or comparison questions: For example, "Which movies have A and B co-starred in?" This type of question is hard to answer by retrieving independent text chunks.
Knowledge Graph-enhanced Retrieval (KG-RAG) provides a more precise and explainable solution. Its core idea is: convert the user's natural language question into a structured query against the Knowledge Graph (such as Cypher), directly obtain precise facts from the graph, and then provide these facts as context to the LLM to generate the final natural language answer.
KG-RAG Workflow:
- Question -> Cypher Conversion: This is the most critical step. We leverage the LLM's powerful code generation capability to convert the user's natural language question (e.g., "Which movies did Tom Hanks appear in?") into a Cypher query statement (
MATCH (p:Person {name: 'Tom Hanks'})-[:ACTED_IN]->(m:Movie) RETURN m.title). To enable the LLM to generate correct Cypher, we need to provide the graph's Schema information (node labels, properties, relationship types) in the prompt. - Execute the Cypher Query: Execute the generated Cypher statement against the Neo4j database.
- Obtain the Structured Result: Get a precise, table-formatted result (e.g., a list of movie titles).
- Result -> Natural Language: Send this structured query result, along with the original question, to the LLM again to have it "polish" these facts into a coherent, human-readable answer.
Example:
User Question: "Who directed the movie Forrest Gump?"
Step 1: Text-to-Cypher
Prompt:
You are an expert Neo4j developer. Given a question and the graph schema, generate a Cypher query to answer the question.
Schema:
Node labels: Person, Movie
Relationship types: ACTED_IN, DIRECTED
Question: Who directed the movie Forrest Gump?
LLM-generated Cypher:
MATCH (p:Person)-[:DIRECTED]->(m:Movie {title: 'Forrest Gump'})
RETURN p.name
Step 2: Execute the Query
Execute the Cypher in Neo4j, result: [{"p.name": "Robert Zemeckis"}]
Step 3: Generate the Final Answer
Prompt:
You are a helpful assistant. Based on the user's question and the retrieved data, provide a natural language answer.
Question: Who directed the movie Forrest Gump?
Retrieved Data: Robert Zemeckis
Answer:
LLM-generated answer: "The movie Forrest Gump was directed by Robert Zemeckis."
Advantages of KG-RAG:
- Precision: Directly retrieves facts from the graph, avoiding the uncertainty of vector retrieval.
- Interpretability: The generated Cypher query itself serves as the best explanation of the answer's source.
- Powerful Reasoning: Can answer complex questions requiring multi-hop reasoning, aggregation, and filtering.
Hybrid Strategy: In practice, we often combine Vector-RAG and KG-RAG. For factual and entity-oriented questions, prioritize KG-RAG. For more open-ended, conceptual questions, fall back to Vector-RAG.
12.5 Hands-On Project: Building a Small Movie Knowledge Graph with LLM Natural Language Querying
Project Objective: We will use a small set of Wikipedia movie synopsis texts to automatically build a Neo4j Knowledge Graph containing movies, actors, and directors, and implement a Q&A system that can convert user natural language questions into Cypher queries and return answers.
Tech Stack: openai (or other LLM library), neo4j, langchain (for simplification)
Step 1: Environment Setup
- Start the Neo4j Docker container (as shown in Section 12.2).
- Install libraries:
pip install langchain langchain-openai neo4j - Prepare some movie synopsis text files, e.g.,
forrest_gump.txt,the_matrix.txt.
Step 2: Build the Knowledge Graph from Text (Simplified with LangChain)
LangChain provides convenient tools to simplify this process.
# build_kg.py
import os
from langchain_openai import ChatOpenAI
from langchain.graphs import Neo4jGraph
from langchain.chains import GraphCypherQAChain
from langchain.prompts import PromptTemplate
from langchain.chains.graph_qa.cypher import GraphCypherQAChain
# --- 1. Connect to Neo4j ---
os.environ["OPENAI_API_KEY"] = "..."
graph = Neo4jGraph(
url="bolt://localhost:7687",
username="neo4j",
password="password"
)
# --- 2. Define a function to extract graph data from text ---
from langchain.chains import LLMChain
from langchain.prompts.prompt import PromptTemplate
from langchain_core.output_parsers import StrOutputParser
llm = ChatOpenAI(model="gpt-4", temperature=0)
extraction_prompt = PromptTemplate(
template="""From the text below, extract the following entities and relationships.
Return the result as a list of Cypher MERGE statements.
Schema:
Nodes: Person, Movie
Relationships: ACTED_IN, DIRECTED
Text:
{text}
""",
input_variables=["text"],
)
def extract_and_store_graph(text):
# Extract graph data using LLM
chain = LLMChain(llm=llm, prompt=extraction_prompt, output_parser=StrOutputParser())
cypher_statements = chain.run(text=text)
# Store data in Neo4j
for stmt in cypher_statements.split('\n'):
if stmt.strip():
try:
graph.query(stmt)
print(f"Executed: {stmt}")
except Exception as e:
print(f"Error executing {stmt}: {e}")
# --- 3. Read text and build the graph ---
with open("forrest_gump.txt", "r") as f:
forrest_gump_text = f.read()
extract_and_store_graph(forrest_gump_text)
with open("the_matrix.txt", "r") as f:
the_matrix_text = f.read()
extract_and_store_graph(the_matrix_text)
print("Knowledge graph construction complete.")
Note: The above extraction_prompt asks the LLM to directly generate Cypher statements. This is a more direct and efficient approach. You need to ensure that the LLM (e.g., GPT-4) has strong enough code generation capabilities.
Step 3: Implement the Text-to-Cypher Q&A Chain
# qa_with_kg.py
import os
from langchain_openai import ChatOpenAI
from langchain.graphs import Neo4jGraph
from langchain.chains import GraphCypherQAChain
# --- 1. Connect to Neo4j and the LLM ---
os.environ["OPENAI_API_KEY"] = "..."
graph = Neo4jGraph(
url="bolt://localhost:7687",
username="neo4j",
password="password"
)
llm = ChatOpenAI(model="gpt-4", temperature=0)
# --- 2. Create GraphCypherQAChain ---
# This LangChain chain encapsulates the complete process of Text-to-Cypher and result synthesis
chain = GraphCypherQAChain.from_llm(
graph=graph,
llm=llm,
verbose=True # Prints the generated Cypher and intermediate steps
)
# --- 3. Ask questions ---
questions = [
"Who acted in the movie Forrest Gump?",
"Which movies did Keanu Reeves act in?",
"Who directed The Matrix?",
]
for question in questions:
print(f"Question: {question}")
result = chain.invoke({"query": question})
print(f"Answer: {result['result']}\n")
When you run qa_with_kg.py, the verbose=True will let you see the magical behind-the-scenes process:
- For the question "Who acted in the movie Forrest Gump?", the LLM will generate a Cypher like
MATCH (p:Person)-[:ACTED_IN]->(m:Movie {title: 'Forrest Gump'}) RETURN p.name. GraphCypherQAChainexecutes this Cypher, getting the list of actors from Neo4j.- Finally, the LLM formats this list into a coherent answer, such as "Tom Hanks, Robin Wright, and Gary Sinise acted in the movie Forrest Gump."
This project perfectly demonstrates how LLMs and Knowledge Graphs can work together, transforming unstructured knowledge into queryable structured assets and ultimately serving users in natural language, achieving a 1+1>2 effect.
Chapter Summary
In this chapter, we explored a highly promising and valuable frontier direction in the AI field -- the fusion of large language models and Knowledge Graphs.
We started with the basics of Knowledge Graphs, understanding their power as a form of structured knowledge representation, and learned how to use the industry-leading graph database Neo4j and its query language Cypher to store and query complex relational data.
We mastered a core engineering capability: automatically building Knowledge Graphs from unstructured text using LLMs. We learned how to design prompts to guide the LLM in extracting entities and relationships and persist them into the graph database, transforming dormant text data into a living knowledge network.
Building on this, we learned a more advanced RAG paradigm -- KG-RAG. We understood how it uses Text-to-Cypher technology to convert user natural language questions into precise queries against the Knowledge Graph, overcoming the limitations of traditional vector retrieval and obtaining more accurate and explainable answers.
Finally, through a hands-on project of building a movie Knowledge Graph Q&A system, we connected all the theoretical and technical points, creating with our own hands an intelligent application where LLMs and KGs work together.
After completing this chapter, another powerful "artifact" has been added to your AI toolkit. You are no longer solely reliant on the LLM's own vague, uncontrollable knowledge. You have learned to equip it with a precise, reliable, and evolving "external factual brain." This ability to build composite systems that combine the language capabilities of LLMs with the structured reasoning capabilities of Knowledge Graphs will allow you to navigate complex AI application scenarios requiring high factual accuracy and strong logical reasoning with ease, demonstrating outstanding engineering design and innovation capabilities.