Microsoft GraphRAG, KG construction, property graphs vs RDF, Neo4j & KuzuDB, hybrid graph+vector retrieval, Cypher in agent loops, and when graphs help.
Standard vector retrieval retrieves documents. It cannot natively express relationships between entities across documents. Two canonical failures:
"Which company founded by the author of Attention Is All You Need was acquired by Google?" — this requires: (1) retrieve author names from the AIAYN paper; (2) retrieve companies founded by those authors; (3) check which was acquired by Google. Vector search over a flat index cannot chain these hops — the relevant passages never co-occur in a single chunk.
In a vector RAG system, a claim in the generated answer may come from any of the retrieved chunks. With a knowledge graph, each triple (subject, predicate, object) carries a provenance pointer to the source document and passage. Traceability is first-class: every fact has an auditable chain back to its origin.
The most effective systems use graphs and vectors together: graph traversal for multi-hop and relational queries, vector similarity for semantic search within a node's neighbourhood. The graph defines what is related; the vectors define what is semantically similar. These are complementary, not competing, capabilities.
GraphRAG (Edge et al. 2024, Microsoft Research) builds a hierarchical knowledge graph from a text corpus and uses LLM-generated community summaries to answer global questions that require synthesising information across many documents.
Building a GraphRAG index over a 1 M-token corpus requires ~1 M LLM tokens for entity extraction (at $0.15/Mtok for GPT-4o-mini: ~$0.15) plus ~500 k tokens for community summarisation. Total: ~$0.25–1 per Mtok of corpus. For a 100 M-token enterprise corpus, this is $25–100 at build time. Updates require re-processing changed documents. Costs have dropped 10× with smaller summarisation models (Phi-3.5-mini fine-tuned on summarisation tasks). Open-source: pip install graphrag (Microsoft, MIT licence).
Building a KG from text requires three non-trivial steps: extraction, resolution, and linking. The quality of the graph is bounded by the worst of these three.
Named Entity Recognition (NER) + Relation Extraction (RE). Options: (1) LLM-based with a structured output schema (GPT-4o + JSON mode — high quality, high cost); (2) GLiNER open-weights span-based NER (fast, lower quality); (3) Rebel (open RE model from BABELSCAPE, seq2seq triples). For domain-specific KGs, few-shot prompting with domain-specific entity types outperforms off-the-shelf NER by 20–40%.
The same entity may appear under different names: "OpenAI", "Open AI", "OpenAI Inc.", "OpenAI, LLC". Resolution merges these into a single canonical node. Approaches: string similarity (Jaro-Winkler, Levenshtein), embedding cosine similarity, and LLM-as-judge for ambiguous cases. FalkorDB and Neo4j GenAI plugin both include entity resolution utilities.
Map extracted entities to a reference KB (Wikidata, DBpedia, UMLS for biomedical). Provides canonical IDs, rich properties (birth date, country, description), and cross-document coreference for free. Tools: mGenre (multilingual entity linking), REL, BLINK (Meta AI, fast bi-encoder entity linker). Linking is optional for closed-domain KGs but essential for open-domain.
Two data models dominate graph storage for RAG: property graphs (node/edge with key-value properties) and RDF triplestores (subject-predicate-object). They differ in expressiveness, query language, and operational complexity.
Nodes and edges each carry arbitrary property dictionaries. Queried with Cypher (Neo4j, Memgraph) or GQL (ISO standard). More intuitive for most developers. No formal semantics — no inferencing. Best for: entity-relationship queries, social graphs, document provenance graphs.
Every fact is a (subject, predicate, object) triple stored as a URI. Supports OWL inferencing (class hierarchies, transitivity). Formally grounded in semantic web standards. Best for: scientific knowledge graphs, regulatory ontologies, Wikidata-style open KGs. Heavier to operate than property graphs.
| System | Type | Deployment | Key strength | RAG integration |
|---|---|---|---|---|
| Neo4j | Property graph | Self-host / AuraDB managed | Mature; LangChain + LlamaIndex native connectors; vector+graph hybrid (GenAI plugin) | Best ecosystem |
| Memgraph | Property graph | Self-host (Docker) | In-memory; streaming (Kafka); MAGE algorithm library | Good Cypher support |
| KuzuDB | Property graph | Embedded (local file) | Zero-ops; DuckDB-style; fast analytical queries; open-source | LangChain connector v0.2+ |
| NetworkX | Python object graph | In-process | Zero overhead; Pythonic; ideal for small graphs (<100k nodes) | Manual integration |
| FalkorDB | Property graph | Self-host (Redis-compatible) | GraphBLAS sparse-matrix backend; fast traversal | LangChain connector |
The most powerful RAG architecture combines graph traversal and vector similarity. The graph restricts the search space to a semantically coherent neighbourhood; the vector index ranks within that neighbourhood by semantic similarity.
from langchain_community.graphs import Neo4jGraph
from langchain_community.vectorstores import Neo4jVector
from langchain_openai import OpenAIEmbeddings
graph = Neo4jGraph(url="bolt://localhost:7687", username="neo4j", password="...")
# Neo4j GenAI plugin: vector index on chunk nodes
store = Neo4jVector.from_existing_graph(
embedding=OpenAIEmbeddings(model="text-embedding-3-small"),
graph=graph,
node_label="Chunk",
text_node_properties=["content"],
embedding_node_property="embedding",
retrieval_query="""
MATCH (node)-[:MENTIONS]->(e:Entity)<-[:MENTIONS]-(other:Chunk)
RETURN other.content AS text, score, {} AS metadata
ORDER BY score DESC LIMIT 10
""",
)When an agent can generate and execute Cypher queries directly against a property graph, it gains the ability to express multi-hop questions precisely — without relying on approximate vector similarity to chain hops.
from langchain.chains import GraphCypherQAChain
from langchain_openai import ChatOpenAI
chain = GraphCypherQAChain.from_llm(
llm=ChatOpenAI(model="gpt-4o", temperature=0),
graph=graph,
verbose=True,
validate_cypher=True, # parse-check before execution
return_intermediate_steps=True,
)
result = chain.invoke(
"Which organisations co-authored papers with researchers at DeepMind between 2022 and 2024?"
)
# chain emits generated Cypher, validates, executes, passes result to LLM for final answerAlways validate before executing. LLMs generate syntactically invalid Cypher or query non-existent labels/properties. Use EXPLAIN before MATCH to catch errors. Restrict the agent to a read-only database user: GRANT ROLE reader TO agent_user. Limit query depth and result size: LIMIT 50 in all generated queries. Log all generated Cypher for audit. Consider a whitelist of allowed node labels and relationship types, enforced by a query validator before execution.
Graph construction is expensive in both build cost and operational complexity. It is not always the right tool.
| Scenario | Graph benefit? | Recommendation |
|---|---|---|
| Multi-hop reasoning over entities (people, orgs, products, events) | High | Use GraphRAG or KG + vector hybrid |
| Source provenance and citation tracing | High | KG with provenance pointers to source passages |
| Global summarisation (“What are the main themes across 10 k documents?”) | High | Microsoft GraphRAG community summaries |
| Simple factual QA over a single-domain corpus | Low | Dense retrieval + cross-encoder reranker is sufficient |
| Semantic similarity search only | None | Vector DB only; graph adds complexity without benefit |
| Dynamic corpus (>10% daily changes) | Low (high update cost) | Avoid KG; use vector DB with incremental upsert |
| Small, well-structured domain (<100 k facts) | Medium | KuzuDB or NetworkX; avoid Neo4j overhead |
Microsoft's own evaluations show GraphRAG improves global query performance (themes, summaries, patterns) by 25–40% over vector RAG, but is worse than vector RAG on specific factual lookups due to the abstraction overhead. Use GraphRAG for analytical, synthesis, and exploration queries; use vector RAG for precision factual retrieval. A router that selects between them is the pragmatic production architecture.
Three frameworks dominate the GraphRAG implementation space. Each has different strengths for KG construction, hybrid retrieval, and agent orchestration.
Best ecosystem for KG integration: Neo4jGraph, GraphCypherQAChain, Neo4jVector, FalkorDBGraph. LangGraph handles multi-step agent loops with explicit state machines. Verbose but configurable. Connector library is the most comprehensive of the three.
KnowledgeGraphIndex builds a KG from documents using LLM triplet extraction. KnowledgeGraphQueryEngine routes queries to graph or vector based on type. PropertyGraphIndex (v0.10+) supports Neo4j and KuzuDB as backends. Better than LangChain for structured document hierarchies and parent–child retrieval.
Framework for optimising LLM pipelines via compilation. For GraphRAG: define a DSPy signature for entity extraction, relation extraction, and graph querying; use MIPRO or BootstrapFewShot to optimise prompts against a labelled evaluation set. DSPy can tune the entity extraction prompt to improve KG quality without manual prompt engineering. Most powerful when you have labelled QA pairs.
KuzuDB (v0.7, 2024) is an embedded property graph database requiring no separate server. pip install kuzu. LangChain's KuzuQAChain supports text-to-Cypher natively. For a pilot GraphRAG system, build the KG with KuzuDB locally, run hybrid retrieval with a pgvector instance, and use LangGraph for the agent loop. Only graduate to Neo4j AuraDB when the team graph exceeds 10 M nodes or requires HA.
Deck 07 covers production RAG: eval pipelines (RAGAS, ARES, Trulens), embedding drift, A/B testing, latency caching, multi-tenancy with namespaces and ACLs, citation and grounding, cost modelling, and the failure modes you will definitely encounter.