Key Moments
AI Dev 26 x SF | Nyah Macklin: The AI Said So? How to Build Auditable AI Agents Using Context Graphs
DeepLearning.AIWant to know something specific about what's covered?
We've already dissected every moment. Ask and we will deliver (with timestamps).
Key Moments
AI agents fail due to fractured context, not poor models; knowledge graphs offer a solution by providing 'tribal knowledge' for auditable decisions, boosting accuracy from 54% to 91%.
Key Insights
95% of AI pilot projects fail, with a significant cause being the lack of context and common knowledge among agentic systems, according to a 2025 MIT study.
Text similarity, a common retrieval method for RAG systems, has a "blind spot the size of a building" for relationships, leading to fractured context.
Integrating a knowledge graph with RAG (Graph RAG) can increase accuracy in domain-specific QA from 37% (base model) to 54% (fine-tuned) and finally to 91% (Graph RAG), as shown in a March 2026 IEEE Communications Magazine paper by Jiang et al. on ComGPT.
Context graphs, which include 'tribal knowledge' from Slack, emails, and Zoom transcripts, provide causal chains and decision traces, enabling agents to be auditable and explainable.
Neo4j's Graph Data Science tooling uses graph embeddings like fast RP to generate embeddings for semantic and structural similarity, which can be used with vector search to identify relevant policies and fraud patterns.
Graph Academy offers free, hands-on courses on building AI agents with graphs, and participation in a hackathon can lead to guaranteed swag like t-shirts.
The high failure rate of AI agents stems from a lack of context
Many AI agents deployed in production fail because they lack sufficient context, leading to incorrect decisions with real-world consequences. A 2025 report by MIT indicated that a staggering 95% of AI pilot projects fail. This failure isn't necessarily due to flawed models themselves, but rather a deficiency in the data and the connections between facts that agents use for reasoning. Without proper context, agents are akin to planes flying without air traffic control, lacking the common knowledge to operate harmoniously. The presentation emphasizes that better models cannot fix "fractured context"; they merely process broken pieces of information. Retrieval methods like text similarity, which find documents with similar meanings, miss crucial relationships between entities that are vital for accurate decision-making.
Knowledge graphs provide essential context for agentic systems
The key to overcoming fractured context lies in structuring data not as tables or vector databases, but as knowledge graphs. These graphs explicitly represent relationships, connections, and groupings, allowing agents to trace and explain their reasoning by hopping between nodes. An architectural layer for knowledge graphs provides structure and meaning, creates a uniform memory and retrieval mechanism for agents, links disparate data sources, and bridges the gap between human and machine understanding. This knowledge layer sits atop existing data platforms like Snowflake and Databricks, feeding contextual information to GenAI applications and LLM orchestration layers. The benefit for agentic development is clear: the schema of a knowledge graph provides context that clarifies information flows when processing user queries.
Graph RAG significantly boosts AI agent accuracy
Empirical evidence supports the efficacy of integrating knowledge graphs into AI agent pipelines. A March 2026 study by Jiang et al. in IEEE Communications Magazine, analyzing a domain-specific foundation model called ComGPT for telecom, demonstrated substantial accuracy improvements. A base model achieved only 37% accuracy. Fine-tuning improved this to 54%, which is still considered abysmal for critical applications. However, when a knowledge graph was combined with retrieval augmented generation (Graph RAG), the accuracy surged to 91%. This significant leap from 37% to 91% was not achieved by RAG alone, but specifically by adding the knowledge graph layer. This finding is representative of hundreds of research papers highlighting the impact of knowledge layers as a common agentic architecture, a trend recognized by industry leaders like Satya Nadella and Swami who have emphasized the critical role of enterprise knowledge graphs for AI applications.
Context graphs enable auditable and explainable AI decisions
The conversation around AI agents is evolving from simple RAG systems to more sophisticated "context graphs." These are not just about providing agents with factual data (the "what"), but also the reasoning behind decisions (the "why"). In production, AI agents make critical decisions in finance, hiring, and healthcare, necessitating auditable and explainable processes. A context graph is essentially a knowledge graph that encapsulates all information needed for decision-making, including "tribal knowledge" from Slack, emails, and meeting transcripts. This provides a broad context beyond simple audit trails, enabling causal chains and decision traces that can be queried. This allows agents to be more trustworthy and reliable, moving away from the "black box" problem.
A financial services demo illustrates context graph application
A practical demonstration showcased how context graphs work in a financial services scenario. An agent was tasked with evaluating Jessica Norris's request for a $25,000 credit line increase. Standard RAG might have approved it, but by using a context graph, the agent identified multiple risk factors: a prior identical rejection, her profile being high-risk with two active accounts in medium and high-risk tiers, her employer being on a sanctions list (C rating), significant adverse history with failed velocity checks (14 transactions in 29 minutes), a geographic anomaly (IP inconsistent with address), and prior compliance violations. The agent correctly rejected the request, providing a clear decision trace based on policies and past precedents within the graph. This highlights how the graph provides the 'why' behind the agent's action.
Hybrid search combines graph and vector capabilities
Effectively querying context graphs often involves "hybrid search," which combines graph search capabilities with vector similarity search. Neo4j's Graph Data Science tooling allows for running graph algorithms (like centrality, PageRank) and generating graph embeddings (e.g., fast RP) that capture semantic and structural information. These embeddings can be stored as vectors, enabling vector similarity search to find semantically similar policies or decisions. Simultaneously, graph structure analysis helps understand relationships between accounts, transactions, and entities to identify fraud patterns. This dual approach leverages both the explicit connections in the graph and the learned representations for comprehensive analysis.
Resources for learning and building with context graphs
For those interested in diving deeper, Neo4j offers several resources. Graph Academy provides free, hands-on online courses for learning about graphs, graph algorithms, and building AI agents. Participants can earn unique swag by completing courses. A hackathon challenges individuals to build agents using context graphs, with guaranteed prizes for submissions. Furthermore, the "Call for Papers" for Nodes 2026, a large conference dedicated to graphs and AI, invites developers to share their work. These resources aim to equip engineers with the knowledge and tools to create more accurate, trustworthy, and auditable AI agents.
Mentioned in This Episode
●Software & Apps
●Companies
●Organizations
●Books
●Studies Cited
●People Referenced
Accuracy Improvement with Graph RAG vs. Base Models
Data extracted from this episode
| Model/Approach | Accuracy Rate |
|---|---|
| Base Model | 37% |
| Fine-tuned on Domain Data | 54% |
| Knowledge Graph + Retrieval Augmented Generation (Graph RAG) | 91% |
Common Questions
AI agents frequently fail because they lack sufficient context and common knowledge to understand the relationships between data points. Better models do not fix fractured context; instead, a robust knowledge layer is needed to provide meaning and connection.
Topics
Mentioned in this video
A domain-specific foundation model for telecom, built by Jiang and their team, used to study the impact of ABN on accuracy.
A graph algorithm used for detecting fraud patterns and topological similarity, mentioned as part of hybrid search.
A graph algorithm mentioned as being used to detect fraud patterns.
A graph algorithm mentioned as being used to detect fraud patterns.
An example of an LLM orchestration layer that can plug into a knowledge layer for AI applications.
An example of an LLM orchestration layer that can plug into a knowledge layer for AI applications.
A graph algorithm mentioned as being used to detect fraud patterns and analyze graph structure.
A free online learning platform offering technical courses on graph technology and GenAI, including hands-on agent building.
A customer in the financial services demo who requested a $25,000 credit line increase.
The speaker, a researcher specializing in graph algorithms, AI, and context engineering, advocating for ethical and responsible AI development.
CEO of Microsoft, who highlighted the critical role of enterprise knowledge graphs for AI applications.
More from DeepLearningAI
View all 80 summaries
33 minAI Dev 26 x SF | Carter Rabasa: File Systems Are the New Primitive for AI Agents
28 minAI Dev 26 x SF | Melissa Herrera: Your Agents Should Be Durable
31 minAI Dev 26 x SF | Vlad Luzin: Herding Cats—The Hidden Challenges of Multi-Agent Autonomy
52 minAI Dev 26 x SF | Eli Schilling: Hands On Agent Context & Memory Engineering with Oracle AI Database
Ask anything from this episode.
Save it, chat with it, and connect it to Claude or ChatGPT. Get cited answers from the actual content — and build your own knowledge base of every podcast and video you care about.
Get Started Free