Key Moments
⚡️ How to turn Documents into Knowledge: Graphs in Modern AI — Emil Eifrem, CEO Neo4J
Latent Space PodcastKey Moments
AI must ingest structured knowledge, not just text chunks, to provide explainable and accurate answers, with graph databases like Neo4j enabling this by explicitly mapping relationships. This makes AI debugging easier but requires significant upstream data processing.
Key Insights
Vector databases as a standalone category are over, with most databases now offering vector search features, making them feel more like search tools than distinct database types.
Combining vector search with graph traversal (GraphRAG) allows AI to start semantically and then expand through explicit relationships, leading to higher accuracy and better explainability compared to opaque vector spaces.
In 2026, 30% of AI conversations for global banks have been with Neo4j, highlighting a significant uptake in financial services, with companies like a massive mortgage lender seeing a 20% increase in conversion rates by using Neo4j for agent support.
The trend in agentic applications has shifted from leading with specialized functions and falling back to generic text-to-Cypher to starting with generic text-to-Cypher and extracting edge cases into specialized functions, driven by improved LLM capabilities.
The context graph is identified as one of four key data sources required for AI agents to achieve 'escape velocity,' alongside operational data stores, cloud data warehouses, and agentic memory, serving as the 'why' behind decisions.
A new Python wrapper called 'uvx' can create a context graph out-of-the-box for 22 different industries, building a full Neo4j instance with a front-end, modeled after 'Create React App'.
The limitations of current AI and the rise of structured context
The current AI landscape, particularly within retrieval-augmented generation (RAG), often relies on finding the 'top K' text chunks. However, Emil Eifrem, CEO of Neo4j, argues that this approach is insufficient. AI systems need more than just semantic similarity; they require structured context that includes entities, relationships, permissions, authorship, provenance, and history. This is where graph databases become crucial. Unlike opaque vector spaces where the reasoning behind a similarity score is unclear (e.g., why an apple and a tennis ball might be 0.7 similar), graph databases offer explicit, inspectable relationships. This transparency in data representation is vital for debugging, explainability, and ultimately, higher accuracy in AI-generated answers. Eifrem emphasizes that 'top-K chunks' are not enough; AI needs knowledge graphs to truly understand data.
GraphRAG: Enhancing AI with explicit relationships
GraphRAG, which incorporates a knowledge graph into the RAG process, offers several advantages. Firstly, it provides higher accuracy due to a richer representation of data. Secondly, it improves developer productivity because the graph structure is explicit and can be visually inspected, unlike the 'black box' of vector spaces. This explicitness is key for explainability; users can audit why specific information was retrieved. While query speed is often associated with graph traversal, Eifrem notes that AI engineers are hearing less about it, possibly because the latency introduced by LLMs often overshadows database query times. However, the ability to efficiently traverse a graph contributes to achieving higher accuracy by allowing the system to explore more connections quickly. This combination of semantic retrieval augmented by graph traversal makes AI outputs more robust and understandable.
The evolving landscape of vector databases
Eifrem expresses a strong opinion on the durability of vector databases as a standalone category, stating, 'vector databases are as a standalone category are over.' He believes they function more like search tools than distinct database types. While many databases now include vector search capabilities, dedicated vector databases face diminishing 'oxygen' as this functionality becomes commoditized. He notes that while some high-end vector databases might still outperform the vector search features of other databases, the gap is closing. The trend is towards integrating vector search as a feature rather than a primary product, leading many former vector database companies to rebrand or reposition themselves as search platforms. The key differentiator at scale and complexity, however, lies in how these technologies integrate and complement each other, rather than competing as isolated solutions.
Integrating graph and vector search in AI pipelines
The typical pattern for AI systems today involves a combined approach where vector search and graph traversal work in tandem. At query time, vector search, often combined with keyword search like BM25, is used to find initial relevant documents or nodes. From these starting points, the system traverses the graph to gather richer context. For example, in a customer support scenario, a query about a laptop's permissions might first be processed by vector search to identify related articles. Then, the graph traversal can reveal additional context, such as the author's reputation (e.g., 'highly ranked' by page rank or stars) or related knowledge that the vector search alone might miss. This hybrid approach, often referred to as GraphRAG, ensures that the retrieved information is not only semantically relevant but also contextually rich and potentially attributed to authoritative sources, enhancing the overall quality of the AI's response.
AI adoption in life sciences and banking
Neo4j is seeing significant adoption in AI-forward industries. In life sciences, companies like Novo Nordisk utilize Neo4j to manage over 60 million documents and billions of nodes and relationships, integrating internal research, patents, and academic papers. This scientific intelligence platform relies on named entity recognition (NER) and entity resolution to make sense of complex data, which is critical for R&D productivity in a PhD-heavy field. Similarly, the banking sector has shown a surge in AI adoption with Neo4j. Approximately 30% of AI conversations in 2026 have involved global banks. One large mortgage lender reportedly increased conversion rates by 20% by building a system on Neo4j that analyzed past successful customer interactions to guide human agents. Recently, this system has moved towards automation, removing the human in the loop for direct customer outreach.
The accelerating shift to customer-facing AI automation
A notable trend observed in the last three to six months is the rapid move towards customer-facing AI automation. While previously many AI applications were internal-facing or focused on drafting content, there's now a tipping point in trust enabling automated customer interactions. This shift is palpable across various industries, leading to significant spikes in compute and database usage charts. This acceleration is attributed, in part, to advancements in LLMs and a growing willingness to deploy AI in direct customer-facing roles. This indicates a maturing market where organizations are not only experimenting with AI but actively embedding it into their core customer engagement processes, often with clear, measurable business outcomes like improved conversion rates. This transition from 'drafting' messages to 'sending' them highlights a new era of AI deployment.
Evolution of agentic application development
The development of agentic applications has evolved significantly. Initially, the recommendation was to create specialized functions (e.g., in Cypher) for common query patterns, with generic text-to-function as a fallback. However, developers found that by logging fallback queries and analyzing failures, they could identify edge cases and build more robust specialized functions. This led to a cycle of refinement. In the last 3-6 months, the approach has flipped: developers now often start with generic text-to-function capabilities (like text-to-Cypher) and then extract the edge cases into specialized functions as needed. This shift is enabled by improvements in LLM capabilities, particularly their proficiency with domain-specific languages (DSLs) like Cypher. With sufficient training data, LLMs can now often handle many queries in a single shot, reducing the need for complex pre-defined function maps. While fine-tuning and post-processing (e.g., using regex for relationship directionality in Cypher) are still sometimes necessary, the barrier to entry for building agentic graph applications has significantly lowered.
Context graphs and the four pillars for agentic 'escape velocity'
Eifrem posits that to achieve 'escape velocity' for AI agents in production, four key data sources are essential. These are: [1] Operational Data Stores (systems of record for the present, e.g., current customer value), [2] Cloud Data Warehouses (systems of record for the past, e.g., historical revenue by region), [3] Agentic Memory (system of record for agentic state, short-term or long-term), and [4] Context Graphs. Context graphs represent the 'why' behind decisions, encoding institutional knowledge, approvals, and rationale that are often not captured in traditional data stores. For instance, a sales discount approved verbally over a call, not logged in Salesforce, forms part of the decision trail. This institutional knowledge is crucial for decision-making and agentic reasoning. While agreement exists on the importance of the first three pillars, agentic memory is seen by some as less established. The challenge for enterprises lies in bootstrapping these context graphs by instrumenting their organizations to capture decision trails digitally, which is a significant hurdle for current, often analog, processes.
Bootstrapping context graphs and the role of startups
Instrumenting an organization to create context graphs is a primary concern for large enterprises. While the vision of agents automatically recording decision-making is clear, current reality often involves manual processes and data scattered across systems like Salesforce. The 'bootstrapping' problem—how to generate enough decision trails to make the context graph valuable—is a key discussion point. For startups, however, the challenge shifts to gaining adoption of their products, which naturally generate data that can form the basis of a context graph. While enterprises like Bank of America generate substantial revenue, Neo4j also values its startup program. AI-native startups are crucial for embedding Neo4j into future architectures, even if their immediate ARR is smaller. This focus ensures Neo4j remains relevant in the evolving tech landscape and supports the next generation of innovation, recognizing that startups often build with a more 'AI-native' mindset.
The knowledge layer and enterprise data unification
Enterprises are increasingly looking to establish a 'knowledge layer' to unify disparate data sources for their agents. The problem arises when agents are given direct access to multiple endpoints (e.g., various databases, cloud storage), leading to conflicting information and unreliable answers because LLMs can generate confident but incorrect responses without a clear source of truth. To address this, enterprises are building layers that consolidate metadata about where data resides. This metadata, expressed in a graph form and married with a business ontology (defining concepts like 'customer' and their relationships), provides consistency, trust, and explainability. This 'knowledge layer' can either point to original data sources using a 'zero-copy' approach (data virtualization) or materialize partial data directly within the layer. This is a popular enterprise-wide use case, aiming to provide agents with a reliable map of the data landscape, overcoming the complexities of siloed information and driving trustworthy AI decision-making.
Tools for getting started: UVX and Cloud Co-Work
To help users get started with context graphs and graph-based AI, tools like 'uvx' and 'Cloud Co-Work' are emerging. UVX is a Python wrapper that can generate a context graph out-of-the-box for 22 industries, setting up a Neo4j instance with a front-end. Modeled after 'Create React App,' it offers an interactive experience and integrates with various agent platforms. Cloud Co-Work provides a similar utility, offering agent memory toolkits including conversational state, domain entities, and decision traces (context graphs), all visualized with a graph UI. These tools can use synthetic data or integrate with SaaS tools like Google Workspace, assisting with tasks like navigating complex dashboards and generating data. While the breadth of features in these tools can be overwhelming, their purpose is to significantly lower the barrier to entry for developing graph-powered agentic applications, demonstrating the rapid innovation in the space, with some tools like Cloud Co-Work being developed in a matter of days.
The duality of innovation: Excitement and apprehension
The current era presents a duality of immense excitement and shocking challenges. Eifrem notes the shift in the 'buy versus build' dynamic, where companies once heavily invested in building internal capabilities are now reconsidering. He humorously points out paying $200,000 annually for hated conference management software, contrasting it with the potential to build a better solution for a fraction of the cost. This highlights the power of modern development tools. However, he cautions managers against assuming AI can instantly replace all human expertise. Deploying AI-generated solutions without proper training for employees can lead to them cleaning up 'messy' code or struggling with unfamiliar systems. While software is becoming more malleable, the 'craft' of development, especially in areas like databases with rigorous testing, remains vital. The ability to build is liberating, but it requires focus, restraint, and respect for the user experience, ensuring that 'vibe-coded' solutions truly serve a purpose and don't just add complexity.
Mentioned in This Episode
●Products
●Software & Apps
●Companies
●Organizations
●Concepts
●People Referenced
Common Questions
Neo4j is primarily known as a graph database, but it has evolved into a broader platform focused on transforming data into knowledge. Unlike traditional databases that store data in tables, Neo4j represents data as nodes and relationships, allowing for more intuitive and efficient querying of complex connections.
Topics
Mentioned in this video
A graph database company that offers a platform for transforming data into knowledge. It is used by many organizations, including Transport for London.
Mentioned in the context of a customer support use case where a user is having issues with laptop permissions.
A pharmaceutical company whose talk at a conference was a surprising example of graph database adoption, particularly in life sciences.
A company in the life sciences sector that uses Neo4j and has a case study involving over 60 million documents.
Mentioned as an example where the recommendation system is LLM-based, tokenizing videos to train an LLM for predicting user watch behavior.
Mentioned as a platform that uses LLM-based recommendation systems, similar to YouTube.
Used as an example of a system of record and a place where sales decisions might be recorded, though often not comprehensively.
A major bank customer of Neo4j, used as a generic example for enterprise revenue comparison.
A cloud data warehousing company, mentioned in contrast to zero-copy architectures.
A company mentioned as an early example of a 'buy versus build' shift in technology, with an unclear outcome.
A query language for graph databases, introduced to the speaker during a coding boot camp, and a key component of Neo4j.
A ranking function used in information retrieval, often combined with vector search in RAG systems.
The company associated with Simon, whose conversation with the speaker was mentioned as enjoyable.
Amazon Simple Storage Service, mentioned in the context of a blob storage-centric view of the world and its use in database architecture.
Java Virtual Machine, mentioned in relation to handling compare-and-swap operations and enabling lock-free concurrency.
A model that the speaker finds unlikely to be related to the observed industry-wide performance shifts.
A generative pre-trained transformer model, mentioned in the context of industry-wide shifts in AI performance.
A large language model used internally at Neo4j for text-to-Cypher translation, which undergoes fine-tuning and post-processing.
A graph query language that evolved from openCypher and is positioned as a sibling language to SQL.
Mentioned as an initial release that includes a small in-memory graph database, highlighting its graph shape and toy-like nature.
A relational database system, mentioned with personification in the context of data virtualization and wrappers.
A Python wrapper that can create a context graph out-of-the-box for various industries, designed to help users get started.
A tool for creating single-page React applications, after which UVX is modeled, suggesting a similar ease of use for developers.
A command-line interface for Google Workspace, that generated excitement and was mentioned as an example of helpful tooling.
Google Cloud Platform, mentioned in the context of navigating its often complex dashboard.
More from Latent Space
View all 207 summaries
86 minNotion’s Sarah Sachs & Simon Last on Custom Agents, Evals, and the Future of Work
58 min⚡️ The best engineers don't write the most code. They delete the most code. — Stay Sassy
78 minExtreme Harness Engineering for the 1B token/day Dark Factory — Ryan Lopopolo, OpenAI Frontier
77 minMarc Andreessen introspects on Death of the Browser, Pi + OpenClaw, and Why "This Time Is Different"
Found this useful? Build your knowledge library
Get AI-powered summaries of any YouTube video, podcast, or article in seconds. Save them to your personal pods and access them anytime.
Get Started Free