Key Moments
AI Dev 26 x SF | Brandon Waselnuk: Building the Context Engine AI Agents Need
DeepLearning.AIWant to know something specific about what's covered?
We've already dissected every moment. Ask and we will deliver (with timestamps).
Key Moments
AI agents lack crucial organizational context, leading them to write incorrect code even when technically sound. A 'context engine' is vital to bridge this gap by providing agents with nuanced business logic and team decisions, not just raw data.
Key Insights
AI agents, despite technical proficiency, are like "day one" employees, lacking the deep organizational context that human engineers build over time.
AI adoption progresses through eight stages, with a 'context engine' being essential for advanced autonomy, enabling agents to ask questions of a system that understands the entire business context.
The effectiveness of AI agents is hampered by the 'satisfaction of search' bias; agents may stop at the first seemingly relevant piece of information (e.g., a 2-year-old document) rather than finding the true root cause.
A larger context window does not inherently solve the context problem; stuffing it with excessive, uncurated data often leads to agents making worse, not better, decisions.
A fully operational context engine can reduce token costs by up to 50% and human wall-clock time by 80% for complex tasks, compared to agents operating without one.
Open-source tools like a social graph builder (identifying experts and their interactions) and a repo rules agent (extracting and organizing codebase rules) can be components of a custom context engine.
The critical gap: context for AI agents
Current AI coding tools excel at generating code but falter when it comes to producing code relevant to a specific organization's unique needs and decisions. This deficiency stems from a lack of "context." Unlike human engineers who build internal instincts and knowledge through experience, mentorship, and iterative feedback, new AI agents are akin to "day one" employees: technically capable but ignorant of the company's history, decisions, and operational nuances. To enable AI agents to provide accurate and relevant code outputs, organizations must build "context engines," which act as reasoning layers that continuously supply these agents with the necessary understanding at a speed acceptable for rapid development. This is crucial for moving beyond basic AI functionalities like tab completion towards sophisticated agent autonomy, where AI can perform complex tasks overnight and deliver valuable outputs like ready-to-merge pull requests.
Four myths about context engines
Several misconceptions hinder the effective development of context for AI. Firstly, a naive Retrieval-Augmented Generation (RAG) over existing documentation is not a context engine; it lacks dynamism, real-time updates, and true intelligence. Secondly, simply connecting numerous data sources (multiple APIs, Notion, etc.) doesn't guarantee context. This approach often leads to the "satisfaction of search" bias, where agents pick the first plausible answer without exhaustive investigation, a problem observed in fields like radiology. Thirdly, the idea that a larger context window (e.g., a million tokens) will solve the problem is flawed. Overloading agents with raw data often introduces noise, leading to incorrect decisions rather than better understanding. Finally, hard-coded rules, even if they align with organizational policies, are insufficient. Agents need to understand deeper intents, roadmaps, and future directions, not just the "what" of the current codebase.
The risks of uncurated and static context
Organizations often hit roadblocks when trying to provide context to AI agents. One common stall point is "curated context," typically managed in personal tools like Obsidian or shared Git repositories. While useful for individuals, this approach fails to scale to team environments. Static documentation quickly becomes outdated and "rots," requiring constant maintenance by someone with perfect judgment for the entire organization, which is an unsustainable burden. This static layer becomes a maintenance bottleneck, detracting from actual development work. The "MCP plateau" is another issue, where connecting multiple data sources without an overarching reasoning layer leads to the satisfaction of search bias mentioned earlier. Agents can easily find outdated or incorrect information and proceed with flawed implementations, believing they have found the solution.
The essential components of a context engine
A robust context engine needs to go beyond simple data retrieval. It must provide "unified system context," allowing it to correlate disparate pieces of information, such as a Slack thread, a pull request, and a Jira ticket, enabling true reasoning across the knowledge base. Conflict resolution is paramount; when different sources offer contradictory guidance (e.g., an architecture diagram says 'do A,' a Slack thread says 'do B'), the engine must be able to determine the correct path, often by considering factors like who made the recommendation (e.g., CTO vs. a junior engineer) and the freshness of the information. This personalization also involves respecting user permissions and governance rules. Crucially, the engine must deliver context dynamically and in a "token-optimized" manner, compressing large amounts of data into the most signal-dense, precisely scoped information for the agent's current task, avoiding the "bloat" of useless data that degrades performance. This ensures that agents receive only what is needed, when it is needed, to perform their tasks efficiently and accurately within their context window limitations.
Practical applications and productivity gains
Implementing a context engine yields significant improvements. For instance, during development, a task executed with a context engine took 25 minutes and cost 10 million tokens, producing code that passed quality checks and would not break the system. In contrast, the same task without a context engine took 2 hours and 32 minutes, consumed 21 million tokens, and generated code that failed miserably and could have caused a service outage. This highlights how rich, hydrated context leads to better agent planning and execution. The quality of an agent's initial plan, informed by accurate context, cascades into better subsequent decisions. This results in code that is not only correct but also delivered significantly faster, often achieving 80% faster human wall-clock time and 50% lower token costs. Context engines can also enrich tickets, aid in triage operations during incidents, assist with incident management, and streamline code reviews, ensuring adherence to organizational intent and best practices over mere syntactical correctness.
Open-source tools for building your own engine
To facilitate the adoption of context engines, Unblocked offers several open-source and open-core tools. The "social graph builder" analyzes GitHub data to map out relationships between developers, identifying experts within specific code areas and their interaction patterns. This is invaluable for personalization and conflict resolution, as it helps weigh advice from recognized experts more heavily. Another tool, the "repo rules agent," scans repositories to extract and de-duplicate all encoded rules from documentation like READMEs or architectural decision records (ADRs). This structured data allows engines to understand and enforce team-specific policies. These components, along with Unblocked's skills and cookbook, provide a foundation for teams to build or integrate their own context engines, customize agent behaviors, and automate workflows for various operational needs, from CI runners to background incident response.
Hard lessons learned in context engine development
Building an effective context engine involves learning from practical challenges. Early on, teams may prioritize "access" over "understanding," connecting to numerous data sources without ensuring the AI can meaningfully interpret the information, leading to the satisfaction of search bias. Ignoring and hiding conflicts between data sources is another pitfall; resolving discrepancies is critical for accurate decision-making. Finally, caching answers, while seemingly efficient for latency, is a bad practice. Cached answers quickly become stale and incorrect as the codebase evolves, suffering from "rot and decay." The commitment to re-computing answers each time ensures that agents always operate with the most up-to-date and relevant context, preventing the deployment of outdated or wrong information.
The future: AI that understands like a seasoned team member
Ultimately, the goal of a context engine is to make AI-generated code indistinguishable from that written by a long-tenured, expert member of the team. By distilling knowledge of who knows what, enabling reasoning across diverse data sources, and understanding organizational intent and roadmaps, context engines empower AI agents to produce outputs that are not only technically sound but also aligned with the business's strategic goals. Every pull request reviewed or code generated should reflect the deep understanding and judgment of the organization's best engineers, moving AI from a code-generating tool to an invaluable, context-aware team member.
Mentioned in This Episode
●Software & Apps
●Companies
●Concepts
●People Referenced
Building and Using a Context Engine: Dos and Don'ts
Practical takeaways from this episode
Do This
Avoid This
Context Engine Performance Comparison
Data extracted from this episode
| Feature | With Unblocked Context Engine | Without Context Engine |
|---|---|---|
| Token Cost | 10 million | 21 million |
| Task Completion Time | 25 minutes | 2 hours 32 minutes |
| Code Quality Outcome | Passed quality bars, would not break systems | Failed quality bars, broke systems |
Common Questions
A context engine is a system that provides AI agents with all the necessary business context to accomplish their tasks. It allows agents to ask questions of a comprehensive knowledge base, understand the organization's workings, and make informed decisions.
Topics
Mentioned in this video
An issue tracking tool mentioned as an example of a system an AI agent can relate to, alongside Notion docs and pull requests, to reason across different data sources.
A note-taking application mentioned as an example of personal productivity tool for curating context, but not scalable for teams due to static nature.
A more advanced AI model of which there was a 'reckoning' in December, enabling more techniques and team-level progress in AI adoption.
A large language model mentioned as an example of an AI agent that can make incorrect decisions without proper context, requiring correction from developers.
A diagramming and charting tool. The AI attempted to generate a Mermaid diagram in the demo, but it was not rendered.
An environment used to run the repo rules agent, demonstrating its functionality in assessing codebase rules.
A company mentioned as an example of an AI-forward team leveraging context engines with their 'minions'.
A platform for code hosting and version control, mentioned as a potential integration point for code review bots.
A platform for code hosting and version control, mentioned as a source of data for building social graphs and a subject for the repo rules agent.
A company mentioned alongside Stripe and Ramp as an example of an AI-forward team using context engines.
A company mentioned alongside Stripe as an example of an AI-forward team using context engines.
The company presenting the context engine solution, offering open-source tools and paid services to build and leverage AI agents effectively.
More from DeepLearningAI
View all 94 summaries
27 minAI Dev 26 x SF | Diamond Bishop: The Next 100 Agents. Building the Agent Native Office
22 minAI Dev 26 x SF | Andrew K. Davies: Deterministic Memory: How to Build an AI That Cannot Lie
29 minAI Dev 26 x SF | Paul Everitt: The Shift to Agentic Engineering
32 minAI Dev 26 x SF | Jerry Liu: My Agent Can't Read a PDF?
Ask anything from this episode.
Save it, chat with it, and connect it to Claude or ChatGPT. Get cited answers from the actual content — and build your own knowledge base of every podcast and video you care about.
Get Started Free