Key Moments
The "Normsky" architecture for AI coding agents — with Beyang Liu + Steve Yegge of SourceGraph
Latent Space PodcastKey Moments
SourceGraph's Cody AI is a 'Normsky' architecture, merging Chomsky and Norvig approaches for superior code intelligence, RAG, and developer productivity.
Key Insights
SourceGraph's Cody AI leverages a 'Normsky' architecture, blending Chomsky's formal systems with Norvig's data-driven learning.
Effective Retrieval-Augmented Generation (RAG) and high-quality context are paramount, differentiating Cody from other AI coding assistants.
Open-source models like StarCoder can compete with proprietary ones when combined with superior context fetching and prompt engineering.
The future of AI in coding lies in providing broad codebase understanding and assisting with complex task management, not just line-by-line code generation.
AI will augment, not replace, developers, shifting their focus from tedious tasks to higher-level creative and architectural work.
Multimodal AI and new interaction patterns hold significant potential for future AI coding assistants beyond current chat and completion interfaces.
FOUNDATIONS OF CODING INTELLIGENCE: FROM GROCK TO SOURCEGRAPH
The conversation opens with introductions to Beyang Liu and Steve Yegge of SourceGraph, highlighting their extensive experience with code indexing and search. Liu's early work at Google with `grok` and `Google Code Search` (later `Kyte`) inspired the founding of SourceGraph with Quinn, aiming to solve the pain points of navigating large codebases. Yegge, with his background at Amazon and Google, shares his insights into tech culture and his appreciation for SourceGraph's engineering prowess, noting his personal use of their tools.
CODY: AN AI CODING AGENT FOCUSED ON CONTEXT QUALITY
Cody is introduced as SourceGraph's AI coding agent, which, while offering familiar features like autocompletion and codebase-aware chat, distinguishes itself through the unparalleled quality of its context. Drawing on SourceGraph's decade-long investment in code understanding for human developers, Cody provides AI with rich, relevant context. This approach prioritizes the RAG mechanism, framing it not as a workaround but as a crucial component for LLMs to accurately understand and interact with proprietary code.
THE "NORMSKY" ARCHITECTURE: MERGING FORMAL SYSTEMS AND DATA-DRIVEN LEARNING
The core of Cody's architecture is dubbed 'Normsky,' a portmanteau of 'Norvig' (representing data-driven machine learning, like LLMs) and 'Chomsky' (representing formal systems, like compilers and parsers). This hybrid approach aims to combine the pattern-recognition power of LLMs with the precise structural understanding derived from code analysis tools. This allows for more reliable code generation, better context utilization, and a more robust overall system, moving beyond the limitations of purely data-driven models.
RAG, CONTEXT, AND THE LIMITATIONS OF AGENTS
The discussion emphasizes that while Retreival-Augmented Generation (RAG) is a critical component, its effectiveness hinges on the quality and relevance of the retrieved context. SourceGraph views RAG as akin to a skilled consultant for the LLM. The team expresses skepticism towards purely agentic approaches that rely solely on LLM orchestration for multi-step processes, arguing that current LLMs lack the consistent reliability needed for fully automated, agentic execution without human oversight.
THE EVOLUTION OF DEV TOOLS AND THE FUTURE FOR ENGINEERING LEADERS
The conversation shifts to the broader impact of AI on software development. While inline code generation tools aid individual developers, the larger challenge is managing code base complexity. SourceGraph aims to equip engineering leaders and tech leads with tools for better codebase understanding, enabling them to manage architectural integrity, identify risks, and track the impact of development on business metrics—a capability enhanced by AI but rooted in their core mission of making code accessible.
DATA PROCESSING AND THE "BIG FRIENDLY GRAPH" (BFG)
A significant portion of the discussion centers on data processing and context fetching. SourceGraph emphasizes that "data mode" refers to a sophisticated pre-processing engine, not just accumulated data. They are developing a new, efficient code graph called 'BFG' (Big Friendly Graph), built using a novel, AI-like iterative experimentation approach. This graph aims to provide rich semantic context, significantly reducing errors like type mismatches in AI-generated code, and is designed for easy integration without complex build system setups.
OPEN SOURCE, MULTIMODAL AI, AND THE FUTURE OF CODING ASSISTANTS
The potential of open-source models is lauded, with StarCoder already competitive for completions when paired with Cody's context. The speakers foresee a future of multimodal AI, opening new interaction paradigms beyond current chat and command interfaces. They suggest that the current form factor of coding assistants is temporary, and new ways of interacting with AI, potentially involving real-time screen analysis or visual input, will emerge, providing more proactive and integrated assistance.
THE ENDURING ROLE OF THE DEVELOPER AND THE DATA ADVANTAGE
Ultimately, the consensus is that AI will augment rather than replace developers. It will handle more of the tedious, boilerplate, and routine tasks, allowing engineers to focus on higher-level creative problem-solving, architecture, and innovation. While agents might eventually automate more complex workflows, the current focus remains on robust, reliable tools like Cody that provide deep code understanding and context, offering a competitive advantage through sophisticated data pre-processing and graph analysis.
Mentioned in This Episode
●Software & Apps
●Companies
●Organizations
●Books
●Concepts
●People Referenced
Common Questions
Cody's core differentiator is the quality of its context, leveraging Sourcegraph's decade-long experience in building code understanding engines for human developers. It provides codebase-aware chat and inline autocompletion.
Topics
Mentioned in this video
Co-founder of Sourcegraph, previously worked in computer vision at Stanford and Palantir, and interned at Google.
Previously worked at Amazon and Google, known for blog posts, and then at Grab as Head of Engineering. Also developed the Grok system at Google. Now works at Sourcegraph.
Built a similar code search system called Hound at Etsy.
Legendary programmer, mentioned speculatively as someone who might find the next big breakthrough in AI architectures.
Created Zoekt, a code search tool heavily inspired by Google's trigram index.
Represents a school of thought in AI focusing on data-driven, machine learning models, believing deterministic approaches had failed.
Represents a school of thought in AI focusing on formal, precise systems like compilers and parsers.
A technologist whose concept of 'tera-ped-flops' (a human worth of compute) is used to contextualize the scaling of GPT models.
CEO of OpenAI, mentioned for his historical context on GPT models and his projection that AI capabilities will reach 'every human ever' levels by the end of the decade.
Company of which Max is the head.
Company where Beyang Liu worked after Stanford, and where he and Quinn started working together.
An AI company whose models (like GPT-4) Sourcegraph partners with and uses, specifically for chat and commands. The recent drama around their CEO's departure and return caused concern among Sourcegraph's customers.
Company where Beyang Liu interned and Steve Yegge worked, known for its internal code search tools like Grok and Kythe.
An inference platform used by Sourcegraph to run StarCoder, allowing them to focus on data fetching and fine-tuning rather than building their own inference stack. Their team includes ex-Meta people knowledgeable in PyTorch.
Mentioned as a potential fallback for Sourcegraph's AI services if OpenAI's stability were to be an issue.
Mentioned as an example of a company with excellent recommender engines, drawing parallels to Sourcegraph's approach to code recommendations.
Online marketplace where Kelly Norton built the Hound open-source code search project.
An AI company whose models (like Claude) Sourcegraph partners with and uses for chat and commands.
Company where Steve Yegge famously worked, described as a giant waterfall of engineers.
Southeast Asian super app where Steve Yegge was Head of Engineering; initially criticized by one host as a customer, but Steve praised its engineering team and laser focus.
Mentioned as a comparison to Grab, highlighting Grab's additional functionality as a super app in Southeast Asia.
An organization driving the open-source AI ecosystem, mentioned for possibly releasing a V2 of StarCoder. Sourcegraph would like to collaborate on benchmarks with them.
Mentioned as an example in a demo of how Cody could generate a stock ticker app.
A coding assistant mentioned as a competitor in the market.
Mentioned as an example of a company with excellent recommender engines, drawing parallels to Sourcegraph's approach to code recommendations.
Mentioned as an example in a demo of how Cody could generate a stock ticker app.
The relational database used by Sourcegraph, which found it performed as well as most graph databases for graph workloads in their experience.
A company founded 10 years ago to index all code on the internet, now focused on AI coding intelligence with Cody. Initially focused on on-prem deployments, now cloud-hybrid.
A coding assistant mentioned as a competitor in the market.
An AI language model that recently celebrated its one-year anniversary, demonstrating the rapid advancement in AI.
A programming language that the speaker spent most of their career writing, noted for being easy to use and read despite not always being the fastest.
Sourcegraph's AI coding agent designed to provide inline autocompletion, codebase-aware chat, and automate tasks like unit test generation and documentation. Differentiates itself by leveraging Sourcegraph's decade-long expertise in code understanding and context fetching.
A coding assistant that 'died', mentioned in the context of other coding assistants.
An external tool for building LLM applications, which Sourcegraph decided against using due to needing full control over their stack for rapid iteration.
The AI model that GitHub Copilot is based on.
A query language, where the speaker notes that few engineers still write queries without AI assistance.
An open-source model used by Cody for inline completions, achieving comparable acceptance rates to Copilot by leveraging Sourcegraph's context fetching.
An internal Google developer tool that indexed code and provided a reference graph, later open-sourced as Kythe.
An AI model used by Sourcegraph for completions for a while, but required prompt engineering to output code without extra text.
The internal system at Google that provided the reference graph for Google Code Search, built by Steve Yegge. It's considered a predecessor to Kythe.
Google's internal protocol for backend queries, which is a different approach than LSP.
The open-source version of Google's code intelligence system, considered 'Grok V3'.
An open-source model that Sourcegraph is not currently using but continuously evaluates against other available models.
Sourcegraph's internal working name for a new skip-based code graph that is blazing fast, requires zero configuration, and doesn't integrate with build systems, addressing issues like type errors in AI completions.
One of the programming languages used in Sourcegraph's AI stack.
An open-source code search project built at Etsy, similar to Google Code Search.
A Transformer-based LLM from OpenAI, mentioned for its ability to utilize context from the top of its window and its decent chess-playing capability. Sourcegraph still uses GPT-4 for chat and commands.
An external tool for building LLM applications, which Sourcegraph decided against using due to needing full control over their stack for rapid iteration.
An early feature from Cody's first launch that allowed multiple parallel requests to modify a source file, but was difficult to make reliable for general code generation.
A code search system created by Hannes Neinhuis, inspired by the trigram index of Google's original code search.
A programming language originally intended for creating rules-based AI systems, representing the Chomsky approach.
A JavaScript library mentioned as a framework for building a hypothetical stock ticker app demo with Cody.
A coding assistant mentioned as a competitor in the market.
One of the programming languages used in Sourcegraph's AI stack.
A prominent AI coding assistant that Cody is often compared against, particularly regarding context utilization and completion acceptance rates. It uses variants of Codex and local context.
A Google coding assistant that initially focused on pulling local context, similar to GitHub Copilot's initial approach.
A new protocol developed by Sourcegraph that aims to combine the best ideas from LSP and Kythe, making it easier to write indexers and model symbolic characteristics of code.
An AI model that reportedly uses context from the bottom of its window better than other models. Sourcegraph uses Claude and GPT-4 for chat and commands.
A coding platform mentioned for its potential to bootstrap its own proprietary dataset through bounties, though it's rumored they're betting on OpenAI.
A logic programming language, mentioned in the context of deterministic Chomsky approaches to AI that didn't scale effectively.
An example of a notoriously challenging API to work with, which the host successfully scraped using Cody web.
Mentioned as an example of a programming language that might become less relevant for human understanding as AI generates more code through natural language.
One of the programming languages now being used in Sourcegraph's AI stack.
Beyang Liu's internal pet name for Sourcegraph's hybrid AI architecture, combining elements of Norvig's data-driven approach and Chomsky's formal systems; stands for Non-agentic Rapid Multisource Code Intelligence.
A common training corpus for AI models, used in combination with The Stack.
The observation that the number of transistors in an integrated circuit doubles approximately every two years, used as an analogy for the rapid scaling of AI capabilities.
A common training corpus for AI models, used in combination with The Pile.
A protocol being developed by Sourcegraph to define a common language for context providers to offer hints and context to AI developer tools like Cody.
A protocol that standardized code intelligence features across editors, praised for making code navigation easier but criticized for its range-based approach rather than symbolic modeling of code.
A post written by Steve Yegge arguing that data models, particularly RAG, are more important than UI or chat responses in AI coding assistants.
A famous talk or post by Peter Norvig that contributed to the shift towards data-driven AI approaches.
More from Latent Space
View all 191 summaries
86 minNVIDIA's AI Engineers: Brev, Dynamo and Agent Inference at Planetary Scale and "Speed of Light"
72 minCursor's Third Era: Cloud Agents — ft. Sam Whitmore, Jonas Nelle, Cursor
77 minWhy Every Agent Needs a Box — Aaron Levie, Box
42 min⚡️ Polsia: Solo Founder Tiny Team from 0 to 1m ARR in 1 month & the future of Self-Running Companies
Found this useful? Build your knowledge library
Get AI-powered summaries of any YouTube video, podcast, or article in seconds. Save them to your personal pods and access them anytime.
Try Summify free