Key Moments
Vibe Coding, AI Risk & AI Evaluation | Stanford CS193T Lecture 6 with Cynthia Bailey
Key Moments
Vibe coding, a method of programming AI using natural language, can create custom tools but its effectiveness hinges on rigorous post-deployment monitoring to catch unexpected behaviors.
Key Insights
Vibe coding, also referred to as agentic engineering, allows for the creation of custom AI tools by using natural language, potentially making AI 'obsolete' for repetitive tasks.
The 'elephant and goldfish' process for vibe coding involves three steps: preparation (detailed scope), generation (prompting the AI), and review (having another AI critique the output).
Post-deployment monitoring is crucial for AI tools as 'distribution drift' (changes in real-world conditions) can introduce new failure modes not present during pre-deployment testing.
The NIST AI Risk Management Framework (AI RMF) outlines a process for responsible AI use, emphasizing planning, measuring, and managing risks throughout the AI lifecycle.
AI evaluation (or 'evals') can be categorized into model evals (assessing AI performance like bias and hallucination) and product metrics (measuring user satisfaction and growth).
The 'Citizen' app and Twitter's image cropping algorithm are cited as examples of how convenience-driven data choices and insufficient pre-deployment testing can lead to biased or flawed AI outputs.
Understanding 'Vibe Coding' and Agentic Engineering
The lecture introduces 'vibe coding,' a method of programming AI using natural language, which has evolved from a novel concept to a more professional practice, sometimes referred to as 'agentic engineering.' This approach allows users to create custom AI tools for specific tasks, potentially making AI obsolete for repetitive actions by building a dedicated tool once. For instance, instead of querying a large language model (LLM) daily for congressional updates, one could use vibe coding to build a dashboard that fetches this information. This aligns with responsible AI use by employing simpler tools when LLMs are not necessary and creating tailored solutions when existing tools are insufficient. The core idea is to leverage AI to automate tasks, making the AI itself unnecessary for each repeated instance.
The Three-Step Vibe Coding Process
The process for 'vibe coding' is broken down into three main steps: preparation, generation, and review. Preparation, discussed in a previous lecture, involves creating a detailed scope using methods like the 'elephant and goldfish' process. The generation step involves taking this detailed scope and feeding it into a new AI prompt, often within environments like Codex or Cloud Code. This 'goldfish' agent, while naive to specific project details, is aware of software engineering practices and may ask clarifying questions. The goal of detailed specifications is to minimize these follow-up questions, but users are encouraged to ask their own follow-up questions to gain necessary context. Finally, the review step involves using another AI, specialized for code review, to assess the generated code. This 'goldfish' is instructed to act as a critical software engineering manager, potentially questioning the adherence to specifications and code quality. This allows for a thorough review without requiring the user to understand coding themselves. The inclusion of unit tests within the detailed scope is also recommended for ongoing quality assessment.
The Criticality of Post-Deployment Monitoring
Even with meticulous pre-deployment testing, AI tools require continuous post-deployment monitoring. This is because AI systems can exhibit new, unforeseen behaviors due to 'distribution drift,' where real-world conditions change. An example is when user behavior shifts in response to the AI tool itself, potentially leading to new attack vectors like prompt injection. A real-world case involved a C++ grading tool developed at Stanford. While it performed well on historical data, students, knowing an AI was grading, began adding 'prompt injection' comments to their code. This revealed a failure mode not present in the historical training data. Furthermore, AI systems that can update their weights while in operation (e.g., smaller, open-weight models) are inherently more prone to evolving behaviors. Experts even advise against deploying AI weapon systems that can continue learning in the field due to the uncontrollable risk of unintended actions, highlighting the extreme implications of inadequate post-deployment oversight.
Navigating AI Risks with the NIST AI Risk Management Framework
The National Institute of Standards and Technology (NIST) AI Risk Management Framework (AI RMF) provides guidance for responsible AI use. It's not about eliminating all risk, but about thoughtfully managing it. The framework has three core components: Planning, Measuring, and Managing. Planning involves a thorough self-assessment of data sources, deployment contexts, and algorithms to map out the risk landscape, similar to the 'what's the worst that can happen?' exercise. Measuring requires creativity in developing methods to quantify AI risks and performance, acknowledging the black-box nature of LLMs and the difficulty of precise risk measurement. Managing involves taking the insights from planning and measurement to actively mitigate identified risks. A central tenet of the AI RMF is fostering a culture of safety and responsibility within organizations, moving beyond mere policy documents to embedding these principles into daily practices throughout the entire product lifecycle, from conception to decommissioning.
Understanding Evaluation ('Evals') vs. Product Metrics
AI evaluation, often shortened to 'evals,' focuses on the performance of the AI model itself, such as its bias or hallucination rate. These can and should be conducted both before and after deployment. In contrast, product metrics, like user growth and satisfaction, are concerned with the business success of an AI-powered product and are only measurable once users are interacting with the system. A common method for evals is using another LLM to judge the output of the first model, akin to the 'goldfish' reviewing code. This is useful for subjective assessments like identifying hallucinations or measuring friendliness, which are hard to automate with simple keyword searches. There's also a growing ecosystem of benchmarks and leaderboards, like Hugging Face's Open LLM Leaderboard, that allow for comparative analysis of different AI models.
The Challenge of Ground Truth and Biased Data
A significant challenge in AI evaluation is establishing 'ground truth'—the actual, objective reality. Datasets used for training and evaluation often fall short, suffering from issues like survivor bias and sampling errors. Sociologist Ruha Benjamin highlights how applications using public police records to map crime risk can create a skewed reality. These maps may not reflect actual crime rates but rather policing intensity and policy decisions, leading to biased representations. This often arises from using 'convenience datasets'—existing datasets repurposed for a new task, rather than collecting data specifically for the intended purpose. The 'Citizen' app, initially called 'Vigilante,' is an example, using readily available city crime statistics that might not capture all crimes or reflect underlying systemic biases.
Case Studies: Bias in AI Systems
The lecture presents two case studies illustrating how AI systems can exhibit bias. The first involves Twitter's image-cropping algorithm. When presented with tall images, the algorithm was observed to consistently favor showing white men (like Mitch McConnell) over Black men (like Barack Obama) in previews, even when their positions were swapped. This bias, exposed through crowdsourced testing, demonstrated a failure in the algorithm's ability to identify the most relevant or interesting part of an image impartially. The second case study, from a Stanford CS class, describes a scenario where an AI predicts which new hires will perform poorly based on their first 90 days. While the AI accurately predicted the performance of retained employees, it suffered from 'survivor bias,' lacking data on how the dismissed employees would have performed had they been kept. This highlights the difficulty of assessing AI decisions when the 'counterfactual'—what would have happened otherwise—is unknown. Such biases can arise from convenience data, lack of diverse testing, and the inherent challenges in defining 'fairness' for AI.
Stanford's AI Policies and Ethical Considerations
Stanford University has established AI policies to guide responsible use, emphasizing data classification and ethical considerations. These policies dictate which AI tools can be used with different data risk levels. A key principle is the 'golden rule' of AI: avoid inputting data about others into generative AI that you wouldn't want them to input about you. This encourages empathy and thoughtful reflection on data privacy. The lecture prompts students to consider various ethical dilemmas, such as personalized pricing based on emotional states, lower wages due to credit history analysis, AI-generated recommendation letters, manipulated photos, and the use of one's creative work for AI training without consent or compensation. These thought-provoking scenarios underscore the broad societal implications of AI deployment.
Mentioned in This Episode
●Software & Apps
●Companies
●Organizations
●Books
●People Referenced
Vibe Coding and AI Risk Management Checklist
Practical takeaways from this episode
Do This
Avoid This
Common Questions
'Vibe coding' is a method that relies on using natural language, like English, as a programming language. It has evolved from experimental and sometimes error-prone practices to more professional software engineering approaches, sometimes referred to as 'agentic engineering'.
Topics
Mentioned in this video
Mentioned as a tool for vibe coding.
Mentioned as a tool for vibe coding.
An app that claims to show neighborhood crime risk by using public police records, used as an example of potential dataset bias.
Mentioned as an example of an LLM that is not necessary for simple tasks like using a calculator.
Mentioned as a programming language taught in another class the speaker teaches, used to illustrate an AI tool development scenario.
The original, controversial name for the Citizen app.
A platform mentioned for its 'Open LLM Leaderboard', which helps in comparing and choosing between different language models.
Co-founded by one of the engineers who originated the concept of 'vibe coding'.
Former employer of an engineer who was a co-founder of OpenAI and originated 'vibe coding'.
Used as a case study for AI bias in its image cropping algorithm, which favored certain individuals over others.
The National Institute of Standards and Technology, a US government agency that develops standards and guidelines, including for AI risk management.
The institution where the lecture is given, with its own AI policies and data classification standards that influence AI tool usage.
More from Stanford Online
View all 183 summaries
59 minHow AI Agents Work: Connectors, MCP & Agent Safety | Stanford CS193T Lecture 5 with Cynthia Bailey
79 minMulti-Agent Systems: When More Agents Help or Hurt | Stanford CS329Z Lecture 8 with Michael Ryan
79 minHow AI Agent Memory Works: MemGPT & Memory Types | Stanford CS329Z Lecture 7 with Michael Ryan
77 minAI Agent Design Patterns: ReAct, CodeAct & More | Stanford CS329Z Lecture 6 with Michael Ryan
Ask anything from this episode.
Save it, chat with it, and connect it to Claude or ChatGPT. Get cited answers from the actual content — and build your own knowledge base of every podcast and video you care about.
Get Started Free