Key Moments
AI Agent Design Patterns: ReAct, CodeAct & More | Stanford CS329Z Lecture 6 with Michael Ryan
Stanford OnlineKey Moments
AI agents are evolving beyond simple workflows with new design patterns like ReAct and CodeAct, but context management remains a critical challenge, impacting agent performance and cost.
Key Insights
Workflows offer predictability and control due to logic residing in code, whereas agents provide adaptability and flexibility with control residing in the LLM, making them less predictable.
Anthropic's five workflow design patterns include prompt chaining, routing, parallelism, coordinator workers, and an evaluator-refiner loop.
The ReAct (Reasoning and Acting) pattern combines chain-of-thought reasoning with tool use, forming the basis for many modern agents.
Reflection enables agents to learn from long-term experience by analyzing past interaction paths and reflections for improved future performance.
Context management is crucial for LLM efficiency, with strategies like context compression, observation hiding, and context isolation helping to manage lengthy interaction histories.
Pie, a simple agent environment, prioritizes adaptability by allowing users to build extensions rather than adapting workflows to Pie, but lacks built-in safety checks.
Distinguishing workflows from agents
The lecture begins by differentiating between workflows and agents. Workflows reside in the code itself, offering predictability and control, making them suitable for structured, repeatable tasks. Agents, on the other hand, have control flow residing within the LLM, offering adaptability and flexibility for exploration and decision-making, though they are less predictable and harder to evaluate. The control flow in workflows is fixed, while agent control flow can differ with each execution. Workflows excel at tasks like classifying support tickets or drafting automated responses, whereas agents are better for problem-solving and exploration.
Anthropic's workflow design patterns
Several design patterns for structuring workflows are presented, based on Anthropic's work. 'Prompt chaining' breaks down problems into sequential LLM calls. 'Routing' directs tasks to different LLMs based on input characteristics, optimizing for cost and capability. 'Parallelism' splits work into independent threads for simultaneous processing, useful for tasks like classifying user input while generating a response or for voting on multiple outputs. 'Coordinator workers' dynamically distribute sub-tasks to workers, especially when the number and nature of sub-tasks are unpredictable. Finally, the 'evaluator-refiner' pattern involves an LLM generating an output, another LLM evaluating it, and feeding back for improvement, creating an iterative refinement loop. These patterns are often combined in real-world systems.
Reasoning and acting patterns for agents
The lecture then delves into agent design patterns, starting with ReAct (Reasoning and Acting). ReAct combines chain-of-thought reasoning with tool use, allowing agents to 'think' before 'acting.' For instance, to answer a query about devices compatible with an Apple remote, an agent using ReAct would first think about needing to search for related information, then perform the search, and then use the search results to formulate an answer. This is contrasted with simple tool use where the agent might directly call tools without intermediate reasoning. Research suggests that while larger models may not always benefit from ReAct with basic prompting, fine-tuning them for this pattern significantly improves performance. The pattern is crucial for agents that need to interact with environments and tools effectively. 'Reflection' is another key pattern, enabling agents to learn from past experiences. An agent generates a trajectory of actions, an evaluator analyzes its success, and a self-reflecting LLM stores insights in long-term memory. This allows the agent to improve performance over time by avoiding past mistakes, as demonstrated in a programming task where an agent learned to correctly check parenthesis matching after initially failing.
CodeAct and planning-then-execution for action
The 'CodeAct' pattern proposes using code itself as a suitable format for actions, treating tools as functions. Instead of defining tool calls via JSON or text prompts, the agent writes code that internally calls these functions. This approach has shown higher success rates and fewer steps compared to other methods, offering greater efficiency and accuracy. The 'plan-then-execute' pattern, exemplified by ReWOO and LLM Compiler, involves generating a plan or a directed acyclic graph (DAG) of operations before execution. This allows for pre-assembly of tool calls and execution logic, potentially reducing the number of LLM calls and improving efficiency. While this pattern can be implemented dynamically, modern approaches often lean towards code generation, where the agent directly generates executable code for the entire task, offering more flexibility.
Context management strategies for LLM efficiency
Effective context management is vital for LLM performance, especially with increasing context windows. 'Prompt engineering' has evolved into 'context engineering,' focusing on curating relevant information for the LLM. Key principles include 'focus' (keeping context concise and task-relevant) and 'continuity' (maintaining system goals and constraints). Patterns for managing context include 'persistent event logging,' where all interactions are recorded and rebuilt for the LLM's input; 'memory dumping,' where information is stored in memory or files for later retrieval (like in MemGPT); and 'compression,' where the LLM summarizes lengthy contexts. 'Observation hiding,' an alternative to compression, involves selectively omitting or summarizing observations that are less critical to the current task, which can be more efficient and easier to implement programmatically. 'Context isolation' further refines this by using sub-agents with their own contexts or employing progressive disclosure, where the LLM requests information (e.g., skill files) only when needed.
The Pie agent environment and its design philosophy
Pie is presented as a simple, extensible agent environment designed to adapt to workflows rather than vice-versa. It prioritizes simplicity and extensibility, allowing users to build numerous add-ons. Key limitations include the absence of sub-agents, permission checks, planning modes, or background Bash execution; all operations must occur within a single session. Pie's built-in tools are limited to read, Bash, edit, and write, with others accessible via Bash scripting. Its agent loop follows the ReAct pattern, managing context through event logs, progressive disclosure via skill files, and standard compression. Notably, Pie operates with the same permissions as the executing process, lacking built-in safety guarantees, making it inherently automatic but potentially risky depending on the application.
Mentioned in This Episode
●Products
●Software & Apps
●Companies
●Organizations
●Books
●Studies Cited
●Concepts
●People Referenced
Common Questions
Workflows have control flow logic embedded in code, making them predictable and easier to optimize. Agents, on the other hand, delegate control flow to an LLM, making them more adaptable and flexible but also less predictable and harder to evaluate.
Topics
Mentioned in this video
One of the LLMs used in the Archon research to generate candidate solutions for mathematical problems.
An example of a software design pattern mentioned to illustrate the concept of design patterns in programming.
A research paper and framework presented as an example of the 'Plan and Execute' design pattern, focusing on generating a plan before execution, exemplified by a quality assurance task.
A command-line shell and scripting language available as a tool within the Pie environment.
A system described as a case study for integrating workflow design patterns, capable of generating detailed Wikipedia-like articles from a given topic, including research and citations.
Another research paper that applies the 'Plan and Execute' principle, focusing on generating a directed acyclic graph for execution and dynamic replanning if execution fails.
A research paper discussed in the context of memory dumping, where agents can recall past interactions and store them in a file system for retrieval, demonstrating memory management for agents.
Mentioned in the context of cost savings, with a question about whether one could use an open-source model instead of proprietary ones like ChatGPT.
Mentioned as an LLM used to evaluate and re-rank solutions generated by other models in the Archon framework.
A blog post from LangChain is referenced for a taxonomy of context management strategies, categorizing them into writing, selection, compression, and isolation contexts.
A project from Carnegie Mellon University that uses an event stream to manage agent context, recording all interactions for rebuilding the LLM's input.
Mentioned in comparison to CodeAct, showing that while text-based actions might have fewer steps in some cases, CodeAct generally leads to more efficient execution.
Specifically mentioned as the model used for re-ranking solutions within the Archon framework.
A command-line interpreter available as a tool within the Pie environment, used for executing code and scripts.
A framework for searching software architecture for inference-time techniques, which helps in automatically building workflows based on specified criteria and available LLMs.
One of the LLMs used in the Archon research to generate candidate solutions for mathematical problems.
A program mentioned in the ReAct example; the agent searches for it to determine which devices it can control besides the Apple remote.
An agent design pattern focused on learning from long-term experience by analyzing past interactions, identifying what worked and what didn't, and storing this for future use.
A foundational concept in software engineering for creating reusable solutions to common problems, also applicable to building agent systems. The lecture emphasizes adapting patterns to specific environments.
A foundational agent design pattern that combines reasoning (thinking) with acting (tool use). It's described as a 'chain of thought' for agents and is considered a basis for many modern agents.
A design pattern for agents where code itself is used as the mechanism for taking actions, treating tools as functions within a programming environment for greater accuracy and efficiency.
A fictional drink used in the ReWOO example to demonstrate the 'Plan and Execute' pattern, where the agent needs to find the cognac house that makes its main ingredient.
A specific smartphone model used in an example to illustrate the CodeAct design pattern, calculating the most cost-effective country to purchase it from.
Used as an example in the explanation of the ReAct pattern to illustrate how an agent reasons and acts to answer a question about its original intended use.
A simple, extensible agent environment with a design philosophy of adapting the environment to workflows. It has minimal built-in features, encouraging community-built extensions.
Mentioned as a potential incorrect answer an agent might give without proper reasoning, in the context of explaining the ReAct design pattern.
Mentioned in the context of the LLM Compiler paper's example question about market capitalization relative to Microsoft.
Mentioned in the context of the LLM Compiler paper's example question about market capitalization relative to Apple.
An organization whose classification of workflow design patterns is used as a basis for the lecture's discussion on building effective agents.
Mentioned as one of the countries considered for purchasing the CodeAct 1 smartphone, in the context of calculating cost-effectiveness.
Mentioned as one of the countries considered for purchasing the CodeAct 1 smartphone, in the context of calculating cost-effectiveness.
Mentioned as one of the countries considered for purchasing the CodeAct 1 smartphone, in the context of calculating cost-effectiveness.
Mentioned as one of the countries considered for purchasing the CodeAct 1 smartphone and identified as the most cost-effective option in the example.
A benchmark test used in research on self-reflection in LLMs, similar to GSM8K, to evaluate performance and the impact of self-reflection.
A benchmark test used in research on self-reflection in LLMs, showing that self-reflection could sometimes degrade performance if not properly implemented or if external feedback is more effective.
More from Stanford Online
View all 178 summaries
78 minDSPy vs. LangGraph: AI Agent Frameworks Explained | Stanford CS329Z Lecture 5 with Michael Ryan
101 minStanford CME295 Transformers & LLMs | Autumn 2026 | Lecture 6 - AI Agents
72 minUsing AI in Research | Stanford CS193T Guest Panel with Mark Algee-Hewitt, Megan Ma & Markus Covert
77 minRAG Explained: From BM25 to Agentic RAG | Stanford CS329Z Lecture 3 with Diyi Yang
Ask anything from this episode.
Save it, chat with it, and connect it to Claude or ChatGPT. Get cited answers from the actual content — and build your own knowledge base of every podcast and video you care about.
Get Started Free