Key Moments
AI Dev 26 x SF | Melissa Herrera: Your Agents Should Be Durable
DeepLearning.AIWant to know something specific about what's covered?
We've already dissected every moment. Ask and we will deliver (with timestamps).
Key Moments
AI agents crash frequently during production use, losing all progress and wasting resources. Temporal offers durable execution to automatically recover from failures and resume agents from their last checkpoint, saving time and costs.
Key Insights
Building AI agents is easy, but making them production-ready is hard due to infrastructure failures, API timeouts, and rate limits in real-world scenarios.
Non-durable agents lose all context and progress upon crashing, forcing a restart from the beginning, which wastes tokens, time, and can lead to non-deterministic outcomes for the same request.
Temporal provides durable execution, acting like checkpoints in a video game, allowing agents to resume exactly from the point of failure without losing progress or context.
Temporal orchestrates AI agents through workflows and activities, where activities handle failure-prone tasks like LLM calls and tool usage, while workflows manage the overall orchestration logic.
Temporal's platform ensures applications can survive crashes and restarts, automatically handle retries, maintain state, and manage long-running tasks, as demonstrated with a news briefing agent where killing the server did not stop the workflow.
Temporal is used by companies like OpenAI for image generation (e.g., ChatGPT's image generation) and Codex, highlighting its capability for large-scale, long-running workflows.
The 'easy to build, hard to produce' problem of AI agents
Building AI agents that utilize Large Language Models (LLMs) and tools can be straightforward in development environments. However, transitioning these agents into production reveals significant challenges. Unlike controlled demos, production agents face real-world instability such as infrastructure failures, API timeouts, and rate limits imposed by external services. When these failures occur, non-durable agents crash, leading to a complete loss of context and progress. This forces a restart from the very beginning, which is not only frustrating for the end-user but also incurs significant costs in terms of wasted computational resources (like LLM tokens) and valuable development time. The inherent non-deterministic nature of LLMs further complicates re-running failed processes, as the same input may not yield the same output path, making recovery even more difficult without a robust system.
What durable execution means and why it's crucial
Durable execution, as implemented by Temporal, guarantees that code will complete its execution, even in the face of distributed system failures. It functions like a checkpoint system found in video games; if the 'console' (your application) crashes or is unplugged, you resume from the last saved checkpoint, not from the start of the level. For AI agents, this means that if an LLM call fails, a tool times out, or the entire server goes down mid-process, the agent can automatically resume from that exact point of failure once the issue is resolved. This capability is essential for long-running tasks that can span minutes, hours, or even days, preventing the loss of progress and resources accumulated up to that point. Temporal's mascot, Ziggy the tardigrade, symbolizes this resilience, as tardigrades are known for their extreme durability and ability to survive harsh environments.
Temporal's core concepts: workflows and activities
Temporal organizes execution into two primary constructs: workflows and activities. Workflows represent the orchestration logic, essentially the 'game level' that needs to be completed. This is where developers define the happy path business logic they want their agent to perform, such as researching a topic, summarizing it, and generating a report. Activities, on the other hand, are the fundamental units of work within a workflow that interact with the outside world. These include LLM invocations, tool calls, or external API requests – essentially any action prone to failure. Activities are designed to be retried and configured to handle specific failure scenarios, such as LLM rate limits or network errors. Temporal ensures that these activities, even if they fail multiple times, will eventually succeed or be retried according to their configuration, all managed within the durable workflow.
Demonstrating durability with a news briefing agent
A live demonstration highlighted the difference between non-temporal and temporalized agents using a news briefing agent. In the non-temporal version, introducing a failure by killing the server caused the UI to hang and the agent to halt indefinitely. In contrast, the temporalized version, even with the server terminated, continued to run in the background. The Temporal UI provided visibility into the workflow's progress, showing that completed activities remained successful. When the server was brought back online, the workflow seamlessly resumed from where it left off, demonstrating true durable execution. This process of killing and restarting the server illustrated how Temporal ensures that failure doesn't mean starting over, but rather a pause and then a continuation.
Handling complex agents with human-in-the-loop and simulated failures
A more advanced demo showcased a deep research agent involving human-in-the-loop interaction and more complex failure scenarios. Initially, an activity attempting to connect to OpenAI failed due to a lack of Wi-Fi but automatically retried seven times before succeeding, turning from red to green in the UI. The agent then prompted the user for more information, which was captured and logged within the Temporal UI, demonstrating fault tolerance and state management. Another simulated failure, where an agent was asked for specific projects, resulted in the activity retrying multiple times (shown in orange) before resolving. Crucially, when the worker (the compute layer executing the code) was killed, the ongoing web search activities that couldn't execute continued to retry. Upon restarting the worker, these activities healed and completed, showing resilience against hardware or software failures. The end-user UI remained responsive throughout these intermittent issues, hiding the backend complexity and ensuring a smooth experience.
Temporal's role in orchestrating production AI systems
Temporal is not just for individual AI agents but serves as a robust orchestrator for various scalable AI use cases. This includes making services like machine learning model inference endpoints (e.g., MCP servers), data ingestion processes, and embedding data into databases more durable. The platform provides comprehensive visibility through the Temporal UI, showing every step, action, and event history, along with crash recovery and automatic state saving. Importantly, data saved is not re-executed but replayed to reconstruct the state. OpenAI itself utilizes Temporal for critical, long-running workflows such as image generation at scale for ChatGPT and for its Codex models, emphasizing its reliability for even the most demanding production AI systems. By integrating Temporal earlier in the development lifecycle, teams can abstract away the complexities of distributed systems, ensuring reliability and durability for their AI applications.
Getting started and integrating Temporal
Temporal offers SDKs in various programming languages, allowing developers to meet the platform where they build. The core philosophy is to let developers code the 'happy path' of their application logic, while Temporal handles the complexities of distributed systems failures, retries, and state management. For developers using frameworks like the Vercel AI SDK, integration is seamless, as these SDKs can automatically wrap LLM calls into Temporal activities. Furthermore, Temporal provides a 'skill' within the Vercel ecosystem (via npx install or skills.sh), enabling developers to quickly apply durable execution to their existing agents. This means building production-ready, durable AI agents is now more accessible than ever, directly leveraging the power of Temporal to ensure reliability and robustness.
Mentioned in This Episode
●Software & Apps
●Companies
●People Referenced
Making Your AI Agents Durable with Temporal
Practical takeaways from this episode
Do This
Avoid This
Common Questions
A durable agent is one that can reliably execute a task to completion, even if it encounters failures like system crashes, network issues, or rate limits. It can resume its progress from the point of failure without losing context or having to restart from the beginning.
Topics
Mentioned in this video
An SDK that integrates with Temporal to automatically wrap LLM calls into activities for durable execution.
The Software Development Kit used to demonstrate Temporal's workflow and activity concepts in code for a TypeScript example.
Mentioned as an example of an application that uses Temporal workflows for image generation.
A tmux-like terminal multiplexer used as an example of how Temporal can be integrated.
The Software Development Kit used to demonstrate Temporal's workflow and activity concepts in code.
Large Language Models are a core component of AI agents, and Temporal helps manage their execution and potential failures.
More from DeepLearningAI
View all 80 summaries
31 minAI Dev 26 x SF | Vlad Luzin: Herding Cats—The Hidden Challenges of Multi-Agent Autonomy
33 minAI Dev 26 x SF | Carter Rabasa: File Systems Are the New Primitive for AI Agents
43 minAI Dev 26 x SF | Paige Bailey: What's New and What's Next in AI
32 minAI Dev 26 x SF | Aditi Gupta: Building SRE Agents with the Redis Context Engine
Ask anything from this episode.
Save it, chat with it, and connect it to Claude or ChatGPT. Get cited answers from the actual content — and build your own knowledge base of every podcast and video you care about.
Get Started Free