Key Moments

How AI Agents Work: Connectors, MCP & Agent Safety | Stanford CS193T Lecture 5 with Cynthia Bailey

Stanford OnlineStanford Online
Education6 min read59 min video
Nov 17, 2026|7,019 views|125|2
Save to Pod
TL;DR

AI agents can now act on your behalf, making them vulnerable to prompt injection attacks that can lead to data loss, resource waste, or even system deletion.

Key Insights

1

Connectors enable AI agents to access and act within your other applications like Gmail, Calendar, and Outlook, streamlining tasks like drafting and sending emails.

2

Agentic AI workflows are built on an analyze-act-observe loop, mirroring how LLMs generate text token by token but structured for tool execution.

3

The Model Context Protocol (MCP) defines how LLMs interact with tools, translating structured text into actionable commands for connectors.

4

Defining an agent requires specifying context, tools, permissions, memory, planned actions, and stopping conditions to guide its behavior.

5

The 'Elephant and Goldfish' strategy uses an interview-based 'Elephant' agent to create detailed scope documents, which a 'Goldfish' agent then vets for completeness, ensuring agents act precisely as intended.

6

Prompt injection remains a significant risk, particularly when agents access external data sources like web pages or LinkedIn messages, potentially causing the AI to ignore its original instructions.

Connectors bridge AI and your applications

Connectors are tools that grant AI agents access to your other applications and accounts, such as Gmail, Google Calendar, Microsoft Outlook, and file systems. This integration allows AI to directly retrieve data and perform actions on your behalf, moving beyond simple content generation to task execution. For example, instead of manually copying and pasting email content into an AI tool for refinement, a connector allows the AI to access your inbox, read the email, draft a response, and even send it, significantly reducing manual steps and saving time. While not all connectors are compatible with every AI model (like Claude, ChatGPT, or Gemini), the tutorial provides a resource to identify compatibility. These connectors are crucial for agentic AI, enabling it to interact with the real world through your digital tools.

Agentic AI builds on generative AI principles

The core mechanism of agentic AI remains rooted in the Large Language Models (LLMs) we've discussed, which operate by predicting the next token in a sequence. When connectors are enabled, this process is augmented. The LLM's output is not just human-readable text but can also be a highly structured format, akin to software code, that instructs a connector module. This structured output, often in JSON format, specifies the tool to be used, its name, the desired action, and any necessary parameters (like location for a weather query). The context window, which includes system prompts, user settings, and memory, is augmented with these connector instructions. When a user prompts the AI, for instance, 'What should I wear today?', the LLM analyzes the request, identifies that checking the weather is a logical step, and generates a JSON request for a weather connector. This structured text then passes through a 'connector manager' which interprets the JSON and triggers the actual tool. The output from the tool is fed back into the context window, allowing the LLM to generate a human-readable response, like 'It's cold and foggy, so wear your Patagonia fleece'.

The model context protocol and agent workflows

The protocol for transforming generated text into actions is termed the Model Context Protocol (MCP). This refers to the entire process, from the LLM generating structured text to the connector executing the action. The servers that handle these queries are known as MCP servers. Agentic AI workflows fundamentally rely on a continuous loop: analyze-act-observe. First, the LLM analyzes the context to determine if an action is needed. For example, analyzing the current weather to advise on clothing. Second, it acts by generating a structured request for a connector (e.g., a JSON request to 'get weather'). Third, it observes the outcome. This observation occurs at two levels: ensuring the action executed correctly (e.g., the weather API returned data) and confirming it had the desired impact (e.g., the weather report indicates cold and foggy conditions). This loop can repeat, allowing agents to perform complex, multi-step tasks. The crucial element here is defining stopping conditions to prevent indefinite loops and ensuring user control through 'human-in-the-loop' mechanisms.

Structuring an agent: A detailed scope of work

Creating an agent involves defining its behavior and capabilities comprehensively. This is akin to creating a detailed scope of work. Key components include defining the agent's goal, the context it should have access to (e.g., chat history, specific memories), the tools (connectors) it can use, and the permissions granted to those tools. Additionally, one must consider memory persistence—whether the agent should remember past interactions across sessions. The planned actions outline the step-by-step process the agent should follow, and stopping conditions define when the agent should conclude its task or report an issue. For example, an agent tasked with scheduling a team dinner would need access to calendars and Slack/email connectors, permissions to view team members' availability, memory of past scheduling preferences, and a clear sequence of actions like checking availability, drafting invites, and confirming RSVPs. The stopping condition would be successfully scheduling the dinner or reporting failure after a set number of attempts.

The 'Elephant and Goldfish' strategy for robust agents

A sophisticated method for building reliable AI agents is the 'Elephant and Goldfish' strategy, proposed by Google engineer Dave Resnick. The 'Elephant' agent, with its strong 'memory,' engages in a lengthy (30-40 minute) interview process with the user to create a highly detailed, 4-6 page 'scope of work' document. This document serves as the primary directive for the agent team. The 'Goldfish' agent, characterized by its lack of long-term memory, then reviews this document. Its role is to verify if the scope of work is so clear and complete that a team of agents could execute it fully autonomously, without needing further clarification. The Goldfish's inability to retain information outside the immediate scope forces it to rigorously test the document's clarity and completeness based solely on its contents and any provided references. If the Goldfish identifies ambiguities or questions, the document is revised by the Elephant, and the process repeats with a new Goldfish until the document is deemed perfectly clear and actionable. This iterative refinement ensures that the final instructions for the implementation agents are exceptionally precise, minimizing the risk of errors or misinterpretations.

Agent safety: Mitigating risks of unintended actions

The core concern with AI agents acting on our behalf is agent safety, especially when granting broad permissions. The lecture highlights three scenarios: reporting a dorm window leak, acting as a job-seeking assistant, and performing disk cleanup. In the dorm leak example, granting an agent access to all photos and the ability to send emails could lead to it mistakenly sending inappropriate images or repeatedly emailing maintenance with incorrect details, wasting resources. The job-seeking assistant, with access to LinkedIn and web search, is vulnerable to prompt injection through public web content or malicious LinkedIn messages, potentially causing it to disregard its original instructions. The disk cleanup agent, with permission to permanently delete files, poses a severe risk of deleting critical system files or important personal data if not carefully constrained. A key principle for mitigation is 'least privilege access,' meaning agents should only be granted the minimum permissions necessary for their specific task. For instance, a disk cleanup agent should not have access to all files but only those meeting specific criteria, and ideally, should prompt for user confirmation before permanent deletion. Prompt injection remains a significant threat, especially when agents access external, unvetted data sources, making careful design and 'human-in-the-loop' oversight critical.

Agent Design Best Practices

Practical takeaways from this episode

Do This

Clearly define the agent's goal and provide necessary context.
Specify the tools (connectors) the agent can use.
Grant only the 'least privilege access' – permissions the agent strictly needs.
Use the 'Elephant and Goldfish' model for robust planning and validation.
Incorporate 'Human-in-the-Loop' checkpoints for critical decisions and validation.
Define clear stopping conditions for agent workflows.

Avoid This

Grant broad access to sensitive information or files.
Allow agents to permanently delete files without human review.
Neglect to define stopping conditions, leading to uncontrolled loops.
Ignore the risk of prompt injection, especially when using web search tools.
Assume agents will always act perfectly; always consider worst-case scenarios.

Common Questions

Connectors are access permissions that link your AI tools to other apps or accounts. They allow the AI to directly collect data or perform actions within those tools, streamlining workflows by reducing the need for manual copy-pasting.

Topics

Mentioned in this video

More from Stanford Online

View all 182 summaries

Ask anything from this episode.

Save it, chat with it, and connect it to Claude or ChatGPT. Get cited answers from the actual content — and build your own knowledge base of every podcast and video you care about.

Get Started Free