What is a 'system prompt' and why is it important for AI?

A system prompt is the set of instructions that tells an AI model who it is and what its job is. In applications like Gmail, these prompts are often generic and hidden, leading to impersonal outputs. Allowing users to customize or have personalized system prompts can significantly improve the AI's utility.

What does the 'AI horseless carriage' analogy mean?

This analogy likens early AI applications to the first automobiles that resembled horse-drawn carriages. They were essentially old designs with a new engine (AI) shoehorned in, failing to leverage the new technology's full potential. True innovation requires redesigning the entire system around the AI.

Can non-programmers use AI to automate tasks?

Yes, as AI technology, particularly LLMs, improves, it becomes more accessible. The ability to explain desired outcomes in natural language, much like programming an agent with simple instructions for email management, allows non-technical users to automate tasks they don't want to do.

Will everyone need to write system prompts in the future?

While direct prompt writing may become less common, the underlying concept will be integrated. Similar to how computer literacy evolved, AI will likely autogenerate personalized system prompts based on user history and feedback, making AI customization intuitive and accessible to most users.

What are 'tools' in the context of AI agents?

Tools are functionalities that AI agents can access and use to perform actions in the real world, going beyond simple Q&A. Examples include labeling emails, archiving them, or drafting replies. Chaining these tools together allows AI to automate complex workflows across different applications.

How should founders approach building AI products?

Founders should rethink their products from the ground up, focusing on AI-native designs that maximize automation of user tasks. Instead of asking 'how to insert AI,' they should ask 'how to design from scratch to offload work,' creating tools that feel like a 'rocket ship for the mind' rather than just an 'AI chatbot'.

How To Design Better AI Apps

Y Combinator

Science & Technology4 min read31 min video

May 23, 2025|96,552 views|2,271|115

YC Y Combinator

Save to Pod

Key Moments

TL;DR

AI apps should be user-programmable, not chatbots bolted onto old software.

Key Insights

AI apps should be designed as user-programmable tools, allowing users to offload unwanted tasks through natural language.

Current AI integrations often feel like chores due to outdated software development methods and generic system prompts.

Editable system prompts enable users to customize AI behavior to their personal style and needs.

The future of AI apps lies in 'AI-native' design, rethinking products from the ground up to leverage AI's full potential.

Tools are essential for AI agents to perform actions in the real world, moving beyond simple Q&A or text generation.

Developers should focus on building AI-native products that automate work, rather than just embedding chatbots.

THE PROMISE AND PITFALLS OF CURRENT AI INTEGRATIONS

The core promise of AI lies in its potential to act as a 'rocket ship for the mind,' enabling users to program software with natural language and automate tasks. However, many current AI applications fall short, feeling more like chores than powerful tools. This disconnect often stems from applying old software development techniques to AI, failing to unlock its full capabilities. Experiences with AI can be radically different: empowering when using tools like Cursor or WindSurf, yet frustrating when integrated into existing applications like Gmail. The latter often adds work rather than saving it, indicating a fundamental misunderstanding of AI's potential in these contexts.

THE PROBLEM WITH GENERIC SYSTEM PROMPTS

A key issue hindering effective AI application design is the misuse of system prompts. These prompts define the AI's persona and job, but in many current tools, they are generic, hidden, and uneditable. For instance, an AI email assistant might produce outputs that don't sound like the user because the system prompt dictates a formal, generic tone, prioritizing safety and brand image over personalization. This 'one-size-fits-all' approach, a relic of traditional software development, limits the AI's ability to act as a personalized assistant, forcing users to adapt to the AI rather than the AI adapting to the user.

EMPOWERING USERS WITH EDITABLE SYSTEM PROMPTS

The solution proposed is to grant users control over system prompts, allowing them to define the AI's behavior according to their specific needs and style. By editing the system prompt, a user can instruct the AI to adopt their personal voice, priorities, and workflow. For example, an email assistant could be programmed to maintain brevity, reflect a specific professional role, or prioritize certain communications. This makes the AI a true collaborator, capable of mirroring the user's own mental model and significantly reducing the manual effort required for repetitive tasks.

THE 'AI-HORSELESS CARRIAGE' ANALOGY AND AI-NATIVE DESIGN

The concept of 'AI-native' design is crucial, drawing an analogy to the early 'horseless carriages' which were essentially carriages with engines, lacking true automotive innovation. Similarly, many current AI applications are described as 'AI-horseless carriages' – old software with AI bolted on, failing to leverage the technology's potential. True AI-native software, like modern cars, is redesigned from the ground up. Products should be conceived with the primary goal of offloading repetitive user work, rather than simply inserting a chatbot into an existing workflow. This involves rethinking entire product paradigms.

TOOLS AS THE ENGINE FOR AI AGENTS

Beyond text generation, the true power of AI agents lies in their ability to interact with the real world through tools. These tools enable AI to perform actions like labeling emails, archiving messages, writing drafts, or even orchestrating complex workflows across different applications like Slack, Google Docs, or GitHub. Companies are emerging that build systems allowing agents to call various tools, transforming AI from a conversational interface into an agent capable of accomplishing tasks on behalf of the user, moving past the limiting chatbot paradigm towards a future of automated work.

THE FUTURE: USER-PROGRAMMABLE, TOOL-ENABLED AI

The future of AI applications is user-programmable and tool-enabled. While the ability to write effective system prompts may not be universal today, it is an accessible skill that will likely become commonplace, similar to computer literacy. Developers should prioritize building AI-native products that empower users to customize AI behavior through editable prompts or intuitive feedback mechanisms, and integrate tools that allow agents to act autonomously. This shift will lead to software that genuinely offloads work, allowing users to focus on higher-level, creative, and strategic tasks, truly realizing AI's potential.

Mentioned in This Episode

●Software & Apps

●Companies

●Organizations

●Concepts

●People Referenced

Designing Better AI Apps

Practical takeaways from this episode

Do This

Design AI applications from the ground up with AI native principles.

Focus on offloading repetitive user tasks to AI.

Empower users by giving them control over system prompts or allowing AI to learn their preferences.

Leverage tools and agents that can accomplish tasks in the real world.

Provide transparency into how AI agents are instructed (system prompts).

Avoid This

Don't simply embed a generic chatbot into an existing product.

Avoid treating system prompts as proprietary code hidden from the user.

Don't rely solely on a one-size-fits-all approach for AI features.

Avoid 'kid glove' mentality; let users leverage the full power of AI models.

Don't force users to manually write complex, long system prompts from scratch.

Common Questions

Many AI features are built using old software development techniques, treating AI integrations like adding an engine to a carriage. The AI often uses generic, hidden system prompts that don't reflect the user's personal style or needs, leading to output that feels unnatural and requires more work to correct.