What were the early challenges for legal technology companies?

Early legal technology struggled with efficiency. Lawyers faced immense difficulties finding crucial evidence or case law, often requiring days of manual document review in basements or libraries, unlike the ease of finding everyday information on the internet.

How did the release of GPT-4 change Case Text's strategy?

Case Text, through an NDA with OpenAI, gained early access to GPT-4. Within 48 hours, they decided to pivot their entire company of 120 people to focus on building Co-Counsel, an AI legal assistant powered by this new technology.

Why was the pivot to AI met with internal resistance at Case Text?

After 10 years of steady growth and revenue with their existing products, many employees and even board members were resistant to such a radical shift. They questioned why they should abandon a working model for a new, unproven technology, requiring significant persuasion from leadership.

How did Case Text ensure the accuracy of its AI legal assistant?

Case Text employed a rigorous test-driven development approach, similar to software engineering, using thousands of tests to validate each prompt and function. They focused on breaking down complex legal tasks into smaller, manageable steps and ensured the AI could accurately cite sources and avoid hallucinations.

What is the difference between GPT-4 and OpenAI's new GPT-O1 model?

GPT-4 is analogous to 'System 1' thinking – fast, intuitive, and pattern-based. GPT-O1 shows promise in 'System 2' thinking, involving slower, more deliberate, and precise reasoning, which is crucial for complex tasks that require executive function and detailed analysis.

Are AI agents just 'GPT wrappers', or is there real IP being built?

Building a truly functional AI application, like Co-Counsel, involves far more than just wrapping an LLM. It requires proprietary datasets, custom integrations with systems like document management, sophisticated OCR handling, and extensive prompt engineering with rigorous testing, all of which constitute valuable IP.

Will AI take away legal jobs?

The Jobs are not expected to disappear entirely. Instead, AI like Co-Counsel will automate tedious, repetitive tasks, allowing legal professionals to focus on more strategic, interesting, and high-level work.

Why Vertical LLM Agents Are The New $1 Billion SaaS Opportunities

Y Combinator

Science & Technology3 min read38 min video

Oct 4, 2024|440,232 views|7,416|176

YC Y Combinator SaaS Vertical AI Open AI o1 ChatGPT Jake Heller Garry Tan Diana Hu Jared Friedman Casetext

Save to Pod

Key Moments

TL;DR

Vertical AI agents, like Casetext's Co-Counsel, are the next $1B SaaS opportunity, leveraging LLMs for critical tasks.

Key Insights

Building successful vertical AI agents requires a deep understanding of the specific industry's problems and workflows.

The shift from general-purpose LLMs to specialized vertical agents is key for creating significant SaaS value.

Early access to advanced LLM technology, like GPT-4, provided a significant competitive advantage.

Pivoting from initial strategies (e.g., user-generated content) to leveraging core AI capabilities is crucial for product development.

Demonstrating value through rigorous testing, prompt engineering, and addressing edge cases is essential for accuracy and trust.

New LLM architectures like OpenAI's '1' signal a move towards more deliberate, step-by-step reasoning, enhancing precision.

THE RISE OF VERTICAL AI AGENTS

The increasing sophistication of Large Language Models (LLMs) presents a new wave of billion-dollar SaaS opportunities, specifically through vertical AI agents. These agents are designed to excel in niche industries, addressing specific user needs more effectively than general-purpose AI. Casetext's journey, culminating in its acquisition by Thomson Reuters, exemplifies this trend, showcasing how deep domain expertise combined with cutting-edge AI can revolutionize an industry.

FROM LEGAL TECH INNOVATION TO LLM REVOLUTION

Casetext's origin story is rooted in a lawyer's frustration with inefficient legal technology. For years, the company focused on incremental improvements, attempting to build better tools for legal professionals. Early efforts, such as encouraging user-generated annotations of case law, failed to gain traction due to the time constraints and billing structures of lawyers, highlighting the importance of understanding the target user's workflow and incentives.

THE GPT-4 CATALYST AND STRATEGIC PIVOT

The advent of GPT-4 marked a profound inflection point for Casetext. Gaining early access to the technology, the company made a swift, company-wide pivot to build a new product, Co-Counsel, an AI legal assistant. This decision, made within 48 hours by the entire 120-person team, demonstrated a bold commitment to leveraging the LLM's capabilities to fundamentally change their product offering and market position.

OVERCOMING INTERNAL RESISTANCE AND MARKET SKEPTICISM

Shifting a decade-old company's focus to a new technology, even a revolutionary one like GPT-4, involved significant internal persuasion. Many long-term employees and even board members required convincing, given the company's existing success and the perceived risks of such a radical change. Demonstrating the technology's power through early customer reactions and the founder's direct involvement in building the initial prototype were key to overcoming this resistance.

ACHIEVING PRODUCT-MARKET FIT AND ACCURACY

The transition from incremental improvements to a fundamental shift with Co-Counsel led to true product-market fit, characterized by overwhelming customer demand and rapid growth. A critical challenge in the legal domain is the mission-critical nature of the work, where errors—or 'hallucinations'—are unacceptable. Casetext addressed this through rigorous test-driven development, detailed prompt engineering, and a methodical approach to breaking down complex tasks into smaller, verifiable steps.

ADDRESSING THE 'WRAPPER' CRITIQUE AND BUILDING IP

Critics often dismiss vertical AI companies as mere 'GPT wrappers.' However, building a truly effective vertical agent involves much more. It requires proprietary data, complex integrations with existing systems, sophisticated OCR processing to handle diverse document formats, and intricate prompt engineering frameworks. This multifaceted development process creates significant intellectual property that is difficult to replicate, going far beyond simple API calls to LLMs.

THE EVOLVING LANDSCAPE: GPT-4 TO OPENAI '1'

The evolution from GPT-3.5, which performed at the 10th percentile on bar exams, to GPT-4, which reached the 90th percentile, highlights the rapid advancements in LLM capabilities. The emergence of models like OpenAI's '1' signifies a shift towards more deliberate, 'System 2' thinking, enabling AI to perform complex reasoning and precise analysis. These advancements are crucial for building trust in high-stakes applications like legal work.

FUTURE OPPORTUNITIES AND INDUSTRY TRANSFORMATION

The journey of Casetext underscores that vertical AI agents represent a significant opportunity for creating new SaaS businesses. By focusing on solving specific user problems with advanced AI, companies can achieve substantial value. This transformation will not only automate tedious tasks but also free up professionals to focus on more strategic and interesting aspects of their work, leading to a more engaging and productive professional landscape across various industries.

Mentioned in This Episode

●Software & Apps

●Companies

●Books

●People Referenced

LLM Performance on Bar Passage Test

Data extracted from this episode

Model	Percentile	Note
GPT-3.5	10th percentile	Performed better than some, possibly due to random answers.
GPT-4 (Early Access)	90th percentile	Significantly improved performance after prompt engineering.

Common Questions

Case Text, founded by Jake Heller, was a legal tech company that initially focused on improving technology for lawyers. Following the release of GPT-4, they pivoted to build an AI legal assistant called Co-Counsel, which significantly increased their valuation, leading to an acquisition by Thomson Reuters.