Key Moments
High Agency Pydantic over VC Backed Frameworks — with Jason Liu of Instructor
Latent Space PodcastKey Moments
Instructorizes LLMs by providing structured data output, simplifying AI development.
Key Insights
Instructor, a Python SDK, simplifies obtaining structured data from LLMs, moving beyond simple string outputs.
The framework emphasizes a 'requests'-like philosophy, aiming for widespread adoption and ease of use.
Instructor supports various use cases including data extraction, knowledge graph generation, and query understanding.
While function calling offers precise schema definition, JSON mode can be more cost-effective for simpler outputs.
The AI engineering landscape is shifting towards enabling motivated software engineers, rather than solely relying on traditional ML expertise.
Jason Liu's consulting approach with Instructor prioritizes solving interesting problems over building a venture-backed company.
FROM STRING OUTPUTS TO STRUCTURED DATA
Jason Liu, creator of Instructor, discusses the fundamental shift LLMs represent: moving from raw string outputs to structured data. Instructor acts as a Python SDK that wraps OpenAI's SDK, focusing on providing typed responses. This allows developers to work with data structures, opening up possibilities for complex applications like data extraction, knowledge graph generation, and sophisticated query understanding, akin to solving LeetCode problems with LLM outputs.
THE 'REQUESTS' PHILOSOPHY FOR ADOPTION
Inspired by the simplicity and ubiquity of the 'requests' library in Python for HTTP calls, Liu has adopted a similar philosophy for Instructor. The goal is for Instructor to become a standard, almost built-in tool in the LLM development ecosystem. This approach prioritizes developer experience and ease of integration, aiming for a state where developers naturally opt for Instructor without overthinking it, much like they do with 'requests'.
NAVIGATING FUNCTION CALLING AND JSON MODE
The conversation delves into the nuances of structured output generation. While function calling offers robust schema definition and validation, allowing for complex relationships and constraints, JSON mode provides a simpler and potentially more cost-effective way to get JSON output. Liu highlights that function calling shines when precise schema definition is crucial, whereas JSON mode might suffice for less critical or simpler output structures.
EMERGING USE CASES AND ARCHITECTURAL SHIFTS
Instructor's capabilities extend to extracting complex graphs, defining nodes and edges between entities, and understanding nuanced user queries. Liu emphasizes that embeddings alone are often insufficient for complex queries. Instructor helps resolve these into structured requests, enabling more sophisticated data manipulation and interpretation, moving beyond simple retrieval to actionable insights and complex data processing.
THE EVOLUTION OF AI ENGINEERING TALENT
Liu argues that the demand for AI capabilities often outstrips the supply of traditional ML engineers. He advocates for recognizing and empowering motivated software engineers to transition into AI engineering roles. The focus is shifting from deep ML expertise to skills like prompt engineering and working with tools like Instructor, enabling rapid development and problem-solving in the AI space.
CONSULTING OVER VENTURE BACKING
Distinct from many in the AI startup scene, Liu intentionally pursues a consulting path with Instructor rather than seeking venture capital. He finds more fulfillment in tackling interesting, diverse problems through consulting, such as building AI for insurance or M&A reporting. This approach allows for a sustainable business model focused on delivering value, rather than the immense pressure of scaling to a billion-dollar valuation.
AGENCY, PROCESS, AND MEASURING SUCCESS
The discussion touches on 'high agency,' defined as the courage to act despite fear and to focus on process over outcome metrics. Liu uses pottery and software development as examples, where focusing on the amount of clay used or the number of commits, respectively, leads to skill development. This contrasts with outcome-based metrics that can be easily gamed or are too outcome-dependent.
THE FUTURE OF WORKFLOWS AND PROMPTS AS CODE
Liu envisions a future where LLM interactions are managed through defined workflows and DAGs (Directed Acyclic Graphs), moving away from continuous looping. He reiterates the importance of prompts being treated as code, emphasizing that Instructor separates instructions, data, and output types. This structured approach allows for better control, maintainability, and adaptation of AI systems as business needs evolve.
Mentioned in This Episode
●Products
●Software & Apps
●Companies
●Organizations
●Concepts
●People Referenced
Common Questions
Jason Lou was initially very skeptical of language models, preferring traditional methods like matrix factorization and classical models for tasks like recommendations and image classification. He dismissed them until the release of ChatGPT, which prompted him to acknowledge their incredible potential.
Topics
Mentioned in this video
The SDK that Instructor is built as a simple wrapper around, aiming to handle response models and simplify LLM interactions.
A recommendation framework developed by Jason Lou at Stitch Fix, achieving high adoption and processing millions of requests daily.
A tool mentioned that, along with Guardrails, used XML and Pydantic for structured data extraction from prompts, built with instruct models in mind.
A Python SDK designed to simplify LLM interactions by providing structured data outputs, inspired by the 'requests' library.
An LLM framework mentioned in the context of interacting with Instructor and the broader LLM ecosystem.
A framework presented as the antithesis to 'Show me the prompt', discussed in relation to short-term metrics and prompt management.
Used in conjunction with GPT-3 embeddings and Fe for a similarity search system at Stitch Fix.
A tool mentioned that, along with Marvin, used XML and Pydantic for structured data extraction from prompts, built with instruct models in mind.
A Python library used in Instructor to define schemas and map them, enabling complex data structures and validation.
A Python HTTP library that serves as the philosophical inspiration for Instructor, aiming for similar ease of use and ubiquity.
A workflow management tool that Jason Lou uses and discusses in the context of AI dags and modular components.
A company Jason Lou applied to in January 2023, where he was rejected due to lack of LLM experience.
Used by Jason Lou for observability and storing prompt/response data, rather than relying on specialized observability startups.
A notable workflow tool mentioned by a host.
An early language model that Jason Lou was skeptical of, but later acknowledged its capabilities after ChatGPT's release.
An Anthropic model praised for its cost-effectiveness and function calling abilities.
An LLM framework mentioned in the context of market share alongside LangChain.
The release of ChatGPT prompted Jason Lou to write an apology letter for his prior skepticism towards language models.
An Anthropic model mentioned as having better function calling capabilities and breaking cost-performance trends.
A company whose models (Sonnet, Haiku) are praised for cost-effectiveness and function calling capabilities, though with minor parsing issues in production.
Mentioned as an example of a large company that provides fully managed solutions, contrasting with the trend of developers wanting more control.
An observability tool used for latency tracking, distinct from LLM-specific observability.
Mentioned as an example of a large company that provides fully managed solutions, contrasting with the trend of developers wanting more control.
A workflow automation tool mentioned for its extensive connectors and relevance to agent workflows.
A platform Jason Lou happily uses, aligning with his preference for developer-centric tools.
Competitor to Prefect, mentioned by a host.
An observability tool used for latency tracking, distinct from LLM-specific observability.
Mentioned as the current employer of C Chan, who previously worked at Tesla with Karpathy. Also, the company behind GPT-3 and ChatGPT, whose release prompted Jason Lou to apologize for his prior dismissal of LLMs.
A company Jason Lou applied to in January 2023, where he was rejected due to lack of LLM experience.
Mentioned as an organization whose member, John, is working on inversion models.
A location where Jason Lou took a sabbatical in New York after a hand injury, during which he explored pottery, Jiu-Jitsu, and LLMs.
Mentioned as a source of the common advice for startups to have co-founders, which Jason Lou suggests is not always necessary.
More from Latent Space
View all 172 summaries
86 minNVIDIA's AI Engineers: Brev, Dynamo and Agent Inference at Planetary Scale and "Speed of Light"
72 minCursor's Third Era: Cloud Agents — ft. Sam Whitmore, Jonas Nelle, Cursor
77 minWhy Every Agent Needs a Box — Aaron Levie, Box
42 min⚡️ Polsia: Solo Founder Tiny Team from 0 to 1m ARR in 1 month & the future of Self-Running Companies
Found this useful? Build your knowledge library
Get AI-powered summaries of any YouTube video, podcast, or article in seconds. Save them to your personal pods and access them anytime.
Try Summify free