Key Moments
Stanford CS547 HCI Seminar | Spring 2026 | Just-in-Time Objectives for Specialized AI Interactions
Stanford OnlineWant to know something specific about what's covered?
We've already dissected every moment. Ask and we will deliver (with timestamps).
Key Moments
Generic AI 'sporks' hinder user goals; 'just-in-time' objectives tailor AI to specific needs, offering specialized tools and insights without manual prompting.
Key Insights
Generic AI systems, likened to 'sporks,' fail to adequately serve diverse user needs due to a single interface attempting to fulfill multiple functions, leading to suboptimal outcomes.
Just-in-time (JIT) objectives automatically infer user goals from interaction traces, enabling AI systems to specialize outputs and generate tailored tools and feedback without explicit user prompting.
The JIT objectives approach was validated with over 200 participants, showing induced objectives were deemed accurate in the vast majority of cases and were selected over user-written objectives in 98% of instances.
A system called Poppins, demonstrating JIT objectives, produced outputs rated significantly higher in quality than baseline models for participants' real-world writing tasks, reducing user effort and augmenting abilities.
The Loom algorithm for topic modeling, a steerable default approach, allows domain experts to refine concept induction based on specific interests, enabling novel research and more interpretable data analysis.
Societal objective functions, translated from social science constructs, were used to re-rank social media feeds, significantly reducing partisan animosity in experiments by prioritizing non-engagement-based values.
The 'spork' problem in generic AI interactions
Current AI systems, particularly large language models (LLMs), often function as 'sporks' – attempting to be a jack-of-all-trades by combining multiple functionalities (personal assistant, expert, editor, confidant) into a single, identical interface. This generic approach results in suboptimal, non-specialized outputs, much like a spork is neither the best fork nor the best spoon. Manually overriding these generic outputs through prompting is a known challenge, being effortful, difficult, and reactive. This 'spork problem' extends beyond LLMs to other AI systems like social media feed rankings and topic models, potentially leading to homogeneous thinking, amplified polarization, and shallow understandings of important information. The reason for these sporks is that developers, lacking knowledge of all future user situations, bake assumptions into training data and fine-tuning processes, favoring generic outputs.
Introducing just-in-time (JIT) objectives for specialization
To combat the 'spork problem,' the research introduces 'just-in-time' (JIT) objectives, which specialize AI interactions by inducing user objectives from observed interaction traces at the moment of use. This approach aims to provide users with the specific tools needed, much like a color drawer holds the exact tools required for a task. The core challenges addressed are determining relevant objectives and embedding them interactively into AI systems. Technical methods include observing users, leveraging social science knowledge, and employing architectures like generator-evaluator models, feed re-ranking, and modular algorithms that expose objectives by design. These methods unlock specialized interactions, such as proactive tools for LLMs, reduced partisan animosity on social media, and steerable concepts for topic modeling that enable novel research.
Inducing accurate objectives from user interactions
The core of the JIT objectives work focuses on enabling everyday end-users to receive specialized AI assistance without explicit prompting. The key insight is to leverage the wealth of information available at interaction time, rather than relying on pre-defined assumptions. For instance, observing a user working on presentation slides can reveal their immediate goals, such as needing to finish an outline or tailor content for an undergraduate audience. JIT objectives automatically induce these in-the-moment goals by analyzing interaction traces like screenshots or web page DOMs. This allows AI systems to optimize on-the-fly towards more specialized outputs. Experiments with over 200 participants showed that JIT objectives were highly accurate, with induced objectives being preferred or selected in approximately 98% of cases when users could also provide their own custom objectives, demonstrating a strong user preference for automatically inferred goals.
Generalizable architecture for JIT objective integration
The JIT objective framework is designed to be generalizable across various AI systems, not just UI generation. It works by replacing the explicit user prompt with automatically induced JIT objectives. This objective then steers existing generators and evaluators within the AI system. For LLMs, this can involve appending the JIT objective to generation prompts or evaluation rubrics. For example, an LLM generating feedback can be tailored to focus on specific JIT objectives, leading to more targeted and useful output. Similarly, evaluators can refine their scoring based on these objectives, moving beyond general quality assessment. This approach allows for specialized outputs by improving objectives, even with off-the-shelf models, and can be integrated into existing architectures without requiring fundamental changes to the models themselves.
The Poppins system: demonstrating JIT objectives in practice
Poppins, an interactive system built using the JIT objectives framework and implemented as a browser extension, demonstrates the practical application of these concepts. When a user's presentation is open, Poppins can induce an objective, such as improving presentation structure, and suggest relevant tool ideas like a presentation flow organizer. It then generates code on the fly to create these tools and provides assistance without any user prompting. In hour-long in-lab sessions with 17 participants using their own writing tasks (e.g., short stories, scholarship applications), Poppins produced outputs rated as significantly higher quality than baseline LLMs. Participants reported reduced user effort, a helpful starting point for articulating goals, and ultimately, improved task performance, with some even achieving outputs on par with advice from human advisors, suggesting augmentation rather than replacement of human abilities.
Steerable topic modeling with the Loom algorithm
For domain experts, JIT principles are applied through the Loom algorithm, enabling steerable topic modeling. Unlike traditional methods that produce generic, vague themes, Loom surfaces high-level concepts as natural language descriptions with explicit inclusion criteria. This allows users to understand and steer the analysis based on their specific interests, moving from broad topics like 'government accountability' to more granular ones like 'federal government accountability.' The Loom algorithm employs modular operators: 'distill' to extract key points, 'cluster' to group related text spans, 'synthesize' to draw unifying concepts, and 'score' to verify concept occurrence. A 'seed' operator allows users to guide any step with specific terms. This approach has led to the development of a steerable text analysis tool, enabling researchers to uncover unanticipated trends and novel scientific research in diverse fields such as political analysis and public science communication.
Addressing societal challenges with JIT objectives and AI
JIT objectives also extend to broader societal issues, particularly social computing systems like social media feeds. To combat harms like increasing partisan animosity, the research focuses on translating social science constructs into 'societal objective functions.' For example, a detailed construct for 'anti-democratic attitudes and partisan animosity' (APA) used in social science has been operationalized into AI objectives. By using LLMs to re-rank social media feeds based on APA, experiments demonstrated a significant reduction in partisan animosity among both Democrats and Republicans. This algorithmic re-ranking, scaled by LLMs, offers a promising way to influence behavioral outcomes in the real world, moving beyond generic engagement metrics to address pressing societal concerns and foster healthier civic discourse.
Vision for user-owned AI and future research directions
The overarching vision is for 'user-owned AI,' where individuals control technology that specializes in real time based on their immediate and long-term goals, akin to Mary Poppins' magical bag of tools. This requires rethinking interfaces to be adaptive and generative, allowing users to curate their AI experiences. Future research also aims to expand AI's visibility beyond current LLM logs by creating an 'AI interaction observatory' to learn from large-scale user objectives and identify new problems and solutions for AI research. Finally, governance is crucial for user-owned AI, necessitating user-friendly methods for fine-tuning models and potentially pooling resources for better performance. This could involve building systems that automatically fine-tune models for frequent user objectives or designing protocols for trusted community members to share data and compute, potentially tackling collective societal goals through structured deliberation and monitoring.
Mentioned in This Episode
●Software & Apps
●Organizations
Common Questions
The 'spork problem' refers to AI interfaces that combine multiple functionalities into a single, generic input and output, leading to suboptimal or 'spork-like' results, similar to how a spork is neither the best spoon nor the best fork.
Topics
Mentioned in this video
More from Stanford Online
View all 86 summaries
48 minStanford CS547 HCI Seminar | Spring 2026 | Toward Ontological Multiplicity in AI and Computing
40 minStanford MS&E435 Economics of the AI Supercycle | Spring 2026 | Infrasctructure, Enterprise AI, SaaS
97 minLive from Stanford Health AI Week
35 minStanford Robotics Seminar ENGR319 | Spring 2026 | Towards Trustworthy Autonomy
Ask anything from this episode.
Save it, chat with it, and connect it to Claude or ChatGPT. Get cited answers from the actual content — and build your own knowledge base of every podcast and video you care about.
Get Started Free