How does OpenAI handle training data for financial models?

OpenAI processes all training data for financial analysis upstream into their large models. This ensures that financial data is integrated into the core model, making it accessible for anyone building on top of it, including startups and financial firms.

What were the key challenges in developing ChatGPT for Financial Services?

A major challenge was demonstrating the value of bundling data when similar connections already existed. OpenAI addressed this by recognizing high error rates in existing data sources and deciding to capture data internally for a controlled, end-to-end experience.

How does OpenAI ensure the safety of AI tools in finance?

The current product is designed as a productivity tool assuming human involvement for review and oversight. Future products aim to have intuitive safety profiles, mimicking human collaboration and leveraging existing hierarchical review systems within financial institutions.

Will AI errors in finance lead to career-ending consequences?

While a factual error that changes a conclusion can be career-ending, AI models are improving significantly in factual accuracy and liability. The focus is shifting from factual errors to 'human taste'—aesthetic and judgment-based aspects, which are also being addressed through training.

How is OpenAI training AI models to understand subjective 'taste' in finance?

OpenAI trains models using reinforcement learning with human feedback, where humans compare options and choose the better one. This approach, historically used in early AI development and applied to coding, is now being refined for financial applications, focusing on aesthetic judgment and nuanced decision-making.

What are the future predictions for AI in investment and trading?

By 2027, AI models are predicted to become much better at generating investment theses and trading ideas. While current applications focus on junior analyst tasks, future developments will move towards more complex roles requiring nuanced judgment, potentially even challenging prevailing market opinions.

How does OpenAI evaluate the performance of its financial AI models?

Creating accurate financial evaluations is challenging. OpenAI acknowledges current evaluations can be saturated or flawed and is actively developing better systems, like an investment banking evaluation, to predict the automation potential of specific roles.

Key Moments

Building ChatGPT for Finance — Joseph Kim, OpenAI's Head of Financial Services

Latent Space Podcast

Science & Technology5 min read28 min video

Oct 31, 2026|3,986 views|27|5

Save to Pod

Key Moments

TL;DR

OpenAI's ChatGPT for Finance integrates firm-specific data and workflows, but bundling essential financial data proved crucial due to extremely high error rates in third-party connectors (up to 40%).

Key Insights

OpenAI's ChatGPT for Financial Services is an enterprise product that combines ChatGPT's capabilities with enterprise-level features like firm-specific templates and data sources such as Dilupa and S&P.

Third-party financial data connectors experienced error rates as high as 40%, necessitating OpenAI to bundle and control the data experience internally for their financial services product.

95% of the development for ChatGPT for Financial Services involved standard startup practices like building on the existing plugin architecture, with only 5% requiring model-specific adjustments.

The team is developing new evaluation methods to better predict the automation potential of work in specific professions, such as investment banking, as current evaluations are considered 'broken' and easily exploitable.

Future AI models are expected to excel in areas requiring human judgment and 'taste,' moving beyond purely factual accuracy to better understand nuances like aesthetic appeal in presentations and subjective investment theses.

OpenAI is actively hiring for roles including Frontend/Backend Engineering, Product Managers (specializing in investment banking, equity analysis, wealth management), and crucial Subject Matter Experts with financial expertise.

Introducing ChatGPT for Financial Services

Joseph Kim, leading ChatGPT for Financial Services at OpenAI, discussed the launch of their new enterprise product designed to merge the user-friendly capabilities of ChatGPT with the specific needs of businesses. This product aims to provide enterprise-level features, including firm-specific templates and access to critical data sources like Dilupa and S&P. Developed in collaboration with design partners such as Morgan Stanley and Evercore, the initial response to the product has been overwhelmingly positive, highlighting a strong market demand for tailored AI solutions in finance.

The critical importance of data bundling

A significant challenge in developing ChatGPT for Financial Services was the integration of financial data. While external data connectors (MCPs) exist, OpenAI discovered extremely high error rates, with some major third-party connectors showing over 40% inaccuracy. This finding led to a crucial product decision: bundling financial data internally. Despite the apparent added cost and potential for perceived duplication with existing licenses, controlling the data pipeline end-to-end was deemed essential for ensuring reliability and quality. This proactive approach to data management has been a key factor in customer satisfaction, with clients frequently praising the bundled data as a primary benefit. This highlights how industry-specific realities, often overlooked in purely theoretical product development, are vital for success.

Research and deployment: OpenAI's dual focus

OpenAI operates as a 'research and deployment' company, prioritizing research while focusing on making its advancements accessible through products. This approach involves first understanding and developing model capabilities, then designing product features and interfaces to democratize that intelligence. The introduction of Codex, for instance, saw a dramatic surge in adoption across sectors like finance, legal, and sales, moving from near zero to 90% utilization after its release. This demonstrates the power of co-developing research with practical product applications, ensuring that cutting-edge AI models reach a broad user base and integrate seamlessly into daily workflows.

Building domain-specific products: The 95% standard approach

Surprisingly, Kim revealed that 95% of the work involved in creating ChatGPT for Financial Services utilized existing, general-purpose tools and architectures, such as the ChatGPT plugin architecture and its primitives. The core thesis was that while Codex/ChatGPT could provide about 80% of the functionality needed for a successful financial product, the remaining 20%—often domain-specific adjustments, user interface enhancements, or proprietary data integration—was critical. This 'secret sauce' is what enables startups to build billion-dollar businesses on top of foundational models like those from OpenAI, leveraging horizontal capabilities with their unique vertical expertise.

Navigating risk and ensuring safety in financial AI

Security is paramount in financial AI, especially as agents become more autonomous. The current ChatGPT for Financial Services is designed as a productivity tool for knowledge workers, operating under the assumption that a human is always involved in reviewing and validating outputs. This 'human-in-the-loop' paradigm simplifies product development by allowing responsibility for oversight to rest with the user. Looking ahead, OpenAI is focusing on developing intuitive safety profiles for AI that mimic human collaboration and oversight. This includes leveraging existing systems like read/write access controls and hierarchical review processes common in finance, where work is passed from junior analysts to VPs, with built-in checks to prevent errors.

The evolution of AI's role: From junior tasks to expert judgment

Initially, AI applications in finance have focused on tasks typically performed by junior analysts, such as creating slides or spreadsheets, leveraging the models' strength in executing specific, concrete instructions. This focus also aligns with user preferences, as many professionals find these repetitive, detail-oriented tasks tedious. However, OpenAI anticipates a shift towards more complex roles by 2027. Models will become increasingly adept at tasks requiring nuanced human judgment, 'taste,' and subjective reasoning, such as generating investment theses or refining presentation aesthetics. This progression requires significant research into areas like comparative analysis and preference modeling, drawing on human feedback to train models on subjective qualities.

Addressing 'taste' and subjective evaluation

Training AI on subjective qualities like 'taste' is technically achievable by training reward models based on human pairwise comparisons (e.g., 'which slide looks better?'). This method was foundational in early reinforcement learning and remains crucial. Humans excel at aesthetic judgment, often making rapid, intuitive decisions that complex models struggle to replicate. OpenAI is applying this by having humans rate pairs of outputs, such as presentation slides, to refine AI's ability to produce work that is not just factually correct but also aesthetically pleasing and contextually appropriate. This mirrors the development of programming, where a combination of verifiable rewards (e.g., code compilation) and human taste (e.g., clean design, maintainability) leads to better software.

The challenge of financial evaluation and future directions

Developing robust evaluations for financial AI is exceptionally difficult, as current benchmarks are often 'broken,' oversaturated, or poorly designed, leading to a false sense of progress. OpenAI is actively working on creating new evaluations that more accurately predict a model's ability to automate real-world financial tasks, such as those performed by junior investment banking analysts. The goal is to develop metrics that reflect true capability rather than susceptibility to 'hacking' the evaluation. Kim expressed confidence that by focusing on human expertise, rigorous data, and scalable foundational models, AI will make significant strides in handling complex financial reasoning and judgment by 2027, moving beyond junior-level tasks towards more sophisticated roles.

Mentioned in This Episode

●Software & Apps

●Companies

●Studies Cited

●People Referenced

Best Practices for Developing Domain-Specific AI Products

Practical takeaways from this episode

Do This

Build upon existing architectures like the ChatGPT plugin system.

Integrate industry-specific realities, data, and user preferences into product design.

Focus on providing unique value propositions that external tools lack.

Leverage existing human oversight systems for AI safety and validation.

Develop robust evaluation systems that accurately reflect desired capabilities.

Train models using human feedback for subjective qualities like aesthetic judgment and taste.

Avoid This

Ignore industry realities, competitive forces, or user goals when developing products.

Over-rely on off-the-shelf data sources with high error rates without internal validation.

Neglect the '20%' critical difference that separates a good foundational model from a successful product.

Assume AI models will fully replace human judgment; focus on augmentation and collaboration.

Develop AI evaluations that are easily 'gamed' or do not truly measure the desired capability.

Common Questions

ChatGPT for Financial Services is an enterprise-grade product from OpenAI that combines the power of ChatGPT's tools with enterprise-level capabilities. It allows for the use of firm-specific templates and data sources like Dilupa and S&P.

Topics

AI & Machine Learning Technology & Innovation Business & Entrepreneurship Large Language Models AI Product Development AI Evaluation Metrics Financial Services AI Enterprise AI Solutions AI Safety And Security Future Of Finance Technology

Mentioned in this video

Studies & Research

GDP-6

A previously used evaluation metric by OpenAI that became saturated and was eventually discontinued for reporting progress.

Software & Apps

ChatGPT

The AI language model discussed as the foundation for financial services tools.

Dilupa

Mentioned as a potential data source for ChatGPT in Financial Services.

ChatGPT Plugin

The architecture and primitives upon which ChatGPT for Financial Services was built, with potential for billion-dollar businesses.

Codex

An earlier AI model from OpenAI, a precursor to current financial applications, particularly useful for software development.

MCP

Referenced as a data source that, despite appearing valuable on paper, had high error rates in testing.

Companies

Evercore

A design partner for ChatGPT for Financial Services.

OpenAI

The organization developing ChatGPT and its financial services applications.

Morgan Stanley

A design partner for ChatGPT for Financial Services.

Google

Mentioned as an example of a company where read access controls are implemented for new software developers.

Organizations

S&P

Mentioned as a potential data source for ChatGPT in Financial Services.

People

Joseph Kim

Head of Financial Services for ChatGPT at OpenAI, featured speaker.

Media

Latent Space

The podcast or media outlet where the discussion is taking place.

Ask anything from this episode.

Save it, chat with it, and connect it to Claude or ChatGPT. Get cited answers from the actual content — and build your own knowledge base of every podcast and video you care about.

Get Started Free