What inspired Ian Webster to create Promptfoo?

Ian Webster was inspired by his experience at Discord, shipping AI to millions of users. He realized the difficulty in managing AI behavior and risks systematically. His previous work at Google on search evaluation, specifically the 'side-by-side' comparison method, also heavily influenced Promptfoo's initial design.

What are the main types of AI application risks Promptfoo addresses?

Promptfoo addresses both foundation risks (like toxicity, largely handled upstream) and application risks. Application risks include issues like improper access control in RAG systems, insecure agent tools, or giving AI capabilities that can be misused. It also covers softer risks like brand reputation and compliance.

How does Promptfoo's testing process work?

Promptfoo works by taking information about your target application and its use case. It then generates attacks tailored to this context using AI models. These attacks are evaluated, and feedback is used to refine them, mimicking a continuous red teaming process.

What are Promptfoo's 'plugins' and 'strategies'?

Plugins represent different risk areas (e.g., OWASP LLM Top 10, compliance, trust & safety) that produce specific attack objectives tailored to your application. Strategies are the attack methods derived from ML research, modified by Promptfoo to be effective on real-world applications.

What should I do after Promptfoo identifies security issues in my AI app?

After identifying issues, you can harden prompts, tune guardrails, or fix underlying code problems like API access controls. Promptfoo's commercial version offers remediation guidance, suggesting prompt updates or guardrail adjustments based on the findings.

Why is Promptfoo seeing traction with large enterprises like Fortune 500 companies?

Large companies have the most to lose from AI security failures and face significant top-down pressure to adopt AI. They are less likely to cut corners on red teaming and security. Promptfoo's approach of integrating security into the development lifecycle resonates with their need for robust, measurable risk management before deployment.

What is the future of AI security tooling, according to Ian Webster?

The future lies in tools that are closer to developers and integrated earlier in the software development lifecycle, ideally within CI/CD pipelines or as IDE plugins. This is because complex AI architectures like agents and RAGs require deeper code awareness and observability than simple front-door testing can provide.

Key Moments

Breaking AI to Fix It: Ian Webster's Journey from Discord's Clyde to Promptfoo's $18M Series A

Latent Space Podcast

Science & Technology4 min read44 min video

Oct 24, 2025|1,455 views|27|1

Save to Pod

Key Moments

TL;DR

Promptfoo's Ian Webster discusses AI security testing, shifting from evals to red teaming on a $18M funding round.

Key Insights

Promptfoo evolved from an AI evaluation tool to a security red teaming platform due to the critical need for production AI risk management.

AI application security focuses on application-specific risks and developer errors, rather than just foundational model toxicity.

Promptfoo uses AI-to-AI testing by generating tailored attacks to find vulnerabilities in AI applications, simulating real-world threats.

The platform supports various risk areas including technical security, compliance, trust & safety, and brand reputation.

Security for AI is shifting earlier in the development lifecycle, moving from runtime solutions to CI/CD integration.

The rise of complex agents and RAGs necessitates more sophisticated testing and observability beyond simple input/output checks.

FROM DISCORD'S CLYDE TO PROMPTFU'S ORIGINS

Ian Webster's journey into AI development began at Discord, where he led efforts on AI initiatives like the social chatbot Clyde. Experiencing firsthand the challenges of deploying AI to millions of users, particularly in managing risks and performing evaluations, sparked the idea for Promptfoo. This realization of a critical gap between AI development and safe production deployment became the foundational motivation for creating what is now Promptfoo, an industry-leading AI security testing platform that recently secured $18 million in Series A funding. The initial pain points of shipping AI at scale directly informed the product's direction.

PROMPTFOO: EVOLUTION FROM EVALUATION TO SECURITY RED TEAMING

Initially conceived as an open-source evaluation (eval) tool inspired by side-by-side comparisons used in search algorithm testing, Promptfoo quickly pivoted its go-to-market strategy. Webster argues that evaluations are becoming table stakes and commodities in the AI space. Instead of competing in a crowded eval market, Promptfoo focused on the higher-value, more critical area of AI security and risk management. This strategic shift recognized that while model providers address foundational risks like toxicity, significant vulnerabilities emerge from application-specific implementations and developer errors, an area Promptfoo aims to rigorously test.

UNDERSTANDING AI APPLICATION RISKS

Promptfoo differentiates between foundational model risks, such as toxicity, which are largely addressed by model providers, and application risks. These application risks stem from how developers implement AI, such as insecure access controls in RAG systems or poorly managed agent tool capabilities. Webster emphasizes that entities like OpenAI and Anthropic focus on model safety, but can't 'fix stupid' in application logic. He highlights that the interface between the AI application and the model is where most new vulnerabilities arise, especially when AI is integrated into enterprise workflows where maximum helpfulness isn't always the desired outcome, leading to softer risks like off-label recommendations or competitor praise.

THE PROMPTFOO TESTING METHODOLOGY

Promptfoo operates on an AI-versus-AI principle, generating tailored attacks against target AI applications. Users provide information about the application's purpose, business context, and intended use cases, which informs the testing process. The platform then utilizes 'plugins,' representing various risk areas, to generate attack objectives. These objectives are fed into 'strategies,' which are adapted ML research methods. An attacker LLM interacts with the target application, while a judge LLM evaluates the outputs, iteratively refining attacks to discover vulnerabilities. This dynamic, on-the-fly generation contrasts with static vulnerability databases, making it highly relevant to application security.

EMERGING TRENDS IN AI SECURITY TESTING

The landscape of AI security is rapidly evolving, with a strong push towards integrating security testing earlier into the Software Development Lifecycle (SDLC), ideally within CI/CD pipelines. While guardrails were an initial focus, the realization that they aren't foolproof has led to a demand for pre-deployment risk measurement. Companies are beginning to see AI security not just as securing the AI model itself, but as a broader category of software security, with tools like Promptfoo addressing this gap. The complexity of emerging AI architectures like agents and advanced RAG systems necessitates move sophisticated testing that goes beyond basic conversational checks.

THE ENTERPRISE ADOPTION AND FUTURE OF AI RED TEAMING

Promptfoo is seeing significant traction with large enterprises, including over 10% of the Fortune 500, due to their higher stakes in case of AI-related breaches. While startups might prioritize speed-to-market, established corporations require rigorous security validation. The enterprise version of Promptfoo focuses on extending the vulnerability management lifecycle, offering features for triage, integration with existing systems (like Jira and SIEM), monitoring, and remediation validation. Looking ahead, Webster anticipates the need for even tighter integration with developer workflows, potentially through IDE plugins and code-aware analysis, to address the security challenges posed by increasingly complex AI systems like agents and RAGs.

Mentioned in This Episode

●Software & Apps

●Companies

●Organizations

●Books

●Concepts

●People Referenced

Promptfoo: AI Application Security Best Practices

Practical takeaways from this episode

Do This

Focus on application-level security risks, not just foundation model toxicity.

Provide detailed business context and user personas for more accurate testing.

Utilize AI vs. AI for generating tailored attacks against your application.

Integrate security testing early in the SDLC, ideally in CI/CD pipelines.

Consider both offensive testing capabilities and defensive measures like proxies for MCP.

If managing MCP, prevent local execution and centralize observability and control.

Avoid This

Assume foundation models alone will solve all security issues.

Rely solely on black-box testing; provide internal context when possible.

Treat guardrails as a complete solution; they often require tuning.

Neglect security testing until after deployment.

Implement MCP in a naive wrapper around APIs without considering security implications.

Common Questions

Promptfoo is a tool designed to find and fix issues in AI applications, primarily focusing on security. It uses an 'AI vs. AI' approach, generating tailored attacks based on your application's context to uncover potential risks that static databases might miss.