Stanford CS153 Frontier Systems | Andreas Blattmann from Black Forest Labs on Visual Intelligence

Latent diffusion introduced a more efficient method for training generative models by using a compressed, perceptually equivalent representation of images. This led to Stable Diffusion, which made high-quality image generation accessible and widely adopted.

The dominant belief in the ML community was that language modeling was the primary path to intelligence, with language seen as the core interface for reasoning. Visual processing was considered secondary or a niche topic.

Natural representations (like video and audio) are derived from sources humans cannot control, while unnatural representations (like text) are inherently human-made and highly efficient for communication. Black Forest Labs believes starting with natural representations is key to developing higher intelligence.

Initially, models were unimodal (e.g., text-to-image), primarily for content creation. Now, the focus is on multimodal models trained on natural data (image, video, audio) to achieve a deeper understanding and enable applications like physical AI and robotics.

Facing resource constraints, BFL focused on improving a specific area within image generation, aiming to be 10x better than existing models, specifically addressing the issue of generating more accurate hands. This initial focus led to product-market fit and customer feedback.

By observing users trying to achieve character consistency with Flux One, BFL learned about the demand for image editing capabilities. This led to the development of Flux One Context, an editing model that significantly improved character consistency.

Open models allow for greater customization by end-users and businesses with diverse preferences. This approach creates value by enabling tailored solutions, which is key to BFL's strategy of providing advanced, customizable visual intelligence infrastructure.

Self Flow is an algorithm published by Black Forest Labs that enables generative models to align multimodal representations. It solves the problem of combining representations from different data types (image, audio, video) to achieve a more comprehensive understanding, crucial for higher intelligence.

They implement comprehensive content filters on their API and comply with EU regulations like the GDPR for data privacy. Users can request deletion of their personal data, and BFL maintains strict guardrails that apply universally to all partners to prevent misuse.

Unlike text, images and videos are complex to label. Black Forest Labs uses automated labeling for large-scale pre-training with noisy data and human labeling for crucial, high-quality annotations in later training stages to ensure alignment with desired outcomes.

Diffusion models are data-inefficient during training but offer efficient inference through techniques like distillation, allowing for steps to be reduced significantly. Autoregressive models are more data-efficient in training but less capable of similar inference speed-ups through distillation.

Laten Adversarial Distillation is crucial for making models exceptionally efficient and commercially viable. It allows for the packaging of core technology into different versions (e.g., fast, high-quality) catering to both open-source developers and enterprise clients.

Black Forest Labs favors learning from natural representations like video and audio, mirroring human childhood development, over explicit 3D coordinate representations. They believe this approach is more flexible and generalizable for achieving higher forms of intelligence.