SAM 3: The Eyes for AI — Nikhila & Pengchuan (Meta Superintelligence), ft. Joseph Nelson (Roboflow)

An earlier open-source model from Meta that Roboflow has been a believer in, cited as a precursor to SAM.

A platform for scientific research publications, mentioned as a place where work citing SAM is published.

A type of neural network architecture for vision tasks that has surpassed CNNs in many areas.

Microsoft Research, where Pengchuan previously worked before moving to Meta.

Likely refers to a specific type of dataset or benchmark used in relation to SAM 3's development.

An institute visited by Pengchuan where SAM was being used for imaging human cells.

Likely a typo or mishearing, but context suggests a biological or medical dataset or research area related to cell imaging.

Vision-Language Action Models, a type of task that users might want to perform with SAM 3 and LLMs.

Reinforcement Learning from Human Feedback, a training paradigm mentioned as a way to go beyond human performance.

Joseph Nelson's company, focused on making the world programmable with AI and providing tools for developers to build and deploy models.

A product/feature from RoboFlow that demonstrates real-time object detection per frame without needing to preserve unique identities.

Abbreviation for the Chan Zuckerberg Initiative, associated with the imaging institute visited by Pengchuan.

Likely refers to 'locus of movement' or a similar biological term, mentioned in the context of segmenting cells.

An interactive environment to try out SAM 3 and build with it.

A dataset commonly used in computer vision, mentioned in the context of creating a new benchmark for SAM 3.

Mentioned as an LLM used in some of the experiments for SAM 3's agent setup.

Supervised Fine-Tuning, a training method mentioned in comparison to RHF.

Roboflow's Detection Transformer model, designed for real-time segmentation and detection on edge devices.

The benchmark created by the Meta team for Segment Anything with Concepts, containing over 200,000 unique concepts.

A shared visual backbone component used in SAM 3, developed by the Fair group at Meta.

Likely a mishearing or typo, context suggests an organization related to AI safety or standards.

A library Nikila worked on briefly before focusing on Segment Anything.

An older model used as a comparison point to demonstrate the advancements made by SAM 3 in visual tasks.

A type of GPU mentioned in the context of real-time transformer performance on edge devices.

An LLM mentioned in Table 8 of the paper regarding SAM 3's agent performance.

A version of LLaMA that was fine-tuned for verification tasks in the data engine.

Mentioned as a device Pengchuan worked on egocentric foundation models for before joining the SAM team.

Agents that leverage large language models for complex reasoning and task execution, for which SAM 3 serves as a visual component.

Another open-source model from Meta that Roboflow has supported, mentioned alongside Mask R-CNN.

A model mentioned in the annotation pipeline for generating captions and non-face masks.

Convolutional Neural Networks, mentioned as a vision task architecture that Transformers have surpassed.

A model mentioned in the annotation pipeline for generating captions and non-face masks.

An aquarium in the US that uses underwater cameras and SAM for species tracking and population monitoring.