BERT

Anthropic Head of Pretraining on Scaling Laws, Compute, and the Future of AI

Y Combinator

A model mentioned as an example of pre-training objectives considered before auto-regressive modeling became dominant.