Llama
large language model by Meta AI
Common Themes
Videos Mentioning Llama

State of AI in 2026: LLMs, Coding, Scaling Laws, China, Agents, GPUs, AGI | Lex Fridman Podcast #490
Lex Fridman
Early well-known open-source LLM whose name is referenced with 'RIP Llama' in the discussion.

Segment Anything 2: Memory + Vision = Object Permanence — with Nikhila Ravi and Joseph Nelson
Latent Space
Mentioned in the context of Meta's research areas and future directions in AI.

E171: DOJ sues Apple, AI arms race, Reddit IPO, Realtor settlement & more
All-In Podcast
An open-source large language model, mentioned as a technology that startups could use with free compute credits.

How AI is Eating Finance - with Mike Conover of Brightwave
Latent Space
A large language model mentioned in the context of its difficulty in handling very long context windows and its role as a potential foundational standard.

E166: Mind-blowing AI Video: OpenAI launches Sora + Is Biden too old? Tucker/Putin interview & more
All-In Podcast
Meta's large language model, used as the base for the TestGen tool.

LLM Asia Paper Club Survey Round
Latent Space
A large language model used in the experiments, with scaled-down, randomly initialized versions being employed.
![Best of 2024: Synthetic Data / Smol Models, Loubna Ben Allal, HuggingFace [LS Live! @ NeurIPS 2024]](https://i.ytimg.com/vi/AjmdDy7Rzx0/maxresdefault.jpg)
Best of 2024: Synthetic Data / Smol Models, Loubna Ben Allal, HuggingFace [LS Live! @ NeurIPS 2024]
Latent Space
Previous version of Meta's models, trained on 1 trillion tokens; contrasted with LLaMA 3's longer training.
![2024 in Post-Transformer Architectures: State Space Models, RWKV [Latent Space LIVE! @ NeurIPS 2024]](https://i.ytimg.com/vi/LPe6iC73lrc/maxresdefault.jpg)
2024 in Post-Transformer Architectures: State Space Models, RWKV [Latent Space LIVE! @ NeurIPS 2024]
Latent Space
An example of a Transformer model where uploading a large book would involve comparing every word to every other word due to quadratic attention.
![Best of 2024: Open Models [LS LIVE! at NeurIPS 2024]](https://i.ytimg.com/vi/jX1nuoTs2WU/maxresdefault.jpg)
Best of 2024: Open Models [LS LIVE! at NeurIPS 2024]
Latent Space
Mentioned as a significant open model released in 2023, alongside Llama 2. Llama's license is noted as not meeting the open source definition due to specific use case restrictions.
![The State of AI Startups in 2024 [LS Live @ NeurIPS]](https://i.ytimg.com/vi/HM1d7kMebEI/maxresdefault.jpg)
The State of AI Startups in 2024 [LS Live @ NeurIPS]
Latent Space
Open-source language model that is highly competitive, ranking among the top evaluated models.

Supervise the Process of AI Research — with Jungwon Byun and Andreas Stuhlmüller of Elicit
Latent Space
A large language model developed by Meta, mentioned as potentially being used for certain tasks or uncertainty estimates within Elicit.
![[Paper Club] BERT: Bidirectional Encoder Representations from Transformers](https://i.ytimg.com/vi/V64q3p7DNjc/maxresdefault.jpg)
[Paper Club] BERT: Bidirectional Encoder Representations from Transformers
Latent Space
A large language model. Mentioned for scale comparison to BERT's parameters.

Why Compound AI + Open Source will beat Closed AI — with Lin Qiao, CEO of Fireworks AI
Latent Space
An open-source model developed by Meta, discussed as a key component of the open-source ecosystem that Fireworks AI builds upon.
![[Paper Club] Upcycling Large Language Models into Mixture of Experts](https://i.ytimg.com/vi/e_mkhFkKPEk/maxresdefault.jpg)
[Paper Club] Upcycling Large Language Models into Mixture of Experts
Latent Space
A large language model mentioned as an example of a large dense model that can be upcycled into MoE.

Why Google failed to make GPT-3 -- with David Luan of Adept
Latent Space
Open-source LLMs that are becoming increasingly capable, posing a challenge to pure-play foundation model companies.

Building the Silicon Brain - Drew Houston of Dropbox
Latent Space
A family of LLMs, with mention of 8 billion parameter versions running locally and later versions supporting 128k context.
![[Paper Club] Molmo + Pixmo + Whisper 3 Turbo - with Vibhu Sapra, Nathan Lambert, Amgadoz](https://i.ytimg.com/vi/8BN9CdIYaqc/maxresdefault.jpg)
[Paper Club] Molmo + Pixmo + Whisper 3 Turbo - with Vibhu Sapra, Nathan Lambert, Amgadoz
Latent Space
A family of open-source language models developed by Meta. Mentioned in the context of fine-tuning for vision tasks and as a comparison point for chat performance.

Production AI Engineering starts with Evals
Latent Space
Meta's family of open-source large language models. LLaMA 3 8B is mentioned as a powerful open-source model that could change the fine-tuning landscape, with many people incentivized for its success.
![[Paper Club] Writing in the Margins: Chunked Prefill KV Caching for Long Context Retrieval](https://i.ytimg.com/vi/VHwrhL_MSV4/maxresdefault.jpg)
[Paper Club] Writing in the Margins: Chunked Prefill KV Caching for Long Context Retrieval
Latent Space
Mentioned as a language model for which the 'Writing in the Margins' implementation provides a demo.

Open Source AI is AI we can Trust — with Soumith Chintala of Meta AI
Latent Space
A family of large language models from Meta AI, designed to be open source and used broadly.

Building an open AI company - with Ce and Vipul of Together AI
Latent Space
A foundational model that generated excitement in the AI community, influencing the development of the RedPajama dataset.

E129: Sam Altman plays chess with regulators, AI's "nuclear" potential, big pharma bundling & more
All-In Podcast
Facebook's open-source model mentioned in the context of AI development and regulation.

Information Theory for Language Models: Jack Morris
Latent Space
A model family that frequently releases model checkpoints.

Beating GPT-4 with Open Source Models - with Michael Royzen of Phind
Latent Space
A family of large language models developed by Meta AI. LLaMA 2 is discussed as a foundation for Phind's own model development.