Mobile LM
Study / ResearchMentioned in 1 video
A paper by Meta that studies models under 1 billion parameters, finding depth is more important than width and that GQA helps.
A paper by Meta that studies models under 1 billion parameters, finding depth is more important than width and that GQA helps.
![Best of 2024: Synthetic Data / Smol Models, Loubna Ben Allal, HuggingFace [LS Live! @ NeurIPS 2024]](https://i.ytimg.com/vi/AjmdDy7Rzx0/maxresdefault.jpg)