Koala V2

[Paper Club] Upcycling Large Language Models into Mixture of Experts

Latent Space

A recent MoE model that uses 64 experts.