Muon Optimizer

Software / App

An optimizer that requires full tensors, posing a challenge for parameter sharding strategies that flatten tensors.

Mentioned in 1 video

Videos Mentioning Muon Optimizer

Stanford CS25: Transformers United V6 I The Ultra-Scale Talk: Scaling Training to Thousands of GPUs

Stanford CS25: Transformers United V6 I The Ultra-Scale Talk: Scaling Training to Thousands of GPUs

Stanford Online

An optimizer that requires full tensors, posing a challenge for parameter sharding strategies that flatten tensors.