CUHK Pion optimizer updates LLMs on iso-spectral manifolds to address AdamW and Muon instability
CUHK and collaborators introduced Pion, an optimizer that preserves weight singular values through orthogonal equivalence transformations, and reported that it kept a 60M normalization-free LLaMA-like model stable for 9.6B training tokens while AdamW and Muon collapsed with NaNs.
Why it matters: HKR-H/K/R pass: the hook is AdamW/Muon NaN instability, with a concrete isospectral update and 9.6B-token run. Niche optimizer math keeps it in 78–84, not same-day product news.