Sakana AI's PC-ALM trains 1000-layer networks without backpropagation

Backprop Alternative: Augmented Lagrangian Predictive Coding

Sakana AI published PC-ALM, a local training method that replaces backprop with layer-wise PI feedback controllers. By adding dual neurons (Lagrange multipliers) per layer, it fixes the signal decay that kills standard predictive coding in deep nets. They tested residual MLPs up to 1000 layers on Fashion-MNIST and CIFAR-10, nearly matching backprop performance. Code and paper are public. The motivation is split: neuroscience (the brain can't do exact backprop) and neuromorphic hardware (local dynamics run cheaper on specialized chips).

Why it matters: Sakana AI's paper proposes a backprop-free training method that works on 1000-layer networks — solid research with a concrete artifact. But the benchmarks are still Fashion-MNIST and CIFAR-10, so it's not yet production-relevant, which caps the score.