Sakana AI's PC-ALM trains 1000-layer networks without backpropagation
Backprop Alternative: Augmented Lagrangian Predictive Coding
Sakana AI published PC-ALM, a local training method that replaces backprop with layer-wise PI feedback controllers. By adding dual neurons (Lagrange multipliers) per layer, it fixes the signal decay that kills standard predictive coding in deep nets. They tested residual MLPs up to 1000 layers on Fashion-MNIST and CIFAR-10, nearly matching backprop performance. Code and paper are public. The motivation is split: neuroscience (the brain can't do exact backprop) and neuromorphic hardware (local dynamics run cheaper on specialized chips).
Why it matters: Sakana AI's PC-ALM replaces backpropagation with layer-local dynamics, training residual MLPs to 1000 layers at performance close to backprop, adding evidence that local training methods can scale.