Turing Award Winner Sutton Uses a 1967 Formula to Improve Streaming Reinforcement Learning
图灵奖得主Sutton新作:用一个1967年的公式,解决流式强化学习一大缺陷
Richard Sutton and coauthors proposed Intentional Updates, which derive the step size from the desired output change; Intentional AC approached SAC on MuJoCo under batch=1 streaming training without replay, while each update used about 1/140 of SAC’s FLOPs.
Why it matters: Sutton's group derives a learning-rate rule from the 1967 Kalman filter, cutting drift in streaming RL that learns one sample at a time with no replay buffer.