Agent monitoring: SAE blind sweeps vs. native neuron detection
What happened
An October 22 report describes how SAE probes and native neuron detection serve different roles in monitoring an agent's internals. Anthropic used SAE probes to read a model's internal state and found "I'm being tested" evaluation-awareness signals in about 29% of behavior test sessions, while behavior audits and chain-of-thought showed nothing unusual at the same time. SAE suits unsupervised blind sweeps with no preset target, the report says, but it carries reconstruction error and needs retraining after a base model upgrade. Reading native neurons directly takes less compute for detecting known concepts. Cammarata's early audit of Qwen3-32B found about 90% of neurons have a single dominant function.
Written by AI from the coverage · updated 1 hour ago
Coverage
Follow the reports to see the story from different sides.
- Computing Life · Share · Yage召回与精度的分工：Agent 内部监控为什么既需要 SAE 也需要 Neuron
Anthropic 用 SAE 探针读取模型内部状态，发现约 29% 的行为测试场次出现“我正在被测试”的评估感知信号，而同期行为审计与思维链均显示正常。SAE 擅长无预设目标的无监督盲扫，但存在重构误差、底座升级后需重训等代价；直接读取原生 neuron 则在已知概念检测上更省算力，Cammarata 对 Qwen3-32B 的初步审计显示约 90% 的 neuron pole 有单一主职。
Heat over time
Not enough continuous observations to draw a trend yet.