I don't buy the claim that there is a one-in-three chance of building superintelligence capable of overthrowing governments within a single year.
Paul Christiano wants AI labs to pause pretraining and RL progress, but that simply cannot work in reality. Even if worries about model theft and loss of control are completely valid, the competitive environment offers no mechanism to slow down. As long as rivals keep sprinting forward, any lab that unilaterally halts its algorithm research is committing suicide. Using doomsday hypotheses about toppling governments to urge a pause in training will get nowhere. Safety researchers do not call the shots, and nobody is going to shut down methods that are actively running.