Humans missed 1 in 3 threats when approving AI coding agent commands
Scale X built a browser game where humans approve or deny commands from an AI coding agent. Across 40k+ runs and 409k decisions, players missed 33.7% of threats on average. The most-missed command was npm run analyze (64.7% miss rate)—it looks routine but exfiltrates data via a script in package.json. Threat miss rates climbed toward the end of sessions, consistent with permission fatigue. Over-blocking was also common: npm config set registry (a safe internal mirror) was blocked 59% of the time.
Why it matters: A security study backed by 40k game runs of behavioral data, with concrete numbers and a counterintuitive finding (64.7% miss rate for npm run analyze). Directly relevant to teams deploying AI agents. Score held at 78 because it's game-simulated data, not production, and Scale...