Skip to content
Computing Life · Share · Yage

Prompt injection defense lives in the harness, not the model

打破 99% 安全假象:为什么 Prompt Injection 的防线与评测都在 Harness?

Ghostcommit showed the same Sonnet model rejected malicious PNG instructions 10/10 times in Claude Code, but obeyed 10/10 times in Cursor and Antigravity, leaking .env secrets. Lab-reported 99% defense rates suffer from five traps: static benchmark overfitting, misleading single-attempt ASR, LLM-as-judge drift, ignored utility-under-attack, and bare-model testing without tool shells. Deeper causes: LLMs lack hard instruction-data separation, and stronger models can follow injections more faithfully—Opus 4.6 with extended thinking saw ASR rise from 14.8% to 21.7%. A joint study by 14 researchers from OpenAI, Anthropic, and DeepMind tested 12 model-layer defenses; over 90% broke under adaptive attacks, with human red-teamers hitting 100%. The engineering fix is architectural isolation: CaMeL separates trusted planner from untrusted executor, and OpenClaw's dual-agent setup cut ASR from 100% to 0.31%. Harness-level deterministic tool gating, hook signature checks, and sandboxed least-privilege are the real controls.

Why it matters: Same Sonnet model refused a malicious image prompt in Claude Code ten times out of ten, but leaked .env secrets every time in Cursor and Antigravity.

Read the original ↗Export Markdown