Why Bots Are Finally Getting ID-Checked After 30 Years

机器人三十年不查证件，为什么现在开始查了

Cloudflare launched BotBase for Operators on Aug 28, letting bot teams register identities and go through review. This is a sharp break: bots now make up 57.4% of web traffic, yet for 30 years the only gate was a voluntary robots.txt. The old equilibrium rested on three assumptions—search engines sent referral traffic back, false positives were cheap, and bot detection was easy. AI agents broke all three. LLM crawlers take content without sending visitors back (Anthropic's crawler generated one referral per 70,900 pages). Agents acting on behalf of paying users can't be blocked indiscriminately. Real browser environments defeat static fingerprinting. The only path left is requiring bots to declare identity and verify it cryptographically. A four-layer stack is forming: Web Bot Auth signing, purpose declaration, registration review, and platform defaults. The first three layers are voluntary; only the defaults have teeth. Cloudflare, serving 24.3% of all websites, controls the defaults, verification pipeline, directory, and payment channel. Blind spots remain: crawlers that refuse to register, private bilateral licensing deals, and API-based intermediaries all operate outside this system. The post notes Web Bot Auth has no formally adopted IETF document yet, and production formats already show intergenerational conflicts.

Why it matters: An insightful industry analysis that frames the BotBase launch within a 30-year arc of bot governance, not just a product announcement. Hits all three HKR axes, but as commentary rather than hard news it lands in the 78-84 band. Not scored higher because no cross-source cluste...