Skip to content
Hacker News front page

Cloudflare blocks AI agents by default, and your agent can't tell

Show HN: We beat Cloudflare's bot detection (open-source stealth browser)

Since July 1, 2025, Cloudflare blocks AI crawlers by default on new domains. Worse, blocked requests return a 403 with a full HTML body—the 'Just a moment' challenge page—which language models read as real content and summarize confidently with fabricated answers. The author tested eight major anti-bot vendors: naive fetches got real content zero times, got a summarizable block page eight times, and got zero signals that the fetch failed. Their open-source Fortress stealth browser clears five of eight, returning live job listings from Indeed, 878 Zillow listings, and StockX's GraphQL pricing API. DataDome and Amazon click-walls remain unsolved in the open-source build; those require the hosted Tilion Cloud layer. The fix: detect block-page signatures and fail loud so agents stop before hallucinating from a challenge screen.

Why it matters: Cloudflare now blocks AI crawlers on new domains by default, and the 403 verification page can fool language models into summarizing answers that were never there.

Read the original ↗Export Markdown