Skip to content
Hacker News front page

TIME Is Serving AI Bots a Different Website, with Ads Built In

TIME returns different pages based on User-Agent. Humans get full HTML; ClaudeBot, PerplexityBot, and OAI-SearchBot get a stripped Markdown version at 1/23 the size. The Markdown contains sponsored FAQs from Ally Bank and PMI that don't appear on the human-facing site. Every bot request generates a fresh UUID, counting each fetch as an ad impression billed by tokens. GPTBot and ChatGPT-User are blocked with 406, while OAI-SearchBot gets through—TIME is selectively serving the bot-only version.

Why it matters: A measured test showing the same URL serves AI crawlers a 13KB plain-text Markdown page, 1/23 the size of the human site, with sponsored content mixed in.

Read the original ↗Export Markdown