Cloudflare lets sites allow search crawlers while blocking AI training bots

Stay discoverable in search while disallowing AI training

Cloudflare introduced new bot management rules that let sites stay indexed by search engines like Google and Bing while blocking crawlers used for AI training by companies such as OpenAI and Anthropic. The key is a new category called 'accountable mixed-use AI crawlers' — bots that serve both search and training purposes. Site owners get a one-click toggle to allow the search side and deny the training side. The post confirms the feature is live in the Cloudflare dashboard but does not specify a launch date.