Cloudflare adds individual access rules for AI browsing agents
The renamed "AI Crawler and Agent access policies" setting lets you allow or block the top Personal Agents one by one, instead of one blanket rule for all of them.
Cloudflare renamed its “Configure AI bot policies” dashboard setting to “AI Crawler and Agent access policies” on 4 November, and added the ability to set an access rule for each of the top Personal Agents individually, instead of one blanket rule covering all of them.
Personal Agents are bots that act on a visitor’s behalf, fetching or browsing a page because that visitor asked an assistant to look something up, as distinct from crawlers that gather content for indexing or model training. The new per-agent controls live under Security > Settings > AI Crawler and Agent access policies. Each Personal Agent inherits your existing Agents policy by default, and Cloudflare’s own recommendation is to leave that at allow rather than block.
Why it matters: blocking “AI agents” wholesale also blocks the ones fetching a page because a real visitor asked a question, which is a worse trade than blocking a training crawler. Separating Personal Agents from crawlers, and letting you set a rule per agent, means a site can keep showing up in an assistant’s answer without reopening the door to scraping. Worth checking this setting if you already have an Agents policy configured on Cloudflare.