Cloudflare now continuously re-checks verified bots' behaviour
A new trust-band system watches whether bots that already passed Cloudflare's identity check still honour robots.txt and back off on HTTP 429s, with scores heading to Cloudflare Radar.
Cloudflare is adding continuous behaviour checks on top of its Verified Bots programme, rather than treating identity verification as a one-off pass. Bots already sign requests through Web Bot Auth and get listed in Cloudflare’s open BotBase catalogue; the new layer watches what they actually do afterwards.
Two signals matter: whether a bot’s traffic still lines up with what a site’s robots.txt allows under RFC 9309, and whether it slows down or keeps hammering a path after getting an HTTP 429. That behaviour rolls up into a “trust band” per bot rather than a fixed pass or fail. Cloudflare cites its 2025 delisting of Perplexity’s bot, which kept requesting resources sites had opted out of and then tried to work around the block, as the kind of pattern this is meant to catch automatically.
Trust-band scores are heading to Cloudflare Radar as a public record. Cloudflare says customers will eventually get a dashboard setting to pick the trust threshold that decides how a verified bot’s traffic gets handled on their own sites.
Why it matters: robots.txt has always worked on trust: there’s nothing stopping a crawler from reading it and ignoring it. Turning compliance into an ongoing, scored signal, backed by Cloudflare’s network-wide view, gives site operators something to point at when an AI crawler stops behaving and a public reference for which bots actually do.
The caveat: the per-site threshold control isn’t live yet. For now this is monitoring and public scoring on Radar, not something you can configure on your own domain.