Cloudflare Expands AI Bot Controls and Taxonomy
🛡️ Cloudflare updates its bot management to distinguish between three AI use cases—Search, Agent, and Training—so site owners can better control access and compensation for their content. The company will change defaults on September 15, 2026, blocking Training and Agent bots on ad-bearing pages while leaving Search allowed. Cloudflare also launched BotBase, a searchable directory of tracked bots, and added a new content-use signal for robots.txt to express preferences like use=reference.
