All news with #bot management tag

Cloudflare outage (18 Nov 2025): feature file duplication

⚠️ On 18 November 2025, Cloudflare experienced a major outage after a permissions change in a ClickHouse database caused duplicated metadata to be emitted into a Bot Management feature file, doubling its size. The oversized file exceeded a preallocated feature limit in the core proxy, triggering a Rust panic and widespread HTTP 5xx errors. Cloudflare halted propagation, restored a known-good file, and restarted the proxy; services were largely restored by 14:30 UTC and fully recovered by 17:06 UTC. The company apologized and pledged architectural and process hardening to prevent recurrence.

Per-Customer Bot Defenses: Behavioral Anomaly Detection

🚨 Cloudflare is introducing per-customer machine learning models that build dynamic baselines of legitimate traffic and detect behavioral anomalies to stop sophisticated, AI-driven scraping. The system ingests zone-level data to identify patterns like sequential page traversal or automated API access and then emits actionable outputs such as new Bot Detection IDs and adjustments to the Bot Score. Early beta results show hundreds of millions of flagged requests, and the capability will be available through Super Bot Fight Mode and Enterprise Bot Management.

Cloudflare Adds AI Crawl Control to Project Galileo

🛡️ Cloudflare is extending Project Galileo to include Bot Management and AI Crawl Control, giving participating journalists, independent publishers, and non-profits free tools to monitor and manage AI crawlers. These services help distinguish legitimate search crawlers from AI scrapers, analyze crawler behavior by type and provider, and apply tailored rules to protect content. The goal is to help news organizations preserve traffic, protect intellectual property, and negotiate fair compensation with AI companies.