All news with #cloudflare bot management tag

Cloudflare Birthday Week 2025: Product and Policy Recap

🚀 Cloudflare’s Birthday Week 2025 summarized a broad set of product, policy, and community initiatives designed to strengthen the open Internet and prepare for AI-era and quantum threats. Highlights included a goal to hire 1,111 interns in 2026, new startup hubs, and expanded free developer access for students and non‑profits, plus sponsorships of open-source projects like Ladybird and Omarchy. Technical announcements ranged from post‑quantum upgrades and a Rust-based core proxy to R2 SQL, the Cloudflare Data Platform, Workers performance and security hardening, and new AI safety and bot-management tools.

AI Crawler Traffic: Purpose and Industry Breakdown

🔍 Cloudflare Radar introduces industry-focused AI crawler insights and a new crawl purpose selector that classifies bots as Training, Search, User action, or Undeclared. The update surfaces top bot trends, crawl-to-refer ratios, and per-industry views so publishers can see who crawls their content and why. Data shows Training drives nearly 80% of crawl requests, while User action and Undeclared exhibit smaller, cyclical patterns.

Signed Agents: Cryptographic Identification of Agent Traffic

🔐 Cloudflare introduces signed agents, a new classification that cryptographically verifies agent-originated traffic using Web Bot Auth HTTP message signatures. Signed agents represent end-user-directed automation rather than operator-owned crawlers, enabling sites to allow or block them with finer granularity. The update adds signed agents to the public Radar directory and to the bots and agents dashboard for visibility and submissions.

Cloudflare Launches AI Crawl Control with 402 Support

🛡️Cloudflare has rebranded its AI Audit beta as AI Crawl Control and moved the tool to general availability, giving publishers more granular ways to manage AI crawlers. Paid customers can now block specific bots and return customizable HTTP 402 Payment Required responses containing contact or licensing instructions. The feature aims to replace the binary allow-or-block choice with a channel for negotiation and potential monetization, while pay-per-crawl remains in beta.

Cloudflare AI Week 2025: Securing AI, Protecting Content

🔒 Cloudflare this week outlines a multi-pronged plan to help organizations build secure, production-grade AI experiences while protecting original content and infrastructure. The company will roll out controls to detect Shadow AI, enforce approved AI toolchains, and harden models against poisoning or misuse. It is expanding Crawl Control for content owners and enhancing the AI Gateway with caching, observability, and framework integrations to reduce risk and operational cost.