All news with #r2 data catalog tag

Cloudflare Data Platform: R2 Pipelines, Catalog, SQL

🧭 Cloudflare announced the Cloudflare Data Platform, combining Cloudflare Pipelines, R2 Data Catalog, and R2 SQL to ingest, store, and query analytical tables directly on R2 object storage. Built on Apache Iceberg and open standards, the platform emphasizes engine interoperability and Cloudflare’s zero-cost egress. Pipelines offers exactly-once ingestion and SQL transforms today; stateful processing is planned. The products are open betas with usage-based pricing signals ahead of GA.

R2 SQL Deep Dive: Serverless Queries over R2 Data Platform

⚡ R2 SQL is Cloudflare’s serverless query engine that runs SQL directly against Iceberg tables stored in R2, eliminating the need for Spark or Trino clusters. The Query Planner uses R2 Data Catalog metadata and multi-level stats to prune manifests, files, and Parquet row groups so only necessary bytes are read. Execution is distributed across Cloudflare’s network using Workers and query workers running Apache DataFusion, with results serialized via Apache Arrow. An ordered, streaming planning pipeline enables early termination for ORDER BY ... LIMIT queries; R2 SQL is currently available in open beta.