All news with #apache iceberg tag

BigLake Metastore Adds Iceberg REST Catalog Support

🔔 Google Cloud announced general availability of BigLake metastore support for the Iceberg REST Catalog, offering a serverless, standards-based runtime metastore that enables interoperability across Iceberg-compatible engines (Spark, Trino) and BigQuery. The service provides credential vending, integrated governance via Dataplex Universal Catalog for lineage and data quality, and a UX console for creating and managing Iceberg catalogs. By removing the need to run custom metastore deployments, BigLake metastore aims to reduce operational overhead while preserving enterprise scale and security.

Amazon Redshift JIT ANALYZE for Apache Iceberg tables

📈 Amazon Redshift now supports Just‑In‑Time (JIT) ANALYZE for Apache Iceberg tables, automatically collecting table‑ and column‑level statistics during query execution. The feature uses intelligent heuristics and lightweight sketch data structures to determine when runtime statistics will improve optimizer decisions and to build high‑quality statistics on the fly. JIT ANALYZE is generally available in all AWS regions with Redshift and requires no configuration changes to begin improving query plans and performance.

Amazon Redshift Adds Apache Iceberg Write Support (GA)

🔔 Amazon Redshift now supports write operations to Apache Iceberg tables in general availability, enabling SQL DDL and DML including CREATE, SHOW, DROP, and INSERT for append-only workloads. Customers can execute concurrent read and write queries against Iceberg tables cataloged in AWS Glue Data Catalog while benefiting from transactional consistency and schema and partition evolution support. The capability is available in all regions where Amazon Redshift is offered.

Cloudflare Data Platform: R2 Pipelines, Catalog, SQL

🧭 Cloudflare announced the Cloudflare Data Platform, combining Cloudflare Pipelines, R2 Data Catalog, and R2 SQL to ingest, store, and query analytical tables directly on R2 object storage. Built on Apache Iceberg and open standards, the platform emphasizes engine interoperability and Cloudflare’s zero-cost egress. Pipelines offers exactly-once ingestion and SQL transforms today; stateful processing is planned. The products are open betas with usage-based pricing signals ahead of GA.

R2 SQL Deep Dive: Serverless Queries over R2 Data Platform

⚡ R2 SQL is Cloudflare’s serverless query engine that runs SQL directly against Iceberg tables stored in R2, eliminating the need for Spark or Trino clusters. The Query Planner uses R2 Data Catalog metadata and multi-level stats to prune manifests, files, and Parquet row groups so only necessary bytes are read. Execution is distributed across Cloudflare’s network using Workers and query workers running Apache DataFusion, with results serialized via Apache Arrow. An ordered, streaming planning pipeline enables early termination for ORDER BY ... LIMIT queries; R2 SQL is currently available in open beta.

Google Cloud and Partners Commit to Apache Iceberg

🔁 Google Cloud and an ecosystem of partners — including Confluent, Databricks, dbt, Fivetran, Informatica, and Snowflake — reaffirm support for the open table format Apache Iceberg to power modern lakehouse architectures. The post highlights Google innovations such as BigLake and a REST Catalog API that unify metadata and enable interoperability across engines like BigQuery, Databricks, and Snowflake. The collaboration aims to reduce data silos, enable time travel and pruning, and accelerate AI-ready analytics.