All news with #bigquery tag

Data Science Agent Adds BigQuery ML, DataFrames, and Spark

🧭 Google Cloud has expanded the Data Science Agent in Colab Enterprise notebooks to support BigQuery ML, BigQuery DataFrames and Spark, enabling large-scale data transformation, model training, and inference directly on BigQuery or via Serverless for Apache Spark. The agent can now auto-retrieve BigQuery table metadata and lets you add tables via an @ mention from your current project to provide prompt context. To invoke frameworks, include keywords such as BigQuery ML, BigFrames, or PySpark; sample prompts are provided to guide forecasting, supervised learning, and dimensionality reduction workflows. Notable limitations: generated PySpark targets Spark 4.0 and @ mentions only search the current project; BigQuery improvements are available now in BigQuery notebooks and coming soon to Vertex AI.

Gemini and Open-Source Text Embeddings Now in BigQuery ML

🚀 Google expanded BigQuery ML to generate embeddings from Gemini and over 13,000 open-source text-embedding models via Hugging Face, all callable with simple SQL. The post summarizes model tiers to help teams trade off quality, cost, and scalability, and introduces Gemini's Tokens Per Minute (TPM) quota for throughput control. It shows a practical workflow to deploy OSS models to Vertex AI endpoints, run ML.GENERATE_EMBEDDING for batch jobs, and undeploy to minimize idle costs, plus a Colab tutorial and cost/scale guidance.

Oklahoma DOT Modernizes Bridge Management with Google Cloud

🔍 ODOT teamed with Google Cloud and North Highland to centralize decades of bridge inspection, location, and maintenance data into BigQuery and govern it with Dataplex, creating a single trusted source for analysis. Non-technical and technical staff can query complex datasets conversationally through Gemini in Looker, while BigQuery ML powers predictive models to flag at-risk bridges ahead of failures. Secure sharing via Analytics Hub and unified governance enables better resource allocation, improved safety, and faster, data-driven decisions across the agency.

Google Cloud and SAP: Unified Data, AI Agents, and HANA

🚀 Google Cloud and SAP announced tighter integration to unify enterprise data and accelerate intelligent automation. SAP Business Data Cloud now connects to BigQuery via Datasphere, enabling bidirectional replication and AI-ready analytics. Procurement is simplified on the Google Cloud Marketplace with SAP BTP. New agent tooling—Agentspace, the Agent Development Kit, A2A and MCP standards—and expanded M4 memory-optimized VMs certified for SAP HANA aim to speed deployments, improve data consistency, and enable autonomous process automation.

New Practical Guide to Data Science with Google Cloud

📘 Google Cloud has published a new ebook, A Practical Guide to Data Science with Google Cloud, aimed at practitioners adopting an AI-first approach across BigQuery, Vertex AI, and Serverless for Apache Spark. The guide emphasizes unified, streamlined workflows enabled by a central notebook experience that blends SQL, Python, and Spark and includes assistive features in Colab Enterprise to generate multi-step plans and code. It explains how a unified data foundation lets teams manage structured and unstructured data together and use familiar SQL to process documents and images. The ebook also offers real-world use cases with linked notebooks so practitioners can run the examples and accelerate delivery.

BigQuery's CMETA: Column Metadata Index for Scale Performance

🔍 BigQuery's new Column Metadata (CMETA) index is an automated, highly scalable metadata index that improves query pruning and reduces compute for extremely large tables. CMETA stores snapshots of block- and column-level statistics and is maintained transparently by BigQuery with no user intervention. Early adopters report up to 60x faster queries and up to 10x lower slot usage for selective filters, particularly on clustered columns.

Tata Steel Enhances Monitoring with Google Cloud MDE

🏭 Tata Steel implemented a unified manufacturing data foundation on Google Cloud, centralizing OT and IT sources into a Manufacturing Data Engine built on BigQuery. The multi-path ingestion architecture leverages partners such as Litmus and ClearBlade to collect real-time PLC telemetry, while SAP, APIs, and in-house sensors feed batch and staging pipelines. The design emphasizes secure upstaging, partitioned storage with archival to Cloud Storage, and enables predictive maintenance, environmental KPI reporting, and reduced human presence in hazardous areas.

StreamSight: AI-Powered Music Royalty Forecasting Tool

🔍 StreamSight is an AI-driven application developed by BMG in partnership with Google Cloud to improve transparency, speed, and accuracy in digital royalty forecasting and anomaly detection. The solution leverages BigQuery ML models (including ARIMA_PLUS and BOOSTED_TREE), uses Vertex AI and Python for training, and surfaces results in Looker Studio dashboards. It flags missing sales periods, rights mismatches, and sudden streaming spikes to reduce manual review and help accelerate fairer payouts. Currently a proof of concept, StreamSight is positioned for broader DSP integrations and richer data inputs to extend its capabilities.

BigQuery Managed Disaster Recovery Adds Soft Failover

🔁 Soft failover in BigQuery Managed Disaster Recovery defers promotion of secondary compute and datasets until replication is confirmed, reducing the risk of data loss during planned disaster recovery tests. Unlike hard failover, which may promote immediately and accept RPO gaps to restore service, soft failover coordinates primary and secondary acquiescence to ensure data integrity. Available via the BigQuery UI, DDL, and CLI, it provides administrators with controlled, realistic DR drills without compromising production data.

What's New in Google Data Cloud: August Product Roundup

🔔 This Google Cloud roundup summarizes recent product milestones, GA launches, previews, and integrations across the data analytics, BI, and database portfolio. It highlights updates to BigQuery, Firestore, Cloud SQL, AlloyDB, and adjacent services aimed at easing ingestion, migration, and AI-driven operations. Notable items include MongoDB-compatible Firestore GA, PSC networking improvements for Database Migration Service, and a redesigned BigQuery data ingestion experience. The post also emphasizes resilience and DR enhancements such as immutable backups and Near Zero Downtime maintenance.

DLA Selects Google Public Sector for Cloud Modernization

☁️ Google Public Sector has been awarded a $48 million DLA Enterprise Platform contract to migrate the Defense Logistics Agency to a DoD‑accredited commercial cloud. The multi‑phased program will move key infrastructure and data to a modern, AI‑ready Google Cloud foundation and enable BigQuery, Looker, and Vertex AI analytics. Emphasizing secure‑by‑design infrastructure and Mandiant threat intelligence, the effort aims to reduce costs, improve resiliency, and accelerate AI‑driven logistics and transportation management.

Storage Insights datasets optimize Cloud Storage spend

📊 Storage Insights datasets put object and bucket metadata into a BigQuery-linked dataset that refreshes automatically, enabling detailed analysis of storage spend, distribution, lifecycle and Autoclass usage. Administrators can run SQL queries or use Gemini Cloud Assist for natural-language insights, then feed outputs into serverless batch operations to relocate, transition or delete data at scale. The feature supports organization-, folder-, project- or bucket-scoped datasets with daily updates and up to 90-day retention for operational and FinOps workflows.

Google Conversational Analytics API Brings Chat to Your Data

💬 The Conversational Analytics API lets developers embed natural‑language data queries and chat‑driven analysis directly into custom applications, internal tools, and workflows. It combines Google's AI, Looker’s semantic layer, and BigQuery context engineering to deliver data, chart, and text answers with trusted access controls. Features include agentic orchestration, a Python Code Interpreter, RAG‑assisted context engineering, and both stateful and stateless conversation modes. Enterprise controls such as RBAC, row‑ and column‑level access, and query limits are built in.

Earth Engine in BigQuery: Raster Analytics & Map Visuals

🌍 BigQuery now integrates Earth Engine, enabling analysts to run raster analytics and join satellite-derived imagery with vector data using familiar SQL workflows. Initial capabilities include the ST_RegionStats() geography function plus a curated set of ~20 Earth Engine raster datasets for land cover, weather and climate analysis. With general availability, Google Cloud adds EU regional deployment, an Image Details tab for enhanced metadata visibility, usage and quota controls, and a preview map visualization in BigQuery Studio to render GEOGRAPHY query results on Google Maps for interactive exploration and stakeholder-ready outputs.

What’s New in Google Cloud: Releases, Previews, and News

🔔 Google Cloud published a consolidated roundup of product releases and previews from early July through Aug 22, 2025, covering GA launches, public previews, and platform enhancements. Highlights include Earth Engine in BigQuery (GA), Vertex AI embedding scaling, new GKE features for NUMA alignment and swap, expanded NodeConfig controls, and Cloud Run with GPUs. Customers should review the linked documentation, request preview access via account teams where needed, and plan upgrades or migrations accordingly.