All news with #bigquery tag

🧭 Google outlines an AI-native stack to transform data scientists into agentic architects, unifying development, real-time data access, and production-grade agent deployment. Enhancements to Colab Enterprise notebooks add native SQL cells, editable visualizations, and an interactive Data Science Agent that can orchestrate BigQuery ML, DataFrames, and Spark workflows. The Lightning Engine is now generally available to accelerate Spark, while previews for stateful BigQuery continuous queries and autonomous embedding generation bring real-time streaming and vector search into analytics. A 'Build-Deploy-Connect' toolkit, including the Agent Development Kit, MCP Toolbox, and Gemini CLI extensions, helps move notebook prototypes into secure, scalable agent fleets.

🚀 Google Cloud announced BigQuery performance and usability enhancements to accelerate generative AI inference. Improvements include >100x throughput for first-party text generation and >30x for embeddings, plus support for Vertex AI Provisioned Throughput and dynamic token batching to pack many rows per request. New reliability features—partial-failure mode, adaptive traffic control, and robust retries—prevent individual row failures from failing whole queries and simplify large-scale LLM workflows.

🧭 Google Cloud has expanded the Data Science Agent in Colab Enterprise notebooks to support BigQuery ML, BigQuery DataFrames and Spark, enabling large-scale data transformation, model training, and inference directly on BigQuery or via Serverless for Apache Spark. The agent can now auto-retrieve BigQuery table metadata and lets you add tables via an @ mention from your current project to provide prompt context. To invoke frameworks, include keywords such as BigQuery ML, BigFrames, or PySpark; sample prompts are provided to guide forecasting, supervised learning, and dimensionality reduction workflows. Notable limitations: generated PySpark targets Spark 4.0 and @ mentions only search the current project; BigQuery improvements are available now in BigQuery notebooks and coming soon to Vertex AI.

🚀 Google expanded BigQuery ML to generate embeddings from Gemini and over 13,000 open-source text-embedding models via Hugging Face, all callable with simple SQL. The post summarizes model tiers to help teams trade off quality, cost, and scalability, and introduces Gemini's Tokens Per Minute (TPM) quota for throughput control. It shows a practical workflow to deploy OSS models to Vertex AI endpoints, run ML.GENERATE_EMBEDDING for batch jobs, and undeploy to minimize idle costs, plus a Colab tutorial and cost/scale guidance.

🔍 ODOT teamed with Google Cloud and North Highland to centralize decades of bridge inspection, location, and maintenance data into BigQuery and govern it with Dataplex, creating a single trusted source for analysis. Non-technical and technical staff can query complex datasets conversationally through Gemini in Looker, while BigQuery ML powers predictive models to flag at-risk bridges ahead of failures. Secure sharing via Analytics Hub and unified governance enables better resource allocation, improved safety, and faster, data-driven decisions across the agency.

🚀 Google Cloud and SAP announced tighter integration to unify enterprise data and accelerate intelligent automation. SAP Business Data Cloud now connects to BigQuery via Datasphere, enabling bidirectional replication and AI-ready analytics. Procurement is simplified on the Google Cloud Marketplace with SAP BTP. New agent tooling—Agentspace, the Agent Development Kit, A2A and MCP standards—and expanded M4 memory-optimized VMs certified for SAP HANA aim to speed deployments, improve data consistency, and enable autonomous process automation.

📘 Google Cloud has published a new ebook, A Practical Guide to Data Science with Google Cloud, aimed at practitioners adopting an AI-first approach across BigQuery, Vertex AI, and Serverless for Apache Spark. The guide emphasizes unified, streamlined workflows enabled by a central notebook experience that blends SQL, Python, and Spark and includes assistive features in Colab Enterprise to generate multi-step plans and code. It explains how a unified data foundation lets teams manage structured and unstructured data together and use familiar SQL to process documents and images. The ebook also offers real-world use cases with linked notebooks so practitioners can run the examples and accelerate delivery.

🚀 Google Cloud announced Dataproc multi-tenant clusters to let many data scientists share a single cluster for interactive notebook workloads while preserving per-user authorization. The feature maps individual Google identities to service accounts, externalizes mappings to a YAML file, and supports updates on running clusters. Jupyter kernels launch via the Jupyter Kernel Gateway across worker nodes, with optional Vertex AI Workbench integration and the BigQuery JupyterLab Extension. Administrators retain IAM-based least-privilege control and cluster hardening isolates credentials and OS users.

🔍 BigQuery's new Column Metadata (CMETA) index is an automated, highly scalable metadata index that improves query pruning and reduces compute for extremely large tables. CMETA stores snapshots of block- and column-level statistics and is maintained transparently by BigQuery with no user intervention. Early adopters report up to 60x faster queries and up to 10x lower slot usage for selective filters, particularly on clustered columns.

🏭 Tata Steel implemented a unified manufacturing data foundation on Google Cloud, centralizing OT and IT sources into a Manufacturing Data Engine built on BigQuery. The multi-path ingestion architecture leverages partners such as Litmus and ClearBlade to collect real-time PLC telemetry, while SAP, APIs, and in-house sensors feed batch and staging pipelines. The design emphasizes secure upstaging, partitioned storage with archival to Cloud Storage, and enables predictive maintenance, environmental KPI reporting, and reduced human presence in hazardous areas.

🔍 StreamSight is an AI-driven application developed by BMG in partnership with Google Cloud to improve transparency, speed, and accuracy in digital royalty forecasting and anomaly detection. The solution leverages BigQuery ML models (including ARIMA_PLUS and BOOSTED_TREE), uses Vertex AI and Python for training, and surfaces results in Looker Studio dashboards. It flags missing sales periods, rights mismatches, and sudden streaming spikes to reduce manual review and help accelerate fairer payouts. Currently a proof of concept, StreamSight is positioned for broader DSP integrations and richer data inputs to extend its capabilities.

🔁 Soft failover in BigQuery Managed Disaster Recovery defers promotion of secondary compute and datasets until replication is confirmed, reducing the risk of data loss during planned disaster recovery tests. Unlike hard failover, which may promote immediately and accept RPO gaps to restore service, soft failover coordinates primary and secondary acquiescence to ensure data integrity. Available via the BigQuery UI, DDL, and CLI, it provides administrators with controlled, realistic DR drills without compromising production data.

🔔 This Google Cloud roundup summarizes recent product milestones, GA launches, previews, and integrations across the data analytics, BI, and database portfolio. It highlights updates to BigQuery, Firestore, Cloud SQL, AlloyDB, and adjacent services aimed at easing ingestion, migration, and AI-driven operations. Notable items include MongoDB-compatible Firestore GA, PSC networking improvements for Database Migration Service, and a redesigned BigQuery data ingestion experience. The post also emphasizes resilience and DR enhancements such as immutable backups and Near Zero Downtime maintenance.

☁️ Google Public Sector has been awarded a $48 million DLA Enterprise Platform contract to migrate the Defense Logistics Agency to a DoD‑accredited commercial cloud. The multi‑phased program will move key infrastructure and data to a modern, AI‑ready Google Cloud foundation and enable BigQuery, Looker, and Vertex AI analytics. Emphasizing secure‑by‑design infrastructure and Mandiant threat intelligence, the effort aims to reduce costs, improve resiliency, and accelerate AI‑driven logistics and transportation management.

📊 Storage Insights datasets put object and bucket metadata into a BigQuery-linked dataset that refreshes automatically, enabling detailed analysis of storage spend, distribution, lifecycle and Autoclass usage. Administrators can run SQL queries or use Gemini Cloud Assist for natural-language insights, then feed outputs into serverless batch operations to relocate, transition or delete data at scale. The feature supports organization-, folder-, project- or bucket-scoped datasets with daily updates and up to 90-day retention for operational and FinOps workflows.

🌍 BigQuery now integrates Earth Engine, enabling analysts to run raster analytics and join satellite-derived imagery with vector data using familiar SQL workflows. Initial capabilities include the ST_RegionStats() geography function plus a curated set of ~20 Earth Engine raster datasets for land cover, weather and climate analysis. With general availability, Google Cloud adds EU regional deployment, an Image Details tab for enhanced metadata visibility, usage and quota controls, and a preview map visualization in BigQuery Studio to render GEOGRAPHY query results on Google Maps for interactive exploration and stakeholder-ready outputs.

💬 The Conversational Analytics API lets developers embed natural‑language data queries and chat‑driven analysis directly into custom applications, internal tools, and workflows. It combines Google's AI, Looker’s semantic layer, and BigQuery context engineering to deliver data, chart, and text answers with trusted access controls. Features include agentic orchestration, a Python Code Interpreter, RAG‑assisted context engineering, and both stateful and stateless conversation modes. Enterprise controls such as RBAC, row‑ and column‑level access, and query limits are built in.

🔔 Google Cloud published a consolidated roundup of product releases and previews from early July through Aug 22, 2025, covering GA launches, public previews, and platform enhancements. Highlights include Earth Engine in BigQuery (GA), Vertex AI embedding scaling, new GKE features for NUMA alignment and swap, expanded NodeConfig controls, and Cloud Run with GPUs. Customers should review the linked documentation, request preview access via account teams where needed, and plan upgrades or migrations accordingly.