All news with #data security tag

🔒 This post introduces the preview of IAM data governance tags for BigQuery column-level security. Built on Google Cloud Resource Manager tags with purpose=DATA_GOVERNANCE, these tags are global, support hierarchical classification up to five levels, and are replicated for disaster recovery. The article explains creating tag keys/values, attaching tags to columns via JSON or SQL, and defining regional BigQuery data policies for masking or raw access. It highlights decoupled governance, regional policy enforcement, and layered security requirements.

🔗 This post introduces the %%bqsql IPython cell magic and the BigFrames library to bridge Python (pandas) and BigQuery SQL within notebooks. It explains setup steps for local or Colab environments, how to enable the BigQuery sandbox, and how to load the bigframes extension. The article walks through a USDA wheat data example showing chained workflows that alternate between Python and BigQuery SQL, plus tips for schema handling and visualization.

🧩 AWS Clean Rooms now supports writing SQL query results to intermediate tables within a collaboration, enabling multi-step analytical workflows between partners. These intermediate tables allow reuse of complex joins and creation of shared ID mapping tables for downstream analyses, all within the collaboration’s privacy boundary. The feature helps reduce costs and improve performance for subsequent analyses such as reach, frequency, and attribution.

🔒 Wearables raise acute privacy concerns for professional athletes because biometric data can directly affect livelihoods. While such data can aid training and injury prevention, access by coaches, teams, or leagues risks misuse in discipline, contract negotiations, and betting markets. Experts warn commercialization could enable gamblers and teams to exploit sensitive signals like sleep or heart rate, and aging or injured players may be most vulnerable. Legal and ethical safeguards remain unresolved.

🚀 Amazon Bedrock Managed Knowledge Base is now generally available as a fully managed retrieval-augmented generation (RAG) service. The offering removes the need to manage vector databases, data pipelines, and retrieval infrastructure by handling ingestion, storage optimization, and advanced retrieval. It supports six native connectors—S3, SharePoint, Confluence, Google Drive, OneDrive, and a web crawler—with automatic syncing and managed vector storage tuned for price-performance. Native integration with Amazon Bedrock AgentCore provides auto-generated permissions and observability for agent deployments.

🔐 AWS Lake Formation now allows reading and writing the underlying Amazon S3 data files for tables registered in the AWS Glue Data Catalog, unifying permissions for SQL and direct file access. It issues temporary, scoped credentials tied to Lake Formation table grants—SELECT for read and SUPER for read/write—and is supported in Amazon EMR 7.13+. You can use Spark or Trino with provided APIs or an open source plugin, and all activity is logged in AWS CloudTrail. This capability is available at no extra charge in supported Regions.

🆕 Amazon Aurora DSQL now supports the PostgreSQL JSONB data type with optional compression. You can create or modify tables to store semi-structured data such as configuration metadata, API parameters, and event logs alongside relational data. With PostgreSQL compression enabled by default, larger JSONB payloads are stored more efficiently to help reduce storage costs. Get started with Aurora DSQL via the AWS Free Tier and check Regional availability in the AWS Region table.

🔒 Children face lasting identity and privacy risks online from school accounts, gaming profiles, apps and devices. These data can be exploited for fraud or synthetic identity creation, often remaining undetected for years. Parents, schools and vendors all share responsibility; practical steps include data minimization, strong passwords, MFA, privacy settings, parental controls and credit freezes.

📈 BigQuery Graph enables building a digital twin of interconnected systems like supply chains and restaurant networks by creating a Graph View over existing tables. The approach maps items, recipes, locations and dependencies into nodes and edges, enabling targeted queries for issues such as recalls, weather disruptions, or procurement leaks. It emphasizes keeping relational data for metrics while using graphs for structure, cleaning keys, and capturing edge properties for richer modeling.

📢 Amazon Simple Email Service (Amazon SES) now supports tenant-level suppression lists to isolate bounces and complaints per tenant. Previously, a single suppression list applied to all tenants in an account, which could cause one tenant's delivery issues to affect others. Administrators can choose suppression scope (TENANT or ACCOUNT) and reason filters (BOUNCE, COMPLAINT, or both). API operations with a TenantName parameter let you view and manage suppressed addresses.

📊 Connected Sheets removes CSV exports and turns Google Sheets into a live, secure interface to BigQuery, enabling business users to analyze petabytes of governed data without SQL. Admins retain security and governance by provisioning table or view access while preventing data alteration from Sheets. End users gain immediate agility using familiar tools like pivot tables, charts, and formulas to analyze billions of rows and create refreshable reports and hybrid models. Connecting requires a Google Workspace account and a billing-enabled Google Cloud project, with connections established either from Sheets or the BigQuery UI.

📊 Cloudflare built Town Lake, a lakehouse-style unified data analytics platform, and Skipper, an AI data agent, to make its vast telemetry and logs queryable and auditable. Town Lake combines Trino, Iceberg on R2, a metadata catalog, PII scanning, access control, and ELT tooling to provide fresh, accurate, and governed data. Skipper lets non-SQL users ask natural-language questions, produce correct SQL-backed answers, and create shareable charts and dashboards.

🔍 Data security posture management (DSPM) tools help security teams locate shadow data across cloud and on-premises systems to reduce data loss risk. These products focus on data discovery, classification, mapping, and risk assessment, complementing CSPM and other security platforms. The market has seen heavy M&A as vendors expand integrations and AI capabilities to address evolving cloud, AI, and hybrid data risks.

🚀 Amazon EMR now supports Apache Spark 4.0.2 across all deployment models, adding ANSI SQL and VARIANT data type support to simplify data pipelines. The release enables fine-grained access control (FGAC) for read and write operations on Lake Formation registered tables and adopts Apache Iceberg v3 for stronger transaction guarantees and lineage tracking. Enhanced streaming capabilities improve stateful processing and monitoring for real-time applications.

🧊 Amazon Redshift now writes directly to Apache Iceberg tables via the auto-mounted awsdatacatalog and supports ALTER TABLE DDL to change schema, partitioning, and table properties. Supported operations include ADD/DROP/ALTER columns, RENAME COLUMN, SET TABLE PROPERTIES, and ADD/DROP/REPLACE PARTITION FIELD to evolve partition strategies and compression settings. Tables modified by Redshift remain interoperable with other Iceberg engines and respect AWS Lake Formation permissions.

🔁 Amazon Aurora DSQL now supports change data capture (CDC) in preview, enabling real-time streaming of database changes directly to Amazon Kinesis Data Streams. The fully managed capability captures insert, update, and delete operations as discrete change events and removes the need to build and maintain custom streaming pipelines. It is designed for zero impact on database throughput and latency, requires no infrastructure setup, and integrates with AWS Lambda and Data Firehose for downstream delivery. CDC streaming in preview is available in all Regions where Aurora DSQL is offered and is billed using Distributed Processing Units (DPUs); standard Amazon Kinesis pricing applies separately.

⚡ Amazon ElastiCache now supports aggregation queries that let developers filter, group, transform, and summarize data directly in cache with a single query. By running aggregations in-memory, applications can deliver microsecond latencies across terabytes of data while returning results that reflect completed writes. The feature reduces architectural complexity by removing the need for a separate analytics engine and enables use cases such as faceted navigation, rollups, leaderboards, and real-time dashboards. Aggregations are available at no additional cost on node-based clusters running Valkey 9.0 in commercial, GovCloud, and China Regions.

🔍 Amazon ElastiCache now supports real-time hybrid search that combines vector similarity and full-text search in a single query, removing the need for a separate search service. New writes are searchable immediately, enabling microsecond latencies and up to 99% recall across billions of embeddings from providers such as Bedrock, SageMaker, Anthropic, and OpenAI. The feature is available at no additional cost for node-based clusters running Valkey 9.0+ in all commercial, GovCloud (US), and China Regions, and Valkey is recommended as a permissive, vendor-neutral alternative to Redis.

🔎 Amazon ElastiCache now provides real-time full-text, exact-match, and numeric range search directly in the cache, eliminating the need for a separate search service. Applications can query terabytes of frequently changing data with microsecond latency and up to millions of operations per second, and can combine search types in a single query to power complex filters. The capability is available in all commercial Regions, AWS GovCloud (US), and China Regions for node-based clusters running Valkey 9.0 at no additional cost; create or upgrade clusters via Console, SDK, or CLI.

🔄 AWS announces General Availability of AWS Entity Resolution incremental ML-based matching workflows. Customers can now process only newly added records rather than reprocessing entire datasets, cutting latency and infrastructure costs. The feature supports up to 50M incremental records against 1B historical records and processes 1M incremental records in under one hour.