All news with #ai red teaming tag

🛡️ AI-assisted research uncovered a years-old use-after-free race in the Linux kernel's net/sched code that permits local privilege escalation to root (CVE-2026-53264). The bug arises from mismatched locking where an entry can be freed before an RCU grace period ends, creating a window for the kernel to access freed memory. The flaw was found by Lee Jia Jie of STAR Labs, who used AI to locate and reliably reproduce the race; a patch defers freeing until after the grace period. Distributions should apply upstream fixes via normal security channels.

🛡️ CREST has introduced optional AI-Enabled Penetration Testing requirements as an add-on to its existing Penetration Testing Accreditation Standard. Launched on July 28, the module lets providers that integrate AI undergo independent assessment to demonstrate responsible AI governance to clients and regulators. Applications are open to existing CREST members and accredited service providers seeking extra assurance for AI use.

🛡️ Microsoft announces the External Red Team Alliance (EXTRA) to broaden AI safety testing by funding and coordinating external academic and operational expertise across six continents. The initiative provides unrestricted gifts to 18 university labs and builds a distributed network of specialists to address multilingual, domain-specific, and regional AI risks. EXTRA aims to advance evaluation methodologies and strengthen collaboration between academia, practitioners, and industry to better identify and mitigate emerging threats in frontier AI systems.

🔍 AWS today announced a research preview of aws-bench, an open-source benchmark designed to measure how accurately and efficiently AI agents complete real-world AWS tasks. The suite includes test cases derived from actual AWS usage—such as investigation, troubleshooting, and infrastructure creation—pairing natural-language queries with defined resource states and ground-truth answers. A CLI tool is included to instantiate test environments, run evaluations, score results, and reset state, and the project is available on GitHub.

🛡️ CodeMender is a managed code security agent now available in preview, offering automated scanning and remediation using Google DeepMind–tuned models via the Gemini Enterprise Agent Platform or as part of AI Threat Defense. It prioritizes fixes by exploitability, runs proof-of-concept exploits in customer-managed sandboxes, and generates validated patches that integrate into developer workflows. The agent supports multiple languages, integrates with CI/CD and IDEs, and enforces enterprise-grade governance and data controls.

🧩 Rapid7 found an exposed delivery server containing 1,048 files: lure templates, tests, droppers, builder notes, and two campaign chains. One campaign targeted Windows users in Mexico via a fake government ID lookup and delivered an infostealer through a WebDAV-hosted exploit. The artifacts included README notes, test matrices, and logs that indicate the operator used generative AI (an open-source coding agent) to create, test, and document phishing delivery at scale. The kit heavily probed a WebDAV working-directory hijack (CVE-2025-33053) and contained tests for other file-handling flaws, while active delivery logs showed thousands of launch events concentrated in Mexico.

🛡️ OpenAI revealed GPT‑Red, an internal automated red‑teaming model that simulates human adversaries to discover prompt injection vulnerabilities at scale. GPT‑Red iteratively probes and refines attacks against production models, helping harden GPT‑5.6 Sol and reduce prompt injection failures by 6× versus GPT‑5.5. OpenAI uses self‑play reinforcement learning to train both attacker and defender models while keeping GPT‑Red segregated to avoid misuse.

🔍 AI security cannot be treated as a one-time certification; it requires an ongoing cycle of adversarial discovery, hardening, and operational resilience. NIST research shows no finite set of guardrails can guarantee permanent robustness, so teams must continuously test, remediate, and monitor systems as models, prompts, and integrations evolve. Effective programs tie red teaming to runtime protection and governance so findings become durable improvements.

🔎 This article from Google Data Cloud explores a rigorous, information‑theoretic approach to evaluating AI data agents by converting binary pass/fail tests into graded difficulty sweeps. The team introduces Discovery Bench and iterative surprisal-based query refinement (iSQR) to generate low/medium/high ambiguity variations of queries, quantify surprisal, and map where agents succeed or fail. The piece highlights how this method reveals cliffs and sweet spots in agent behavior and calls for auditing benchmarks themselves to avoid misleading conclusions.

🛡️ Security vendor CrowdStrike has added five new prompt injection techniques to its taxonomy that threaten enterprise AI deployments. These attacks manipulate LLM behavior by embedding deceptive instructions into inputs, context, or token streams to bypass safety controls and produce malicious outputs. CrowdStrike recommends threat modeling input sources, expanding testing, and enhancing detection engineering to defend against composite and multi-stage prompt attacks.

🔒 The UK National Cyber Security Centre (NCSC) has launched the Cyber Shield initiative to build a national cyber-defence capability powered by agentic AI. The project will use coordinated red and blue agents to discover and mitigate vulnerabilities at scale, enable national automated scanning, and support real-time intelligence sharing. The NCSC says success requires partnerships with government, critical infrastructure and frontier AI providers.

🔒 Fortinet describes its integration of frontier AI models (Anthropic’s Glasswing/Mythos and OpenAI’s Daybreak/GPT 5.5 Cyber) alongside on-premises models to scale security testing across firmware, source code, and penetration testing. The company emphasizes responsible innovation, mature vulnerability management, and human validation of AI findings. Fortinet reports limited exploitable firmware issues but greater findings from source-code analysis and commits to mitigation, virtual patching, and secure-by-default deployments.

🛡️ Researchers uncovered a macOS malware family named macOS.Gaslight that embeds fabricated error messages and debugging data inside a Rust binary to mislead AI-assisted analysis tools. The 3.5 KB payload contains 38 fake system messages — including memory dumps, token-expiration warnings, and build errors — designed to appear as legitimate developer logs. SentinelOne attributes the sample with high confidence to a North Korean-linked actor and notes the strings aim to prompt-inject LLM pipelines, causing them to abort or distrust their session. The malware retains standard backdoor and data-stealing capabilities alongside the deceptive messaging tactic.

🛡️ The Cobalt State of Pentesting Report 2026 surveyed roughly 450 cybersecurity professionals across 2025 and 2026 and found trust in fully automated AI vulnerability testing has dropped sharply. Reliance on AI-only testing fell from 29% to 9%, while 47% now prefer a hybrid human-plus-AI model. Respondents reported that 78% of fully automated scanners missed critical vulnerabilities, and AI/LLM issues showed longer MTTR and lower fix rates.

🛡️ Enterprise AI red teaming must go beyond simple prompt tests to examine full systems — models, prompts, retrievals, tools, permissions, workflows, and APIs — because risks appear when components interact. Check Point argues that threat intelligence plus threat modeling enables identification of realistic attack paths, evidence-based findings, and prioritized remediation. Continuous, comparative testing and re-testing after changes ensures fixes are effective and keeps pace with rapid AI adoption.

🔒 AWS has introduced Continuum, a service to continuously discover, investigate, and remediate vulnerabilities across first-party and third-party codebases. The platform uses AI to validate exploitability, generate remediation recommendations, and propose fixes that integrate with existing development workflows. New capabilities include automatic threat modeling in STRIDE format, while more established features derive from the Security Agent product. Continuum supports graduated trust from human-in-the-loop review to an "enforce mode" for autonomous remediation.

🔍 AI red teaming identifies how deployed AI systems can be manipulated or misused in real operational contexts. It tests the interaction of models with prompts, retrieval, tools, and workflows to produce actionable attack paths rather than isolated examples. This adversarial, continuous approach complements traditional security by focusing on intent, context, policy, and business impact. Teams should inventory systems, threat model by risk, red team early and often, and re-test after changes.

🔍 At Google Cloud, collaboration with customers guided a technical sprint in January 2026 with a major telecommunications partner to refine Model Armor, the runtime security service for generative AI. By embedding with the customer's developers and security teams, the Google Cloud Developer Advocacy group observed real-world workflows and identified friction points such as search-first documentation needs, tuning confidence levels to avoid false positives, clearer enforcement guidance, and IAM-related 403 errors during integrations. The team translated these findings into tested code samples, a confidence level matrix, explicit integration guides for Apigee, Gemini Enterprise Agent Platform, and GKE, and deeper technical documentation to improve operational utility and reduce deployment friction.

🔎 The UK government's GC3 ran weekly in-person hackathons using frontier AI models to scan public code repositories across nine departments, identifying 407 findings including authentication bypasses, data exposure and remote code execution. Teams built diverse pipelines combining models and traditional tools like Gitleaks, Trivy and Semgrep, and all exploitable critical and high-risk issues were remediated. The initiative highlighted the benefits of tightly scoped model components, the need for human triage, and cost-effective scanning, though export restrictions on some models may affect future work.

🔍 AI red teaming has rapidly matured since Microsoft created its first team in 2019, driven by the arrival of large language models that broke traditional testing methods. Teams must now assess probabilistic behaviors, socio-technical risks, and agentic systems rather than only deterministic software flaws. Organizations are expanding expertise beyond security to include safety, psychology, and domain specialists to evaluate harms like misinformation and operational failures.