< ciso
brief />
Tag Banner

All news with #ai red teaming tag

161 articles · page 5 of 9

Customer-driven improvements to GenAI security

🔍 At Google Cloud, collaboration with customers guided a technical sprint in January 2026 with a major telecommunications partner to refine Model Armor, the runtime security service for generative AI. By embedding with the customer's developers and security teams, the Google Cloud Developer Advocacy group observed real-world workflows and identified friction points such as search-first documentation needs, tuning confidence levels to avoid false positives, clearer enforcement guidance, and IAM-related 403 errors during integrations. The team translated these findings into tested code samples, a confidence level matrix, explicit integration guides for Apigee, Gemini Enterprise Agent Platform, and GKE, and deeper technical documentation to improve operational utility and reduce deployment friction.
read more →

UK government patches 400+ vulnerabilities via AI

🔎 The UK government's GC3 ran weekly in-person hackathons using frontier AI models to scan public code repositories across nine departments, identifying 407 findings including authentication bypasses, data exposure and remote code execution. Teams built diverse pipelines combining models and traditional tools like Gitleaks, Trivy and Semgrep, and all exploitable critical and high-risk issues were remediated. The initiative highlighted the benefits of tightly scoped model components, the need for human triage, and cost-effective scanning, though export restrictions on some models may affect future work.
read more →

AI Red Teaming Evolves into Core Security Practice

🔍 AI red teaming has rapidly matured since Microsoft created its first team in 2019, driven by the arrival of large language models that broke traditional testing methods. Teams must now assess probabilistic behaviors, socio-technical risks, and agentic systems rather than only deterministic software flaws. Organizations are expanding expertise beyond security to include safety, psychology, and domain specialists to evaluate harms like misinformation and operational failures.
read more →

XBOW Evaluates Anthropic’s Mythos Preview Model

🔎 XBOW received early access to Anthos Mythos Preview and ran a structured evaluation across benchmarks, interactive workflows, and live-site integrations. The model excels at reading source code, finding vulnerability candidates, and aiding native-code analysis and reverse engineering. While powerful for generating leads and precise technical analysis, Mythos Preview is less effective at exploit validation and exhibits mixed judgment that benefits from human orchestration.
read more →

AI Agent Uncovers 21 FFmpeg Zero-Days, Chrome Ships 429 Fixes

🛡️ depthfirst's autonomous agent discovered 21 previously unknown zero-day vulnerabilities in FFmpeg, producing reproducible PoC inputs for each at a reported cost of about $1,000 for the run. In the same week, Google released Chrome 149 with fixes for a record 429 security bugs, over 100 of which are critical or high severity, following an overhaul of its bounty program to cope with a surge of AI-generated reports. The findings illustrate how AI is accelerating vulnerability discovery and increasing pressure on triage and patching processes across widely used software.
read more →

Lloyds’ Practical Playbook for Agentic AI Security

🛡️ Lloyds Banking Group treats agentic AI as an engineering problem to be designed, constrained and tested at scale. At OWASP’s GenAI Security Summit, Lloyds’ security leads explained an “AI safe adoption” strategy spanning lifecycle governance, an internal agent marketplace, and multidisciplinary feature teams. Key challenges include agent identity, runtime observability and automated red‑teaming, while prioritizing low‑risk, high‑value use cases for customers.
read more →

Updated Taxonomy of Agentic AI Failure Modes

🔎 The Microsoft AI Red Team released a v2.0 update to the Taxonomy of Failure Modes in Agentic AI Systems, grounded in twelve months of red team engagements and operational data. The revision adds seven new failure mode categories—such as agentic supply chain compromise, goal hijacking, and visual attacks against computer-use agents—expands mitigations, and emphasizes supply chain, zero‑trust, and session hardening.
read more →

Benchmark Shows Mythos Outperforms GPT‑5.5 on Chrome Exploits

🔍 At Infosecurity Europe 2026, Bugcrowd unveiled ExploitBench, a graded benchmark assessing AI models' ability to chain vulnerability discovery into staged exploits against a vulnerable V8 build. Anthropic’s Claude Mythos outperformed OpenAI’s GPT‑5.5 in head‑to‑head runs, achieving higher average scores and more top‑tier exploits, often with occasional human nudges. The report highlights rising offensive potential of frontier LLMs and urges defenders to adopt automated remediation and prioritization.
read more →

Anthropic expands Glasswing access to 150 partners

🛡️ Anthropic has broadened Project Glasswing, giving 150 additional organizations access to its most capable model, Claude Mythos Preview, to help find vulnerabilities in critical software. The program, first opened to roughly 50 partners in April, claims more than 10,000 high- or critical-severity flaws discovered to date. New participants span 15+ countries and underrepresented sectors like power, water, healthcare and hardware, chosen for the potential catastrophic impact of breaches. Anthropic warned that while discovery is accelerating, safe public release of Mythos-class models remains restricted due to incomplete safeguards.
read more →

AI-Driven Cybercrime Tools Surge Over 3800%

🔍 Halcyon research reveals a dramatic rise in AI-powered cybercrime tooling across underground markets, jumping from 38 mentions in December to 1,486 in February. Cynthia Kaiser, SVP of Halcyon’s Ransomware Research Center, detailed four product categories: weaponized LLMs, AI-enabled identity fraud, AI-augmented malware/infrastructure, and jailbroken or stolen AI services. She warned that automated distribution, freemium models and redundant channels have lowered the financial barrier to entry and increased resilience against takedown efforts.
read more →

Frontier AI models more vulnerable under iterative attacks

🔍 Cisco researchers found that popular frontier LLMs from OpenAI, Anthropic, Google, xAI, and Amazon exhibit substantially higher risk when subjected to multi-turn adversarial attacks than when assessed with single-prompt safety benchmarks. The team ran tens of thousands of single-turn and multi-turn attacks across 15 models and multiple configurations, revealing wide gaps in attack success rates (ASRs) and configuration-dependent safety behavior. They urge improved benchmarks, transparency on configuration impacts, and publication of paired single- and multi-turn ASRs to better inform procurement and governance decisions.
read more →

Microsoft Open-Sources Rampart and Clarity for AI Safety

🔒 Microsoft has open-sourced two tools, Rampart and Clarity, intended to embed safety engineering into the AI agent development lifecycle rather than leaving it as a periodic checkpoint. Rampart converts red-team findings into structured, repeatable tests that can be automated in CI/CD pipelines and is built on top of PyRIT for continuous adversarial and benign scenario execution. Clarity targets an earlier phase, guiding engineers through structured conversations to clarify assumptions, expected behaviors, permissions and trust boundaries, storing outcomes as markdown in a .clarity-protocol/ directory for review. Both projects join Microsoft’s broader open-source agent governance stack to address risks such as prompt injection, unsafe tool use, privilege escalation, and unintended autonomous actions.
read more →

Microsoft Open-Sources RAMPART and Clarity for AI

🛡️ Microsoft has released two open-source tools, RAMPART and Clarity, to help developers test and clarify AI agent safety early in the development lifecycle. RAMPART is a Pytest-native framework for writing and running adversarial and benign safety tests against agents, building on prior work such as PyRIT. It evaluates test outcomes via simple adapters that connect an agent to the suite, while Clarity acts as a structured thinking partner to surface assumptions, explore failure modes, and guide design decisions before coding begins.
read more →

RAMPART and Clarity: Open Tools for Agent Safety Workflow

🔒 Microsoft has open-sourced two engineering tools—RAMPART and Clarity—to make agent safety a continuous part of development. RAMPART provides a pytest-style framework that brings red-team and adversarial tests into CI, evaluating tools invoked and side effects. Clarity is a structured design companion that captures problem statements, failure analyses, and decisions in a .clarity-protocol directory. Both aim to create living safety artifacts integrated into normal workflows.
read more →

Agentic AI Drives Surge in Mobile App Cyberattacks

📈 Digital.ai's 2026 Application Security Threat Report found that 87% of monitored customer-facing apps were attacked in 2026, up sharply from 55% in 2022. The firm says agentic AI has lowered the skill and time required for threat actors to inspect code, generate exploits and adapt malware. Financial services, automotive and medical device apps were most targeted, and iOS attacks have nearly closed the gap with Android.
read more →

When China's AI Catches Up: Mythos and Global Risks

🔒 Anthropic's Mythos Preview, shared last month with a limited set of security partners, has demonstrated the ability to autonomously find zero-day vulnerabilities across major operating systems and browsers. Anthropic paired the release with Project Glasswing and $100 million in usage credits to help defenders, but reports of unauthorized access and denied requests from Chinese entities have already emerged. The development challenges the assumption of a durable US lead and has injected cybersecurity into high-level US–China summit talks, prompting urgent questions about access, regulation, and international cooperation.
read more →

Pen Tests Reveal AI Flaws More Severe Than Legacy Bugs

🔒 Penetration testing shows AI and LLM deployments contain a disproportionate share of severe vulnerabilities. Cobalt’s State of Pentesting Report finds 32% of LLM findings rated high risk versus 13% for legacy enterprise tests, and only 38% of those high-risk LLM issues are remediated. Experts point to emerging attack surfaces — notably prompt injection, now OWASP’s top LLM risk — broader blast radii from model integrations, and fragmented ownership for fixes. Recommended countermeasures include threat modeling, red teaming, least-privilege access, strict output validation, and human approval gates for high-consequence actions.
read more →

Nutanix and Palo Alto Networks: Integration for Model Trust

🔒 Nutanix and Palo Alto Networks have integrated Prisma AIRS into the Nutanix Enterprise AI platform to embed automated AI model scanning and continuous red teaming directly into the MLOps pipeline. The integrated solution scans models at check-in, analyzes dependencies for known vulnerabilities and license issues, and validates provenance and file formats to block backdoors or unsafe execution paths before deployment. It also provides API-driven red teaming with a context-aware agent and a large, continuously updated attack library so teams can test resilience and prioritize business-relevant risks without complex setup.
read more →

Defending Against Attacks from Frontier AI Models: Readiness

🔒 A new generation of frontier AI models is changing how cyberattacks are developed, enabling speed, scale, and accessibility previously unseen. Early testing of advanced models, including Claude’s Mythos, shows they can identify code vulnerabilities, map attack paths, and generate working exploits with minimal effort. Organizations must treat these as fully AI-powered attacks and prioritize proactive readiness, detection, and mitigation strategies.
read more →

AI-Driven Vulnerability Discovery and Defensive Response

🤖 In the latest Adversary Universe podcast, CrowdStrike leaders discuss how AI is accelerating vulnerability discovery and could produce a rapid surge of new flaws — a potential 'vuln-pocalypse'. They urge prioritizing remediation based on active exploitation and prevalence in environments. CrowdStrike recommends leveraging AI for agentic red teaming, vulnerability scanning, and crowdsourced telemetry to detect post-exploitation behaviors. They point to Project Glasswing and OpenAI's Trusted Access for Cyber as examples of defense-focused collaboration.
read more →