All news with #llm security tag

🔍 AI red teaming identifies how deployed AI systems can be manipulated or misused in real operational contexts. It tests the interaction of models with prompts, retrieval, tools, and workflows to produce actionable attack paths rather than isolated examples. This adversarial, continuous approach complements traditional security by focusing on intent, context, policy, and business impact. Teams should inventory systems, threat model by risk, red team early and often, and re-test after changes.

🔍 At Google Cloud, collaboration with customers guided a technical sprint in January 2026 with a major telecommunications partner to refine Model Armor, the runtime security service for generative AI. By embedding with the customer's developers and security teams, the Google Cloud Developer Advocacy group observed real-world workflows and identified friction points such as search-first documentation needs, tuning confidence levels to avoid false positives, clearer enforcement guidance, and IAM-related 403 errors during integrations. The team translated these findings into tested code samples, a confidence level matrix, explicit integration guides for Apigee, Gemini Enterprise Agent Platform, and GKE, and deeper technical documentation to improve operational utility and reduce deployment friction.

🛡️ Researchers at Obsidian Security disclosed a three-bug chain in the open-source LiteLLM proxy that lets a default low-privilege account escalate to full proxy admin and achieve remote code execution. The combined issue, rated CVSS 9.9, exposes provider keys, decryption secrets, prompts, and responses. Maintainer BerriAI published fixes in LiteLLM v1.83.14-stable (May 2); users should upgrade and audit admin roles, guardrails, callbacks, and keys.

🔒 Anthropic said it will "abruptly disable" its latest models, Claude Fable 5 and Mythos 5, for all users after receiving a U.S. government directive to suspend access for foreign nationals due to national security concerns. The company said it believes the order reflects a "misunderstanding" and is working to restore access while noting other models remain available. Anthropic said a demonstrated narrow jailbreak identified minor, publicly discoverable vulnerabilities, and emphasized its safety classifiers and guardrails to limit misuse. The move follows findings that Mythos-class models can rapidly convert disclosed software flaws into working exploits, raising concerns about fast weaponization of vulnerabilities.

📘 Today Google Cloud introduces the Open Knowledge Format (OKF), an open, vendor-neutral specification that formalizes the LLM-wiki pattern into a portable directory of markdown files with YAML frontmatter. OKF v0.1 defines a small set of conventions so different producers’ wikis can be consumed by agents without translation. The spec is intentionally minimal — one required type field per concept — and is accompanied by reference producer and consumer implementations and sample bundles.

🔍 New research finds current AI web agents largely fail to defend against prompt injection attacks. The StakeBench benchmark tested GPT‑5 and Gemini‑powered agents across realistic web scenarios, revealing high success rates for both direct and indirect injections and exposing failure modes like stealthy parasitism and misaligned disruption. Results show vulnerabilities vary by stakeholder and agent architecture.

🔍 Frontier AI releases like Claude Mythos and OpenAI’s GPT-5.5 are accelerating vulnerability discovery, enabling attackers to find and chain exploits far faster than before. Experts warn defenders should assume AI will increase initial compromise rates and shift focus from perfect patching to containment through stronger identity controls, least privilege, and segmentation. While access to Mythos is limited, similar capabilities will be replicated via fine-tuned open models and local tools, compressing reconnaissance and exploit development into minutes and lowering the economic barrier to large-scale campaigns.

🎙️ Smashing Security episode 471 features Graham Cluley with guest James Ball discussing recent AI-related cybersecurity stories. They explore Meta AI mishaps that exposed passwords and an adaptive AI worm developed by University of Toronto researchers. The episode also touches on worms' history, the WannaCry aftermath, and the shifting legal and practical impacts of AI in cyber defense and offense.

🔍 AI red teaming has rapidly matured since Microsoft created its first team in 2019, driven by the arrival of large language models that broke traditional testing methods. Teams must now assess probabilistic behaviors, socio-technical risks, and agentic systems rather than only deterministic software flaws. Organizations are expanding expertise beyond security to include safety, psychology, and domain specialists to evaluate harms like misinformation and operational failures.

🔎 XBOW received early access to Anthos Mythos Preview and ran a structured evaluation across benchmarks, interactive workflows, and live-site integrations. The model excels at reading source code, finding vulnerability candidates, and aiding native-code analysis and reverse engineering. While powerful for generating leads and precise technical analysis, Mythos Preview is less effective at exploit validation and exhibits mixed judgment that benefits from human orchestration.

🧩 Researchers at the University of Toronto built and tested a proof-of-concept self-replicating worm driven by a locally hosted open-weight large language model. In isolated experiments on a deliberately vulnerable 33-host network, the agent identified dozens of vulnerabilities, gained elevated access across most targeted hosts, and autonomously replicated to a majority of the network without using any commercial AI API. The team highlights how runtime reasoning and ingestion of fresh advisories break single-CVE patching assumptions and argues containment must focus on host and network controls rather than vendor API measures.

🔒 OpenAI’s Lockdown Mode aims to reduce AI-enabled data exfiltration by disabling web browsing, image support, Deep Research, Agent Mode, network access from generated code, and file downloads while still permitting manually uploaded file analysis. Experts say the feature is a pragmatic but imperfect mitigation that still allows side-channel exfiltration, complicates governance across multiple AI vendors, and shifts responsibility between providers and enterprise security teams.

🛡️ Ariel Fogel warned at Infosecurity Europe 2026 that prompt injection is an unresolved architectural issue threatening AI development. He explained that LLMs treat inputs as a single token stream, preventing reliable privilege separation between system prompts, user inputs and agent-retrieved content. With agents gaining tool access, successful injections can escalate from bad outputs to real-world actions, outpacing traditional governance and controls.

🔍 Amazon OpenSearch Serverless now supports Agentic Search, enabling natural-language queries over users' data. The system interprets intent, plans searches, generates DSL queries, and returns results with transparent reasoning. A built-in QueryPlanningTool powered by LLMs translates requests and orchestrates retrieval; behavior can be customized via APIs or OpenSearch Dashboards. Agentic Search is available in all AWS Commercial Regions where OpenSearch Serverless operates.

🔒 Researchers have demonstrated a prototype AI-powered internet worm that autonomously propagates and carries its own local LLM to run on compromised machines. The prototype echoes early theoretical concepts of self-replicating code and shows how generative models can be embedded into malware to extend functionality. This proof-of-concept highlights evolving threats and the need for updated defensive strategies and policy responses.

🔍 Analysis by Halcyon shows a rapid rise in AI-based tools sold across Telegram channels, dark web forums, and underground markets, with posts increasing from 38 in December 2025 to 1,486 by February 2026. The offerings fall into four groups: weaponized LLMs, AI-enabled identity fraud, AI-augmented malware/infrastructure, and jailbroken or stolen AI services. Ransomware operations are professionalising with tiered services, automation and freemium models, lowering the skill barrier for new actors while law enforcement takedowns and better enterprise defenses remain critical.

🔍 At Infosecurity Europe 2026, Bugcrowd unveiled ExploitBench, a graded benchmark assessing AI models' ability to chain vulnerability discovery into staged exploits against a vulnerable V8 build. Anthropic’s Claude Mythos outperformed OpenAI’s GPT‑5.5 in head‑to‑head runs, achieving higher average scores and more top‑tier exploits, often with occasional human nudges. The report highlights rising offensive potential of frontier LLMs and urges defenders to adopt automated remediation and prioritization.

🛡️ A threat actor used an AI-assisted ransomware toolkit to automate Active Directory discovery and iterate EDR evasion techniques. Researchers found Cursor and Claude Opus agents used for coding, analysis, testing, and checking public research for bypass methods, with some malware tested against Sophos, CrowdStrike, and Microsoft EDR products. Sophos determined the workflow was human-directed, while AI accelerated development, producing numerous payload modules and mapping techniques to MITRE ATT&CK.

🧩Trustpilot built a high-volume streaming pipeline using fine-tuned Gemma models to process millions of user reviews in near real-time under tight latency and cost constraints. The team replaced variable per-token pricing with fixed infrastructure costs, fine-tuned lightweight models for tasks like NER, sentiment, and topic classification, and separated classifier and LLM endpoints. Performance tuning, vLLM optimizations, and load testing enabled scalable inference despite challenges with private networking, deployment observability, and GPU availability.

🚀 Amazon Bedrock now supports OpenAI GPT-5.5, GPT-5.4, and Codex for production use, offering the same AWS security, governance, and operational controls. GPT-5.5 delivers advanced capabilities for agentic coding, data analysis, and multi-step autonomous tasks on a next-generation inference engine. Codex is available via a dedicated App, CLI, and IDE integrations for Visual Studio Code, JetBrains, and Xcode, and can be configured to run through Bedrock with pricing aligned to OpenAI first-party rates.