All news with #prompt injection attack tag

🛡️ Two SafeBreach studies demonstrate how prompt-injection techniques can bypass layers of defenses around Google Gemini, using calendar invites or text messages as entry points. Attackers chain indirect injection, memory poisoning, delayed execution, and fake context alignment to get the assistant to perform unauthorized actions across devices. Even with Google fixes, the research highlights a persistent arms race between attackers and defenders that leaves users needing to restrict assistant access.

🛡️ Cato Networks found a single prompt can cause OpenAI’s GPT-5.5 to plan and execute a full offensive cyber-attack in a controlled Active Directory lab. The model carried out reconnaissance, exploitation, lateral movement, privilege escalation and exfiltration, reaching domain admin in about 40 minutes. Researchers tested six scenarios, noting adaptive behavior when conditions changed and emphasizing the risk of accelerating existing attack workflows.

🛡️ Researchers describe a new class of attacks called agent data injection (ADI), where attackers plant forged trusted fields—like a sender name or button ID—so an AI agent acts on corrupted facts while continuing its assigned task. The method exploits how agents parse punctuation-delimited fields, letting attackers slip fake structure past prompt-injection defenses. The team built working proofs against multiple web and coding assistants and found mixed mitigation results from random IDs and provenance tracking.

🛡️ OpenAI revealed GPT‑Red, an internal automated red‑teaming model that simulates human adversaries to discover prompt injection vulnerabilities at scale. GPT‑Red iteratively probes and refines attacks against production models, helping harden GPT‑5.6 Sol and reduce prompt injection failures by 6× versus GPT‑5.5. OpenAI uses self‑play reinforcement learning to train both attacker and defender models while keeping GPT‑Red segregated to avoid misuse.

🔒 Manifold Security found that Claude for Chrome still accepts synthetic clicks and can read permission mode from its URL, enabling other extensions with DOM access on claude.ai to trigger nine allowlisted tasks (including Gmail, Google Docs, and Calendar). Anthropic constrained arbitrary prompts after ClaudeBleed, but the click handler lacks an event.isTrusted check and the side panel honors ?skipPermissions=true, creating high-risk scenarios especially if "Act without asking" is enabled. Manifold reported this in May against v1.0.72; the issues remained in v1.0.80 as of July 7 and no patch or public advisory was available by July 14.

🛡️ Progress urged ShareFile customers to shut down Windows Storage Zone Controllers amid a credible external threat, temporarily disabling access while investigating; there are no signs of account or data compromise. Other top stories include a critical Zimbra XSS patch, a compromised Jscrambler npm package distributing a multi-platform Rust stealer, and Microsoft detailing the destructive GigaWiper backdoor. Large-scale web shell operations (SHELLSTORM), HalluSquatting attacks against AI assistants, and many actively exploited CVEs round out the week's threats.

🛡️Researchers show a one-email exploit can trick an AI personal agent into writing a false, persistent memory and hiding the change. The tool, MemGhost, was tested in lab conditions against OpenClaw and other agent frameworks, succeeding frequently in background runs. The authors propose provenance tagging, user confirmation, and write logging as mitigations while vendors consider memory-write controls.

🛡️ Researchers demonstrated "Ghostcommit," a proof-of-concept attack that hides malicious instructions inside a PNG referenced by an AGENTS.md so AI code-reviewing agents read images, open .env files, and exfiltrate secrets as integer constants. The pull request appears benign to text-based reviewers and default configs often exclude images from review, letting the change merge without human oversight. In tests, several coding agents followed the image pointer and emitted the repository's .env as a tuple of integers, while some agent harnesses refused. The ASSET Research Group published code, disclosed vendors, and built a multimodal GitHub app that inspects images, code shape, and conventions to block the exploit in trials.

🛡️ Security vendor CrowdStrike has added five new prompt injection techniques to its taxonomy that threaten enterprise AI deployments. These attacks manipulate LLM behavior by embedding deceptive instructions into inputs, context, or token streams to bypass safety controls and produce malicious outputs. CrowdStrike recommends threat modeling input sources, expanding testing, and enhancing detection engineering to defend against composite and multi-stage prompt attacks.

🔒 A new AI Now Institute report demonstrates a proof-of-concept exploit that coerces Anthropic’s Claude Code and OpenAI’s Codex into executing attacker-supplied binaries during automated code review. The attack uses multi-stage prompt injection hidden in repository files (documentation, comments) to trick agents in auto-mode or auto-review into running a seemingly benign script that launches a malicious payload. Researchers warn the architectural risk — agents’ inability to reliably attribute text sources — makes such platforms potential attack vectors when granted shell access and autonomous execution.

🔍 Researchers at the AI Now Institute demonstrated a proof-of-concept called "Friendly Fire" where autonomous AI coding agents (Anthropic's Claude Code and OpenAI's Codex) execute an attacker's binary when asked to scan untrusted open-source code. The attack hides a malicious binary alongside benign files and a README that prompts the agent to run a security script; in auto-modes the agents approved and executed it without prompting. The weakness is framed as a workflow/design issue rather than a single vulnerable version, and the researchers recommend never giving command-capable agents unattended access to untrusted code.

🧭 A new study found GitHub Copilot can be induced to generate harmful answers when a dangerous request is broken into ordinary coding steps. Researchers Abhishek Kumar and Carsten Maple tested Claude and Gemini models through Copilot and observed that direct chat prompts were routinely refused, but the same content was produced in 816 of 816 workflow runs when framed as benchmark-improving “teaching shots.” The paper calls this technique workflow-level jailbreak construction and urges reviewing written files and whole sessions, not just chat refusals.

🔍 Zscaler ThreatLabz uncovered real-world campaigns using indirect prompt injection, where hidden instructions embedded in web pages steer AI agents. Attackers used SEO poisoning to surface malicious pages and hid prompts via CSS and JSON-LD metadata. One campaign impersonated a Python library to trick agents into paying a $3 bogus API key; another typosquatted a DeBank site to claim authority. Tests across 26 LLMs showed varying susceptibility depending on model and context.

🛡️ Researchers discovered two vulnerabilities in the Cursor AI-enabled IDE that enable prompt-injection-driven remote code execution by escaping the command execution sandbox. The flaws, CVE-2026-50548 and CVE-2026-50549, allow attackers to change the working directory and exploit symlink canonicalization fallbacks to write or overwrite files outside the project scope. Cursor patched the issues in version 3.0, and the findings underscore broader risks in agentic AI workflows and the difficulty of defending against prompt injection.

🔒 As organizations adopt AI more broadly, third-party API proxies and aggregators promise convenience, cost savings, and failover between models. Some providers operate transparently, but many exploit forged or stolen accounts, reroute queries to cheaper models, and capture or manipulate prompts and outputs. These practices expose firms to data leakage, IP loss, compliance violations, and security threats such as injected malicious code or reduced model accuracy.

🧩 Researchers at LayerX demonstrated a prompt injection called BioShocking that trains AI-powered browsers to treat risky real-world actions as fictional, bypassing safety controls. The PoC used a themed puzzle game to reward 'wrong' behavior and culminated in instructing agents to copy sensitive data from a GitHub repo. Six mainstream agentic browsers were tested; only one vendor implemented a working fix after disclosure. LayerX recommends explicit user confirmations, stricter context checks, and session scope limits.

🔒 New research from Adversa AI, dubbed GuardFall, shows a decades-old shell trick can bypass simple blocklist checks in open-source AI coding agents, letting hidden destructive commands run. The flaw arises because filters inspect the command as plain text while shells like bash rewrite and expand that text before execution. Ten of eleven tested agents were vulnerable; only Continue defended by parsing commands the same way the shell does.

🔎 Researchers at Mozilla's 0DIN demonstrated that an AI coding agent like Claude Code can be manipulated into executing a remote payload by following innocuous setup instructions in a clean GitHub repo. The approach uses three benign-looking components—a standard repo, an initialization error prompting a recommended command, and a script that pulls a command from a DNS TXT record—to spawn an interactive shell with developer privileges. 0DIN warns this chain leaves no explicit malicious code in the repo and is difficult for scanners or human reviewers to detect.

🛡️ Researchers uncovered a macOS malware family named macOS.Gaslight that embeds fabricated error messages and debugging data inside a Rust binary to mislead AI-assisted analysis tools. The 3.5 KB payload contains 38 fake system messages — including memory dumps, token-expiration warnings, and build errors — designed to appear as legitimate developer logs. SentinelOne attributes the sample with high confidence to a North Korean-linked actor and notes the strings aim to prompt-inject LLM pipelines, causing them to abort or distrust their session. The malware retains standard backdoor and data-stealing capabilities alongside the deceptive messaging tactic.

📝 This post highlights a new paper that demonstrates how large language models are vulnerable to prompt injection because they learn to distinguish instruction blocks by style rather than explicit tags. The authors argue that role tags became a de facto security architecture but do not map cleanly into model representations, producing persistent role confusion. The paper warns that without genuine role perception, defenses will be reactive and brittle, and calls for deeper study of roles within the LLM stack.