All news with #prompt injection tag

Prompt Hijacking Risks MCP-Based AI Workflows Exposed

⚠️ Security researchers warn that MCP-based AI workflows are vulnerable to "prompt hijacking" when MCP servers issue predictable or reused session IDs, allowing attackers to inject malicious prompts into active client sessions. JFrog demonstrated the issue in oatpp-mcp (CVE-2025-6515), where guessable session IDs could be harvested and reassigned to craft poisoned responses. Recommended mitigations include generating session IDs with cryptographically secure RNGs (≥128 bits of entropy) and having clients validate unpredictable event IDs.

Model Armor and Apigee: Protecting Generative AI Apps

🔒 Google Cloud’s Model Armor integrates with Apigee to screen prompts, responses, and agent interactions, helping organizations mitigate prompt injection, jailbreaks, sensitive data exposure, malicious links, and harmful content. The model‑agnostic, cloud‑agnostic service supports REST APIs and inline integrations with Apigee, Vertex AI, Agentspace, and network service extensions. The article provides step‑by‑step setup: enable the API, create templates, assign service account roles, add SanitizeUserPrompt and SanitizeModelResponse policies to Apigee proxies, and review findings in the AI Protection dashboard.

Four Bottlenecks Slowing Enterprise GenAI Adoption

🔒 Since ChatGPT’s 2022 debut, enterprises have rapidly launched GenAI pilots but struggle to convert experimentation into measurable value — only 3 of 37 pilots succeed. The article identifies four critical bottlenecks: security & data privacy, observability, evaluation & migration readiness, and secure business integration. It recommends targeted controls such as confidential compute, fine‑grained agent permissions, distributed tracing and replay environments, continuous evaluation pipelines and dual‑run migrations, plus policy‑aware integrations and impact analytics to move pilots into reliable production.

Securing AI in Defense: Trust, Identity, and Controls

🔐 AI promises stronger cyber defense but expands the attack surface if not governed properly. Organizations must secure models, data pipelines, and agentic systems with the same rigor applied to critical infrastructure. Identity is central: treat every model or autonomous agent as a first‑class identity with scoped credentials, strong authentication, and end‑to‑end audit logging. Adopt layered controls for access, data, deployment, inference, monitoring, and model integrity to mitigate threats such as prompt injection, model poisoning, and credential leakage.

Agentic AI and the OODA Loop: The Integrity Problem

🛡️ Bruce Schneier and Barath Raghavan argue that agentic AIs run repeated OODA loops—Observe, Orient, Decide, Act—over web-scale, adversarial inputs, and that current architectures lack the integrity controls to handle untrusted observations. They show how prompt injection, dataset poisoning, stateful cache contamination, and tool-call vectors (e.g., MCP) let attackers embed malicious control into ordinary inputs. The essay warns that fixing hallucinations is insufficient: we need architectural integrity—semantic verification, privilege separation, and new trust boundaries—rather than surface patches.

Preparing for AI, Quantum and Other Emerging Risks

🔐 Cybersecurity must evolve to meet rapid advances in agentic AI, quantum computing, low-code platforms and proliferating IoT endpoints. The author argues organizations should move from static defenses to adaptive, platform-based security that uses automation, continuous monitoring and AI-native protection to match attackers' speed. He urges early planning for post-quantum cryptography and closer collaboration with partners so security enables — rather than hinders — innovation.

Microsoft: 100 Trillion Signals Daily as AI Fuels Risk

🛡️ The Microsoft Digital Defense Report 2025 reveals Microsoft systems analyze more than 100 trillion security signals every day and warns that AI now underpins both defense and attack. The report describes adversaries using generative AI to automate phishing, scale social engineering and discover vulnerabilities faster, while autonomous malware adapts tactics in real time. Identity compromise is the leading vector—phishing and social engineering caused 28% of breaches—and although MFA blocks over 99% of unauthorized access attempts, adoption remains uneven. Microsoft urges board-level attention, phishing-resistant MFA, cloud workload mapping and monitoring, intelligence sharing and immediate AI and quantum risk planning.

Encoding-Based Attack Protection with Bedrock Guardrails

🔒 Amazon Bedrock Guardrails offers configurable, cross-model safeguards to protect generative AI applications from encoding-based attacks that attempt to hide harmful content using encodings such as Base64, hexadecimal, ROT13, and Morse code. It implements a layered defense—output-focused filtering, prompt-attack detection, and customizable denied topics—so legitimate encoded inputs are allowed while attempts to request or generate encoded harmful outputs are blocked. The design emphasizes usability and performance by avoiding exhaustive input decoding and relying on post-generation evaluation.

MAESTRO Framework: Securing Generative and Agentic AI

🔒 MAESTRO, introduced by the Cloud Security Alliance in 2025, is a layered framework to secure generative and agentic AI in regulated environments such as banking. It defines seven interdependent layers—from Foundation Models to the Agent Ecosystem—and prescribes minimum viable controls, operational responsibilities and observability practices to mitigate systemic risks. MAESTRO is intended to complement existing standards like MITRE, OWASP, NIST and ISO while focusing on outcomes and cross-agent interactions.

AI-aided malvertising: Chatbot prompt-injection scams

🔍 Cybercriminals have abused X's AI assistant Grok to amplify phishing links hidden in paid video posts, a tactic researchers have dubbed 'Grokking.' Attackers embed malicious URLs in video metadata and then prompt the bot to identify the video's source, causing it to repost the link from a trusted account. The technique bypasses ad platform link restrictions and can reach massive audiences, boosting SEO and domain reputation. Treat outputs from public AI tools as untrusted and verify links before clicking.

Security Risks of Vibe Coding and LLM Developer Assistants

🛡️AI developer assistants accelerate coding but introduce significant security risks across generated code, configurations, and development tools. Studies show models now compile code far more often yet still produce many OWASP- and MITRE-class vulnerabilities, and real incidents (for example Tea, Enrichlead, and the Nx compromise) highlight practical consequences. Effective defenses include automated SAST, security-aware system prompts, human code review, strict agent access controls, and developer training.

Indirect Prompt Injection Poisons Agents' Long-Term Memory

⚠️This Unit 42 proof-of-concept shows how an attacker can use indirect prompt injection to silently poison an AI agent’s long-term memory, demonstrated against a travel assistant built on Amazon Bedrock. The attack manipulates the agent’s session summarization process so malicious instructions become stored memory and persist across sessions. When the compromised memory is later injected into orchestration prompts, the agent can be coerced into unauthorized actions such as stealthy exfiltration. Unit 42 outlines layered mitigations including pre-processing prompts, Bedrock Guardrails, content filtering, URL allowlisting, and logging to reduce risk.

Researchers Identify Architectural Flaws in AI Browsers

🔒 A new SquareX Labs report warns that integrating AI assistants into browsers—exemplified by Perplexity’s Comet—introduces architectural security gaps that can enable phishing, prompt injection, malicious downloads and misuse of trusted apps. The researchers flag risks from autonomous agent behavior and limited visibility in SASE and EDR tools. They recommend agentic identity, in-browser DLP, client-side file scanning and extension risk assessments, and urge collaboration among browser vendors, enterprises and security vendors to build protections into these platforms.

GitHub Copilot Chat prompt injection exposed secrets

🔐 GitHub Copilot Chat was tricked into leaking secrets from private repositories through hidden comments in pull requests, researchers found. Legit Security researcher Omer Mayraz reported a combined CSP bypass and remote prompt injection that used image rendering to exfiltrate AWS keys. GitHub mitigated the issue in August by disabling image rendering in Copilot Chat, but the case underscores risks when AI assistants access external tools and repository content.

Security firm urges disconnecting Gemini from Workspace

⚠️FireTail warns that Google Gemini can be tricked by hidden ASCII control characters — a technique the firm calls ASCII Smuggling — allowing covert prompts to reach the model while remaining invisible in the UI. The researchers say the flaw is especially dangerous when Gemini is given automatic access to Gmail and Google Calendar, because hidden instructions can alter appointments or instruct the agent to harvest sensitive inbox data. FireTail recommends disabling automatic email and calendar processing, constraining LLM actions, and monitoring responses while integrations are reviewed.

Salesforce launches AI security and compliance agents

🔒 Salesforce introduced two AI agents on its Agentforce platform that monitor security activity and streamline compliance workflows for the Security Center and Privacy Center. The security agent analyzes event logs to detect anomalous behavior, accelerates investigations by assembling context and remediation plans, and can autonomously freeze or isolate suspicious accounts when authorized. The privacy agent maps metadata and policies against frameworks like GDPR and CCPA, surfaces exposures, and can reclassify or apply erasure policies to reduce compliance risk.

Google won’t fix new ASCII smuggling attack in Gemini

⚠️ Google has declined to patch a new ASCII smuggling vulnerability in Gemini, a technique that embeds invisible Unicode Tags characters to hide instructions from users while still being processed by LLMs. Researcher Viktor Markopoulos of FireTail demonstrated hidden payloads delivered via Calendar invites, emails, and web content that can alter model behavior, spoof identities, or extract sensitive data. Google said the issue is primarily social engineering rather than a security bug.

Gemini Trifecta: Prompt Injection Exposes New Attack Surface

🔒 Researchers at Tenable disclosed three distinct vulnerabilities in Gemini's Cloud Assist, Search personalization, and Browsing Tool. The flaws let attackers inject prompts via logs (for example by manipulating the HTTP User-Agent), poison search context through scripted history entries, and exfiltrate data by causing the Browser Tool to send sensitive content to an attacker-controlled server. Google has patched the issues, but Tenable and others warn this highlights the risks of granting agents too much autonomy without runtime guardrails.

CometJacking: One-Click Attack Turns AI Browser Rogue

🔐 CometJacking is a prompt-injection technique that can turn Perplexity's Comet AI browser into a data exfiltration tool with a single click. Researchers at LayerX showed how a crafted URL using the 'collection' parameter forces the agent to consult its memory, extract data from connected services such as Gmail and Calendar, obfuscate it with Base64, and forward it to an attacker-controlled endpoint. The exploit leverages the browser's existing authorized connectors and bypasses simple content protections.

AI and Cybersecurity: Fortinet and NTT DATA Webinar

🔒 In a joint webinar, Fortinet and NTT DATA outlined practical approaches to deploying and securing AI across enterprise environments. Fortinet described its three AI pillars—FortiAI‑Protect, FortiAI‑Assist, and FortiAI‑SecureAI—focused on detection, operational assistance, and protecting AI assets. NTT DATA emphasized governance, runtime protections, and an "agentic factory" to scale pilots into production. The presenters stressed the need for visibility into shadow AI and controls such as DLP and zero‑trust access to prevent data leakage.