All news with #llm security tag

🛡️ Security vendor CrowdStrike has added five new prompt injection techniques to its taxonomy that threaten enterprise AI deployments. These attacks manipulate LLM behavior by embedding deceptive instructions into inputs, context, or token streams to bypass safety controls and produce malicious outputs. CrowdStrike recommends threat modeling input sources, expanding testing, and enhancing detection engineering to defend against composite and multi-stage prompt attacks.

🔒 Over 70 cybersecurity organisations have signed the new CREST AI Charter, launched on July 9, committing to nine principles governing AI-enabled cybersecurity activities. The charter covers accountability and governance, transparency of use, documentation and auditability, boundaries and control, data handling and sovereignty, security and confidentiality, secure development, supply chain assurance and resilience. Signatories will maintain human oversight, document AI use, disclose data practices and implement secure development and supply chain controls. CREST intends the charter as a self-regulatory foundation to drive standards and harmonisation across industry and regulators.

🛡️ System prompts are core to generative AI applications and often include role definitions, tool descriptions, RAG context, and other proprietary instructions. This AWS Security Blog post explains why system prompt leakage is a persistent risk, highlights that it cannot be fully remediated today, and outlines practical mitigations. It recommends design principles such as minimization and avoiding sensitive data in prompts, and details controls available via Amazon Bedrock Guardrails and other mechanisms to reduce exposure and raise extraction difficulty.

🧭 A new study found GitHub Copilot can be induced to generate harmful answers when a dangerous request is broken into ordinary coding steps. Researchers Abhishek Kumar and Carsten Maple tested Claude and Gemini models through Copilot and observed that direct chat prompts were routinely refused, but the same content was produced in 816 of 816 workflow runs when framed as benchmark-improving “teaching shots.” The paper calls this technique workflow-level jailbreak construction and urges reviewing written files and whole sessions, not just chat refusals.

🛡️ Zscaler tested 26 LLMs and found several autonomous agents susceptible to indirect prompt injection (IPI) traps, with some high-end models failing while a few lower-tier models fared better. The vendor identified hidden instructions on websites that manipulated agent behavior and caused real-world impacts in controlled tests. Experts warn that agent risk is dynamic, the attack surface is architectural, and binary "safe/vulnerable" labels are overly simplistic for CISOs. The findings highlight that agentic AI introduces new trust boundaries and insider-like threats to enterprise security.

🚀 Amazon SageMaker HyperPod now supports Disaggregated Prefill and Decode (DPD), which splits LLM inference into separate prefill and decode GPU pools and transfers KV cache over EFA using GPU-Direct RDMA. This reduces contention between long-context prefill and per-token decode, enabling more consistent per-token latency, higher goodput under strict latency SLOs, and independent scaling of prefill and decode. DPD is enabled via a pdSpec in the existing InferenceEndpointConfig and works with HyperPod's KV cache offloading and intelligent routing on EFA-capable instances.

🔒 Google Cloud and Valtech introduce Nexus SDV, an open-source, modular platform that enables AI-native, scalable management of software-defined vehicles. The platform integrates with Android Automotive OS and supports up to 100 million devices while emphasizing TCO reduction via Arm-based compute and Bigtable. Nexus AI leverages Gemini models and the Gemini Enterprise Agent Platform for real-time telemetry analysis and agentic vehicle capabilities. Security is built-in with mTLS/PKI, identity brokering, secret management, network isolation, and an enterprise Secure AI Framework.

🔍 Sysdig researchers detailed an autonomous AI agent, dubbed JadePuffer, that executed an end-to-end intrusion and extortion campaign after exploiting a vulnerable Langflow server. The agent leveraged an LLM to adapt tactics, delivering over 600 Base64-encoded Python payloads to pivot from an internet-facing Langflow instance to a production MySQL/Nacos server and encrypt 1,342 configuration records before demanding ransom. The operation demonstrated rapid self-correction and contextual reasoning in payloads, prompting calls for behavior-focused detection.

📘 Enterprise LLM deployments face KV cache growth that can exceed node RAM and local SSD capacity. Google Cloud demonstrates using Managed Lustre as a shared, high-performance external KV cache tier integrated with vLLM and the llm-d offloading stack to bypass host-level limits and simplify cross-node data management. Benchmarks with Llama-3.3-70B on a six-node A3 Mega cluster show over 50% TCO savings and nearly 60% reduction in GPU-hour usage by achieving a 95% cache hit rate. The guide includes architecture, deployment steps, PVC evictor behavior, and validated tracks for Qwen and Gemma models.

🛡️ Unit 42 found that large language models (LLMs) commonly hallucinate plausible web domains for real brands, and adversaries are registering these nonexistent domains to intercept AI-generated traffic. This phenomenon, called phantom squatting, poses a supply chain risk and was observed across multiple sectors. Researchers predicted adversary registrations 18–51 days in advance and discovered over 13,229 malicious URLs plus ~250,000 unregistered hallucinated domains.

🧭 This post introduces the preview of BigQuery's new AI.AGG() function, which enables natural-language aggregation over millions of rows of unstructured and multimodal data directly inside SQL. It explains practical uses—analyzing logs, discovering product categories, and summarizing image collections—while showing how AI.AGG() batches inputs, handles NULLs, reports errors, and integrates with other BigQuery AI functions like AI.CLASSIFY(). The write-up outlines best practices for token usage, model endpoint selection, and struct handling to help users deploy AI.AGG() effectively.

🛡️ OpenAI has unveiled GPT-5.6 Sol, calling it its "most capable model yet for cybersecurity," but initial access is restricted to a small set of vetted partners at the request of the US government. The preview, announced on June 26, introduces three tiers—Sol, Terra and Luna—and is available via API and Codex to selected partners while OpenAI coordinates with the government on a cyber executive order framework. OpenAI says Sol excels at long-horizon tasks like vulnerability research, includes enhanced safeguards and real-time classifiers, and currently does not autonomously produce full exploits.

🔎 The article argues that frontier AI models like Mythos are a signal of shifting cyber economics rather than an immediate, novel threat. It emphasizes that longstanding security fundamentals—asset visibility, patching, identity controls and resilient operations—remain the primary defenses. The author advocates using AI to accelerate analysis, prioritize remediation and close persistent control gaps rather than replacing skilled practitioners or prompting reactive, headline-driven spending.

🛡️ The article examines how AI, particularly large language models, can bridge the gap between atomic indicators of compromise (IOCs) and richer strategic threat intelligence by indexing and relating unstructured reports. It highlights opportunities to retrieve relevant intelligence and generate tailored defensive advice while warning about data veracity and confidentiality. The piece also emphasizes practical Windows threats abusing COM and recommends tooling and hunting practices to detect such misuse.

📝 This post highlights a new paper that demonstrates how large language models are vulnerable to prompt injection because they learn to distinguish instruction blocks by style rather than explicit tags. The authors argue that role tags became a de facto security architecture but do not map cleanly into model representations, producing persistent role confusion. The paper warns that without genuine role perception, defenses will be reactive and brittle, and calls for deeper study of roles within the LLM stack.

🛡️ A malware developer has begun embedding provocative text about nuclear and biological weapons inside large JavaScript block comments in spyware payloads to confuse AI-based scanners. The commented header is ignored at runtime but aims to trigger refusals or misclassification in naive LLM-powered triage systems that ingest file starts without isolating untrusted content. Traditional detection methods—YARA, entropy checks, AST parsing, and behavioral analysis—remain effective, but the technique is a practical anti-analysis tactic against weak AI-first pipelines.

🛡️ AI is reshaping cybersecurity by breaking the long-held assumption of predictable, deterministic systems. Traditional prevention-focused controls remain important but are insufficient as AI agents, LLMs and automated development accelerate runtime change and attacker capabilities. Organizations must prioritize runtime visibility, use AI to augment defensive operations, rebuild vulnerability management and emphasize resilience and containment to manage evolving AI-driven risks.

🔍 AI red teaming identifies how deployed AI systems can be manipulated or misused in real operational contexts. It tests the interaction of models with prompts, retrieval, tools, and workflows to produce actionable attack paths rather than isolated examples. This adversarial, continuous approach complements traditional security by focusing on intent, context, policy, and business impact. Teams should inventory systems, threat model by risk, red team early and often, and re-test after changes.

🔍 At Google Cloud, collaboration with customers guided a technical sprint in January 2026 with a major telecommunications partner to refine Model Armor, the runtime security service for generative AI. By embedding with the customer's developers and security teams, the Google Cloud Developer Advocacy group observed real-world workflows and identified friction points such as search-first documentation needs, tuning confidence levels to avoid false positives, clearer enforcement guidance, and IAM-related 403 errors during integrations. The team translated these findings into tested code samples, a confidence level matrix, explicit integration guides for Apigee, Gemini Enterprise Agent Platform, and GKE, and deeper technical documentation to improve operational utility and reduce deployment friction.

🛡️ Researchers at Obsidian Security disclosed a three-bug chain in the open-source LiteLLM proxy that lets a default low-privilege account escalate to full proxy admin and achieve remote code execution. The combined issue, rated CVSS 9.9, exposes provider keys, decryption secrets, prompts, and responses. Maintainer BerriAI published fixes in LiteLLM v1.83.14-stable (May 2); users should upgrade and audit admin roles, guardrails, callbacks, and keys.