All news with #llm security tag

🔎 Our new research introduces perturbation probing, a two-pass, low-cost method that identifies the small set of feed-forward neurons causally responsible for targeted behaviors in aligned LLMs. Applied to Qwen3-4B and Qwen3.5-2B, the method found that tens of neurons (a tiny fraction of the model) control refusal and agreement behaviors, showing alignment can be highly concentrated. The study also defines the FFN/Skip ratio as a quick diagnostic predicting fragility across models.

🔒 The article argues that poorly designed external AI guardrails can erode defenders' advantages by blocking or delaying agentic SOC investigations, giving attackers time to succeed. It urges organizations to retain operational sovereignty by embedding customizable guardrails within their own systems and testing LLMs against real workflows. Cisco Talos evaluated many models and stresses balancing efficacy, cost, speed, and consistency when selecting AI for security.

🔒 Security researchers demonstrated that an AI email summarizer can be tricked into reading hidden content different from what a user sees. Forcepoint X‑Labs embedded invisible HTML in emails that remained hidden in Outlook but were passed to an LLM-driven summarizer, allowing prompt-injection instructions to alter summaries silently. Their proof-of-concept showed consistent manipulation of invoice dates and omitted names across repeated tests, highlighting risks when untrusted email content is fed to models without guardrails. Forcepoint recommends extracting only visible content, detecting hidden styling, separating headers from body, and validating AI summaries against source material.

🔒 Microsoft describes attacks targeting AI infrastructure components such as gateways, retrieval platforms, orchestration services, and container runtimes that centralize credentials and execution privileges. Observed intrusions against LiteLLM, RAGFlow, and Kestra aimed to harvest secrets, persist on hosts, and monetize compute. The advisory emphasizes inventorying exposed AI surfaces, restricting administrative access, and monitoring gateway-originated execution and secret access to mitigate risk.

🔍 This post explains how Microsoft Foundry helps reduce the cost per successful AI outcome by optimizing each model request at runtime. It outlines four levers—model routing and deployment choices, caching, prompt and agent optimization, and observability—that teams can apply, measure, and reverse if necessary. The guidance emphasizes matching model and deployment choices to task complexity, using caching and fine-tuning to lower repeated costs, and relying on observability and evaluation to validate savings without degrading quality.

🛡️Cloud-hosted AI has become the default for many SOCs, but external guardrails introduce a "safety penalty" that blocks legitimate defensive work. When models refuse to analyze malware or explain exploits, defenders lose crucial time while adversaries operate without such constraints. The article argues defenders must regain operational sovereignty over models or create reliable fallback paths to avoid asymmetric advantage.

🔒 AI gives defenders powerful discovery tools but grants attackers the same advantages, forcing CISOs to manage AI risks both externally and internally. External threats include AI-enhanced phishing, rapid exploit development, and autonomous agent attacks, while internal risks arise from uncontrolled employee use of consumer AI platforms, shared copilots, and compromised API billing. The author advocates a Risk-First approach: map AI use, prioritize controls like RBAC and data classification, improve continuous testing, and run tabletop exercises to prepare for AI-specific failures.

🛡️ This week’s ThreatsDay highlights multiple vectors where trusted components and weak checks are repurposed for attack. Research shows Microsoft-signed drivers can be abused for kernel operations, and a critical Gogs RCE (CVSS 10.0) enables remote code execution via Git hooks. Other items include a large-scale Iran-linked academic espionage case, DLL sideloading campaigns, BYOVD abuse, guardrail-free AI services, and exposed refrigeration controllers.

🔍 This post outlines ten essential questions startups must address when moving from AI prototype to production on Google Cloud. It contrasts Google AI Studio for rapid prototyping with the Gemini Enterprise Agent Platform for enterprise controls, and emphasizes sequencing migration before you have real users. The article highlights operational pitfalls—API key leaks, IAM ownership gaps, and quota 429s—and provides practical checklist items, role guidance, and mitigation strategies including regional endpoints, retries, and consumption models.

🔎 A brief report explains how threat-modeling expert Adam Shostack developed PHANTOM-B, a focused framework for quickly identifying LLM-specific risks such as prompt injection, hallucination, and bias. The approach complements existing methods like STRIDE by targeting components that interact with large language models and enabling useful results in short sessions. The article outlines why traditional threat modeling falls short for generative and agentic AI and stresses that fundamentals of application security must still be applied alongside new AI-focused controls.

🧭 Bruce Schneier examines recent research on AI and contextual integrity, focusing on how large language models manage persistent memory and the risks of inappropriate information disclosure. He highlights two papers: one (CIMemories) showing widespread attribute-level leakage across tasks and runs, and another demonstrating that explicit reasoning and RL training can improve context-aware disclosure. Schneier emphasizes that solutions require reasoning capabilities, not just better prompting or scaling.

🔍 Recent studies show LLMs now excel at discovering zero-day vulnerabilities yet continue to produce insecure code at scale. Veracode found 44% of AI-generated code contains OWASP Top 10 issues and no model exceeded a 68% security pass rate, despite near-perfect syntax correctness. Other research from SIG, Xint.io, and 1Password’s Off-By-1 Labs similarly reports high rates of security violations and low patch success. Specialized harnesses and deterministic tooling improve outcomes, but human oversight and contextual organization controls remain essential.

🛡️ The Solicitors Regulation Authority (SRA) has issued a warning to solicitors and law firms about using AI responsibly after spotting hallucinations and data leaks. The notice emphasizes that regulated individuals remain accountable for AI outputs and must maintain appropriate human oversight, governance and secure handling of client data. The SRA highlighted risks including false case citations, potential contempt of court and breaches of client confidentiality when information is entered into public AI tools.

🔒 Amazon OpenSearch Service now supports automatic semantic enrichment for VPC-enabled domains, allowing customers with private network configurations to use AI-powered semantic search without exposing domains to the public internet. The feature converts keyword searches into context-aware retrieval, handling all semantic processing automatically and removing the need to manage ML models. It requires OpenSearch version 2.19 or later and is available across 11 Regions globally.

🔍 The OWASP GenAI LLM Top 10 for 2026 updates a core security reference, keeping Prompt Injection at the top while elevating Excessive Agency and broadening Context concerns. The ranking highlights persistent data, supply chain and output risks and signals that AI security must cover models, surrounding systems and downstream impact.

🔍 Security operations research shows that AI-driven SOC workflows depend more on the fidelity of underlying telemetry than on specific LLM choices. The Provably Better Data project evaluated multiple LLMs across controlled CTF and incident response benchmarks using Corelight, firewall, Snort, and NetFlow telemetry normalized to OCSF. Results found that high-fidelity protocol-aware logs produced 2–4x better outcomes in accuracy, evidence coverage, and investigation time, and reduced analyst rework and hallucinations. The study advises SOC leaders to prioritize data quality and structured telemetry when investing in AI automation.

🛡️ Zhipu has released GLM-5.3, a coding-focused AI that its makers say developed stronger-than-expected cybersecurity capabilities during post-training scaling. The model scored 84.5% on CyberGym for vulnerability identification, slightly ahead of Anthropic’s Mythos 5 and OpenAI’s GPT-5.6 Sol, but lagged on ExploitBench where it scored 54.4%. Zhipu reports large gains over GLM-5.2 through reinforcement learning in complex environments and plans an open-weight release after safety hardening.

🔒 July 2026 saw a marked uptick in cyber incidents, with weekly attacks averaging 2,336 per organization and ransomware victims rising sharply. Education, Latin America, and Business Services were among the most affected, while GenAI use exposed sensitive data through risky prompts. Email remained a primary entry point as organizations confront multi-vector threats and growing operational exposure.

🔒 A new research paper shows a flaw in how OpenAI, Anthropic, and Google carry encrypted reasoning between API calls, enabling recovery of hidden internal reasoning and secrets from session logs. The team demonstrated replay and decoding attacks that recovered API keys, passwords, and other private artifacts from publicly available agent traces. Vendors implemented mitigations and the authors say the main extraction no longer reproduces as of August 2026, but developers are urged to strip opaque reasoning blocks from shared logs.

🛡️ Researchers from Tracebit report that embedding prompt injections alongside secrets stored on AWS can disrupt AI hacking agents by triggering LLM guardrails. These injected prompts instruct the model to perform forbidden actions, causing the LLM to shut down or stop following prior commands—a technique the researchers call context bombing. The approach succeeds only when attackers use models with built-in safety filters; locally run or unguarded models remain unaffected.