All news with #model security tag

🔎 Our new research introduces perturbation probing, a two-pass, low-cost method that identifies the small set of feed-forward neurons causally responsible for targeted behaviors in aligned LLMs. Applied to Qwen3-4B and Qwen3.5-2B, the method found that tens of neurons (a tiny fraction of the model) control refusal and agreement behaviors, showing alignment can be highly concentrated. The study also defines the FFN/Skip ratio as a quick diagnostic predicting fragility across models.

🧭 Bruce Schneier examines recent research on AI and contextual integrity, focusing on how large language models manage persistent memory and the risks of inappropriate information disclosure. He highlights two papers: one (CIMemories) showing widespread attribute-level leakage across tasks and runs, and another demonstrating that explicit reasoning and RL training can improve context-aware disclosure. Schneier emphasizes that solutions require reasoning capabilities, not just better prompting or scaling.

🔒 Three high-severity vulnerabilities in Hugging Face’s diffusers library allowed crafted model repositories to execute arbitrary code during model loading by bypassing the trust_remote_code safeguard. Zafran Security published findings showing the trust check ran separately from the code load, creating timing and path-based bypasses exploited by crafted files and configuration changes. Hugging Face patched the issues in diffusers 0.38.0 in May and acknowledged related concerns in transformers.

🧰 Amazon SageMaker AI now supports serverless model customization for Gemma 4 E4B and 31B models using supervised fine-tuning (SFT), direct preference optimization (DPO), and reinforcement fine-tuning (RFT). You can adapt these Google DeepMind-built Gemma models to specific domains and workflows, and SageMaker AI handles infrastructure provisioning and training orchestration so teams pay only for what they use. The launch expands serverless customization to include models from Nova, Nemotron 3, Qwen, Llama, gpt-oss, and DeepSeek families, and is available in US East (N. Virginia), US West (Oregon), Asia Pacific (Tokyo), and EU (Ireland).

🔒 This article examines the rising need for AI security posture management (AI-SPM) as enterprises adopt AI across workflows. It outlines how AI maturity stages — from AI-assisted to AI-native — change security requirements and why agents and model services expand the attack surface. The piece surveys vendor approaches, key features, and integrations, and provides guidance for selecting AI-SPM solutions to avoid coverage gaps.

🔒 Nutanix and Palo Alto Networks have integrated Prisma AIRS into the Nutanix Enterprise AI platform to embed automated AI model scanning and continuous red teaming directly into the MLOps pipeline. The integrated solution scans models at check-in, analyzes dependencies for known vulnerabilities and license issues, and validates provenance and file formats to block backdoors or unsafe execution paths before deployment. It also provides API-driven red teaming with a context-aware agent and a large, continuously updated attack library so teams can test resilience and prioritize business-relevant risks without complex setup.

🔐 Anthropic announced Claude Mythos Preview can autonomously discover and weaponize software vulnerabilities, prompting the company to restrict access to a small set of partners. The claim unsettled security researchers and analysts, in part because details remain sparse and speculation ranges from capacity limits to safety-driven restraint. The authors view Mythos as a real but incremental advancement that highlights the need to separate patchable from unpatchable systems and the verifiable from the hard-to-verify. They recommend tighter isolation, least-privilege design, continuous testing, and the use of defensive AI agents to reduce risk.

⚠️ A critical vulnerability (CVE-2026-5760) in SGLang allows remote code execution via specially crafted GGUF model files. The flaw targets the /v1/rerank endpoint, where a malicious tokenizer.chat_template containing a Jinja2 SSTI payload is rendered using an unsandboxed jinja2.Environment(), enabling arbitrary Python execution. Researcher Stuart Beck reported the issue to CERT/CC, which recommends replacing jinja2.Environment() with ImmutableSandboxedEnvironment to mitigate the risk. No patch was obtained during coordination.

💾 Cloudflare's Unweight is a lossless compression system for LLM weights that reduces model size by roughly 15–22% while preserving bit-exact outputs and requiring no special hardware. It compresses only the exponent byte of BF16 tensors—using Huffman coding, palette/transcoding and row-level fallbacks—while leaving sign and mantissa untouched. Decompression happens into GPU shared memory to feed tensor cores directly, and Cloudflare has published a technical paper and open-sourced GPU kernels.

🛡️ At Nutanix .NEXT 2026, Palo Alto Networks highlighted an expanded integration delivering native, automated security across Nutanix environments and was named Nutanix 2026 Global Security Partner of the Year. The partnership extends Layer‑7 protection via VM‑Series virtual firewalls, consistent hybrid cloud policies for Nutanix Cloud Clusters (NC2), and Panorama-driven automation. A forthcoming integration embeds Prisma AIRS into Nutanix Enterprise AI (NAI) to enforce AI Model Security, continuous AI Red Teaming, and unified visibility so only validated models reach production.

🔒 Service providers face a generational opportunity to become AI factories, hosting high-performance, low-latency AI for enterprises while meeting sovereignty and compliance needs. Palo Alto Networks argues that securing these environments requires layered defenses from physical infrastructure through models and agents, combining ML-led NGFWs, Prisma AIRS, CyberArk and Cortex. The aim is real-time governance of data, nonhuman identities and autonomous agents to prevent poisoning, prompt injection and credential theft.

🔎 Three recent papers show that encrypted LLM traffic can leak sensitive information through timing, packet-size, and speculative-decoding side channels. The studies demonstrate that attackers can infer conversation topics, fingerprint prompts, and in some cases recover PII or confidential datastore tokens on open-source and production systems. The authors evaluate mitigations such as padding, batching, and token aggregation, but find trade-offs and no complete solution yet.

🛡️ Microsoft has developed a scanner to detect hidden backdoors in open-weight language models, focusing on triggers and malicious behaviors inserted during training or fine-tuning. The tool flags three observable signatures — attention hijacking, leakage of poisoned training fragments, and sensitivity to partial triggers — and runs using forward passes only without retraining or backpropagation. It is designed to work with most causal, GPT-style models and to serve as an added layer of supply-chain security for enterprises using third-party or open-source models.

🔒 Researchers propose a defensive data-poisoning tool called AURA to protect proprietary knowledge graphs that feed LLMs. The method injects plausible but false entries that authorized users can filter out with a secret key, while stolen graphs become unreliable for attackers. The authors report degrading unauthorized accuracy to 5.3% and preserving 100% fidelity for key-holders with under 14% max latency overhead.

🔐 OpenAI says rapid model gains have reshaped its planning and prompted expanded defensive measures. Internal CTF assessments rose from 27% on GPT-5 in August 2025 to 76% on GPT-5.1-Codex-Max in November 2025, leading the company to warn some systems may reach 'High' levels on its Preparedness Framework. OpenAI outlined a layered defense-in-depth strategy — including access controls, infrastructure hardening, egress monitoring, model steering, detection tools and end-to-end red teaming — and is preparing a trusted access program alongside private-beta tools such as Aardvark to steer capabilities toward defensive outcomes.

🔒 Three critical zero-day vulnerabilities in PickleScan, a widely used scanner for Python pickle files and PyTorch models, could enable attackers to bypass model-scanning safeguards and distribute malicious machine learning models undetected. The JFrog Security Research Team published an advisory on 2 December after confirming all three flaws carry a CVSS score of 9.3. JFrog has advised upgrading to PickleScan 0.0.31, adopting layered defenses, and shifting to safer formats such as safetensors.

🔍 Running into CUDA out-of-memory errors is a common blocker when fine-tuning models; High Bandwidth Memory (HBM) holds model weights, optimizer state, gradients, activations, and framework overhead. The article breaks down those consumers, provides a simple HBM sizing formula, and walks through a 4B-parameter bfloat16 example that illustrates why full fine-tuning can require tens of GBs. It then presents practical mitigations—PEFT with LoRA, quantization and QLoRA, FlashAttention, and multi‑GPU approaches including data/model parallelism and FSDP—plus a sizing guide (16–40+ GB) to help choose the right hardware.

🔒 Prisma AIRS 2.0 presents a unified AI security platform that addresses the “AI black box” by combining AI Model Security and automated AI Red Teaming. It inventories models, inference datasets, applications and agents in real time, inspects model artifacts within CI/CD and model registries, and conducts continuous, context-aware adversarial testing. The platform integrates curated threat intelligence and governance mappings to deliver auditable risk scores and prioritized remediation guidance for enterprise teams.

🔒 VirusTotal and Hugging Face have announced a collaboration to surface security insights directly within the Hugging Face platform. When browsing model files, datasets, or related artifacts, users will now see multi‑scanner results including VirusTotal detections and links to public reports so potential risks can be reviewed before downloading. VirusTotal is also enhancing its analysis portfolio with AI-driven tools such as Code Insight and format‑aware scanners (picklescan, safepickle, ModelScan) to highlight unsafe deserialization flows and other risky patterns. The integration aims to increase visibility across the AI supply chain and help researchers, developers, and defenders build more secure models and workflows.

🔒 CrowdStrike announced its intent to acquire Pangea to deliver the industry’s first AI detection and response (AIDR) capability, securing enterprise AI use and development across data, models, agents, identities, infrastructure, and interactions. Unveiled at Fal.Con 2025 by Michael Sentonas, the deal will integrate Pangea’s prompt‑layer and interaction security with the Falcon platform to provide unified visibility, governance, and enforcement across the AI lifecycle. The combined solution targets prompt injection, model manipulation, shadow AI and sensitive data exfiltration while enabling developers and security teams to innovate faster with built‑in safeguards.