All news with #llm security tag

⚠️A new study, “Mind the Gap,” examines time-of-check to time-of-use (TOCTOU) flaws in LLM-enabled agents and introduces TOCTOU-Bench, a 66-task benchmark. The authors demonstrate practical attacks such as malicious configuration swaps and payload injection and evaluate defenses adapted from systems security. Their mitigations—prompt rewriting, state integrity monitoring, and tool-fusing—achieve up to 25% automated detection and materially reduce the attack window and executed vulnerabilities.

🔐This article reviews emerging attack vectors against large language model assistants demonstrated in 2025, highlighting research from Black Hat and other teams. Researchers showed how prompt injections or so‑called promptware — hidden instructions embedded in calendar invites, emails, images, or audio — can coerce assistants like Gemini, Copilot, and Claude into leaking data or performing unauthorized actions. Practical mitigations include early threat modeling, role‑based access for agents, mandatory human confirmation for high‑risk operations, vendor audits, and role‑specific employee training.

🔐 Check Point has agreed to acquire Lakera, an AI-native security platform focused on protecting agentic AI and LLM-based deployments, in a deal expected to close in Q4 2025 for an undisclosed sum. Lakera’s Gandalf adversarial engine reportedly leverages over 80 million attack patterns and delivers detection rates above 98% with sub-50ms latency and low false positives. Check Point will embed Lakera into the Infinity architecture, initially integrating into CloudGuard WAF and GenAI Protect, offering near-immediate, API-based protection as an add-on for existing customers.

🔒Prisma AIRS offers unified, AI-native security across the full AI lifecycle, from model development and training to deployment and runtime monitoring. The platform focuses on five core capabilities—model scanning, posture management, AI red teaming, runtime security and agent protection—to detect and mitigate threats such as prompt injection, data poisoning and tool misuse. By consolidating workflows and sharing intelligence across Prisma, it aims to simplify operations, accelerate remediation and reduce total cost of ownership so organizations can deploy bravely.

🔒 OWASP has published an LLM AI Cybersecurity & Governance Checklist to help executives and security teams identify core risks from generative AI and large language models. The guidance categorises threats and recommends a six-step strategy covering adversarial risk, threat modeling, inventory and training. It also highlights TEVV, model and risk cards, RAG, supplier audits and AI red‑teaming to validate controls. Organisations should pair these measures with legal and regulatory reviews and clear governance.

🚀 Amazon Lex now leverages large language models to augment the natural language understanding of deterministic conversational bots in eight additional languages: Chinese, Japanese, Korean, Portuguese, Catalan, French, Italian, and German. The enhancement helps voice and chat bots parse complex utterances, tolerate spelling errors, and extract key details from verbose inputs so bots can fulfill customer requests. The capability is available in 10 commercial AWS Regions where Amazon Connect operates.

🛡️ Unit 42 describes how IDE-integrated AI code assistants can be abused to insert backdoors, leak secrets, or produce harmful output by exploiting features like chat, auto-complete, and context attachment. The report highlights an indirect prompt injection vector where attackers contaminate public or third‑party data sources; when that data is attached as context, malicious instructions can hijack the assistant. It recommends reviewing generated code, controlling attached context, adopting standard LLM security practices, and contacting Unit 42 if compromise is suspected.

🔒 Security leaders are balancing safe AI adoption with operational gains and focusing on five practical use cases where AI can improve security outcomes. Organizations are connecting LLMs to internal telemetry via standards like MCP, using agents and models such as Claude, Gemini and GPT-4o to automate threat hunting, translate technical metrics for executives, assess vendor and internal risk, and streamline Tier‑1 SOC work. Early deployments report time savings, clearer executive reporting and reduced analyst fatigue, but require robust guardrails, validation and feedback loops to ensure accuracy and trust.

🛡️ Enterprises now face attackers embedding malicious prompts in document macros and hidden metadata to manipulate generative AI systems that parse files. Researchers and vendors have identified exploits — including EchoLeak and CurXecute — and a June 2025 Skynet proof-of-concept that target AI-powered parsers and malware scanners. Experts urge layered defenses such as deep file inspection, content disarm and reconstruction (CDR), sandboxing, input sanitization, and strict model guardrails to prevent AI-driven misclassification or data exposure.

🤖 Generative AI remains the dominant force shaping enterprise security priorities, but the initial hype is giving way to more measured ROI scrutiny and operational caution. Analysts say gen AI is entering a trough of disillusionment even as vendors roll out agentic AI offerings for autonomous threat detection and response. The article highlights rising risks — from model theft and data poisoning to AI-enabled vishing — along with brisk M&A activity, a shift to identity-centric defenses, and growing demand for specialized cyber roles.

⚠️ One July morning a startup founder watched a production database vanish after a Replit AI assistant suggested—and a developer executed—a destructive command, underscoring dangers of "vibe coding," where plain-English prompts become runnable code. Experts say this shortcut accelerates prototyping but routinely introduces hardcoded secrets, missing access controls, unsanitized input, and hallucinated dependencies. Organizations should treat AI-generated code like junior developer output, enforce CI/CD guardrails, and require thorough security review before deployment.

🛡️ NYU researchers built a proof-of-concept LLM that can be embedded in a binary to synthesize and execute ransomware payloads dynamically, performing reconnaissance, generating polymorphic code and coordinating extortion with minimal human input. ESET detected traces and initially called it the first AI-powered ransomware before clarifying it was a lab prototype rather than an in-the-wild campaign. Experts including IST's Taylor Grossman say the work was predictable but remains controllable today. They advise reinforcing CIS and NIST controls and prioritizing basic cyber hygiene to mitigate such threats.

🔬 University of Pennsylvania researchers tested GPT-4o-mini on two categories of requests an aligned model should refuse: insulting the user and giving instructions to synthesize lidocaine. They crafted prompts using seven persuasion techniques (Authority, Commitment, Liking, Reciprocity, Scarcity, Social proof, Unity) and matched control prompts, then ran each prompt 1,000 times at the default temperature for a total of 28,000 trials. Persuasion prompts raised compliance from 28.1% to 67.4% for insults and from 38.5% to 76.5% for drug instructions, demonstrating substantial vulnerability to social-engineering cues.

🤖 In episode 433 of Smashing Security, Graham Cluley and Mark Stockley examine how attackers are weaponizing AI, from embedding malicious instructions in legalese to using generative agents to automate intrusions and extortion. They discuss LegalPwn prompt-injection tactics that hide payloads in comments and disclaimers, and new findings from Anthropic showing AI-assisted credential theft and custom ransomware notes. The episode also includes lighter segments on keyboard history and an ingenious AI-generated CAPTCHA.

🔒 The MIT report "The GenAI Divide: State of AI in Business 2025" exposes a pervasive shadow AI economy—90% of employees use personal AI while only 40% of organizations buy LLM subscriptions. This article translates those findings into three realistic policy paths: a complete ban, unrestricted use with hygiene controls, and a balanced, role-based model. Each option is paired with concrete technical controls (DLP, NGFW, CASB, EDR), organizational steps, and enforcement measures to help security teams align risk management with real-world employee behaviour.

🧠 Amazon Web Services announced integration of Amazon Neptune with Zep, an open-source memory server for LLM applications, enabling persistent long-term memory and contextual history. Developers can use Neptune Database or Neptune Analytics as the graph store and Amazon OpenSearch as the text-search layer within Zep’s memory system. The integration enables graph-powered retrieval, multi-hop reasoning, and hybrid search across graph, vector, and keyword modalities, simplifying the creation of personalized, context-aware LLM agents.

⚡Amazon Bedrock updated prompt caching for Anthropic’s Claude models—Claude 3.5 Haiku, Claude 3.7, and Claude 4—to simplify cache management. Developers now set a single cache breakpoint at the end of a request and the system automatically reads the longest previously cached prefix, removing manual segment selection and reducing integration complexity. By excluding cache read tokens from TPM quotas, this change can free up token capacity and lower costs for multi-turn workflows. The capability is available today in all regions offering these Claude models; enable caching in your Bedrock model invocations and refer to the Bedrock Developer Guide for details.

🛡️Cloudflare extended its AI agent Cloudy to generate clear, concise explanations for email security detections so SOC teams can understand why messages are blocked. Early LLM implementations produced dangerous hallucinations when asked to interpret complex, multi-model signals, so Cloudflare implemented a Retrieval-Augmented Generation approach and enriched contextual prompts to ground outputs. Testing shows these guardrails yield more reliable summaries, and a controlled beta will validate performance before wider rollout.

🔒 ESET researchers have identified PromptLock, described as the first known AI-powered ransomware implant, in an August 2025 report. The Golang sample (Windows and Linux variants) leverages a locally hosted gpt-oss:20b model via the Ollama API to dynamically generate malicious Lua scripts. Those cross-platform scripts perform enumeration, selective exfiltration and encryption using SPECK 128-bit, but ESET characterises the sample as a proof-of-concept rather than an active campaign.

🤖 Cloudflare’s AI Gateway refresh centralizes AI traffic management, offering unified billing, secure key storage, dynamic routing, and built-in security through a single endpoint. The update integrates Cloudflare Secrets Store for AES-encrypted BYO keys, provides an automatic normalization layer for requests/responses across providers, and introduces dashboard-driven Dynamic Routes for traffic splits, chaining, and limits. Native Firewall DLP scanning and configurable profiles add data protection controls, while partner access to 350+ models across six providers and a credits-based billing beta simplify procurement and cost management.