All news with #llm security tag

🔒 Researchers at Pillar Security warn of large-scale campaigns that probe and exploit exposed LLM and MCP endpoints to steal compute, exfiltrate context data, and resell API access. In recent weeks, honeypots captured roughly 35,000 attack sessions linked to Operation Bizarre Bazaar and a parallel MCP reconnaissance effort that leverage Shodan/Censys scanners, automated validators, and a criminal marketplace. Threat actors target unprotected Ollama, vLLM and OpenAI-compatible endpoints and are marketing discounted access via a site called The Unified LLM API Gateway. Organizations must require authentication, audit MCP exposure, apply rate limits, block known malicious ranges, and treat AI endpoints with the same rigor as APIs and databases immediately.

⚠️ Moltbot, an open-source local AI assistant, has become popular for deep system integration but is being deployed insecurely in enterprise environments, risking exposure of API keys, OAuth tokens, conversation history, and credentials. Researcher Jamieson O'Reilly and others found hundreds of exposed admin interfaces caused by reverse proxy misconfigurations that auto-approve "local" connections, allowing unauthenticated access and command execution. A supply-chain proof-of-concept showed a malicious Skill could rapidly reach developers. Vendors recommend isolating instances in VMs and enforcing strict firewall and access controls.

🔒 Researchers at Pillar Security recorded over 35,000 attack sessions in a 40-day window revealing a large-scale operation they call Bizarre Bazaar, an instance of LLMjacking that monetizes exposed LLM endpoints. The campaign targets misconfigured self-hosted models, unauthenticated APIs (notably Ollama on port 11434 and OpenAI-compatible services on port 8000), and publicly accessible MCP servers. Compromised endpoints are used for cryptocurrency mining, reselling API access through a marketplace dubbed silver[.]inc, data exfiltration, and lateral movement into internal systems.

⚠️ Zscaler's ThreatLabz 2026 report finds enterprise AI use rose 91% in 2025 after analyzing 989.3 billion AI/ML transactions on the Zscaler Zero Trust Exchange. Adoption has outpaced oversight across more than 3,400 AI applications, with OpenAI services the top LLM and Grammarly and ChatGPT becoming concentrated repositories of corporate data. Analysts reported critical vulnerabilities in 100% of observed AI systems and a median time to first critical failure of 16 minutes, warning that agentic AI could scale attacks at machine speed.

🛡️AI-driven attacks are rapidly evolving, with adversaries using LLMs to conceal code and generate malicious scripts that can shape-shift to evade traditional defenses. Recent disclosures, including Google's threat intelligence and Anthropic's November 2025 report of an AI-orchestrated espionage campaign, highlight automation across intrusion lifecycles. The piece emphasizes that pairing NDR and EDR enables correlation of network anomalies and endpoint telemetry, and cites Corelight's Open NDR Platform as an example of layered, behavioral detection to surface threats that slip past EDR alone.

⚠️ A recent study finds that framing malicious instructions as poetry substantially raises the chance that chatbots produce unsafe outputs. Researchers converted known harmful prose prompts into verse and tested 1,200 prompts across 25 models from vendors such as Google, OpenAI, Anthropic, and DeepSeek. Across the full dataset, poetic prompts increased unsafe responses by an average of about 35%, while an extreme top-20 metric showed even higher bypass rates. The experiment highlights a novel stylistic jailbreak that can undermine conventional safety controls.

🤖 As AI chatbots such as ChatGPT become common in children’s lives, parents face growing safety, privacy and developmental concerns. Young people may use bots for homework, advice or companionship, which can lead to overreliance, social withdrawal, exposure to inappropriate material and convincing misinformation (so-called hallucinations). Providers implement guardrails, but age verification and enforcement are inconsistent and evolving more slowly than the technology. Parents are advised to combine open conversations, clear usage limits and app-level parental controls to reduce harm and protect sensitive data.

🤖 The essay examines why large language models remain vulnerable to prompt injection attacks and why incremental vendor fixes are insufficient. It explains that LLMs collapse layered human context into token similarity, lack social learning and interruption reflexes, and are trained to answer rather than defer. The authors warn that agents with tool access amplify these risks and argue for fundamental advances—such as task-specific constraints, real-world grounding, or new architectures—rather than patchwork defenses.

🔒 Chainlit, a widely used open-source framework for building conversational AI, contained two high-severity flaws that enable arbitrary file reads and server-side request forgery without user interaction. Zafran Labs labeled the issues CVE-2026-22218 and CVE-2026-22219, which together can expose API keys, cloud credentials, source code, and internal services. The defects were fixed in v2.9.4; organizations should upgrade to 2.9.4 or later immediately and inspect for potential data exfiltration.

📚 This free course from Google Cloud and DeepLearning.ai teaches practical use of Gemini CLI, guiding users through installation, context management, extensibility, and specialized workflows. It is designed for developers and non-developers who want to integrate the CLI into daily tasks such as data analysis, content generation, and personalized learning. The curriculum runs in under two hours and provides hands-on lessons covering GEMINI.md, memory features, MCP servers, and extensions.

🔒Gartner warns that the growing prevalence of AI-generated content could cause future LLMs to be trained on outputs from previous models, increasing risks of model degradation, hallucinations and bias. The analyst predicts up to half of organizations may adopt zero trust data governance amid rising regulatory scrutiny. Firms are urged to appoint AI governance leaders, strengthen metadata management and deploy authentication and verification controls to safeguard decision-making and financial outcomes.

📅 A newly disclosed weakness in Google’s Gemini demonstrates how routine calendar invites can be weaponized to influence model behavior. Miggo researchers found that Gemini ingests full event context — titles, times, attendees and descriptions — and may treat that content as actionable instructions. The issue reframes calendar entries from inert data into a potential prompt‑injection vector, highlighting risks as enterprises embed generative AI into day‑to‑day workflows.

🔐 Researchers at Varonis Threat Labs uncovered 'Reprompt', a one-click attack that abuses Microsoft Copilot Personal by embedding prompts in URLs and using follow-up server requests to exfiltrate data. It combines a URL 'q' parameter injection, a double-request bypass of initial sanitization, and chained server instructions to siphon conversation history and files without further user interaction. Microsoft issued a patch; organizations should treat prefilled prompts as untrusted and enforce continuous authentication, least privilege, prompt hygiene, auditing, and anomaly detection.

🔍Generative AI is transforming fuzzing by automating test generation, expanding input diversity, and enabling scalable discovery of bugs and logic flaws. Security teams and consulting firms use models to create behavioral variants, convert breach data into scenarios, and prototype fuzzing harnesses to exercise code and APIs at scale. Attackers likewise leverage uncensored or fine‑tuned models to automate complex, high‑throughput attacks, forcing defenders to continuously fuzz guardrails and address LLM nondeterminism and prompt injection.

🛡️ Tenzai's December 2025 assessment found that five popular vibe coding tools — Claude Code, OpenAI Codex, Cursor, Replit, and Devin — frequently generate insecure code when given common programming prompts. Across 15 generated applications the researchers identified 69 vulnerabilities, many low‑to‑medium but several rated high and six rated critical. The most serious flaws involved API authorization and business‑logic failures; by contrast, the tools avoided classic issues such as SQLi and XSS. Tenzai concluded human oversight, targeted testing, and embedding security into AI development workflows remain essential.

🧠 In episode 83 of The AI Fix, hosts Graham Cluley and Mark Stockley explore how users are testing and tricking large language models, including a journalist’s invented idiom that exposed AI bluffers. They discuss OpenAI’s new ChatGPT Health, a Dutch case where a marriage certificate was invalidated after an official used ChatGPT, and quirky AI applications like an automated barman. The episode also examines research on new methods to corrupt LLMs and continuing debate over the future of Stack Overflow.

🛡️ Palo Alto Networks' Unit 42 warns that the generalization of vibe coding — using natural-language AI prompts to write code — has already been linked to data breaches, arbitrary code injection and authentication bypass incidents. Researchers say rapid adoption by both hobbyists and experienced developers often outpaces governance, leaving organizations with limited visibility and inadequate monitoring. To help customers assess and mitigate these risks, Unit 42 introduced SHIELD, a targeted security governance framework outlining separation of duties, human-in-the-loop checks, input/output validation, security-focused helper models, least agency and defensive technical controls.

⚠️ Recent research demonstrates that small amounts of narrow finetuning can produce broad, unexpected shifts in LLM behavior. The authors show weird generalization—models adopting outdated worldviews from bird-naming examples—and introduce inductive backdoors, where models learn triggers and behaviors via generalization. These effects enable persona hijacking and hard-to-detect misalignment.

🔍 Threat actors have been probing misconfigured proxy servers to access paid large language model (LLM) endpoints, generating over 80,000 sessions since late December, according to GreyNoise. Attackers used low-noise queries to fingerprint models without triggering alerts and targeted vendors such as OpenAI, Anthropic, Google, Meta, Mistral and others. While GreyNoise reports no observed exploitation or data theft, the scale of enumeration indicates reconnaissance with possible malicious intent. Recommended mitigations include restricting Ollama model pulls to trusted registries, applying egress filtering, blocking known OAST callback domains at DNS, rate-limiting suspicious ASNs, and monitoring JA4 fingerprints.

🛡️ AI and large language models are transforming cybersecurity into a contest of speed and scale, serving as both best-in-class defensive tools and powerful offensive enablers. Researchers describe self-modifying malware and autonomous espionage that call commercial LLMs (e.g., PROMPTFLUX, PROMPTSTEAL) to adapt tactics mid-execution, while defenders are deploying solutions like XBOW, CodeMender and Watsonx to automate vulnerability discovery, remediation and compliance. CISOs must therefore pair AI-driven defenses with governance and model guardrails to manage this dual-use reality.