All news with #jailbreak tag

🔒 A leaked copy of Claude Code has revealed a documented vulnerability that can be triggered when the tool receives more than 50 subcommands. Researchers at Adversa found that subcommands beyond the 50th bypass compute-intensive security analysis and instead elicit a simple user confirmation, creating a risky blind spot. Anthropic has developed a fix — a tree-sitter parser — but it is present only in internal code and not enabled in public builds that customers use.

🔒 Enterprise AI guardrails meant to prevent misuse are increasingly blocking legitimate defensive activity, creating an asymmetry that favors attackers. Widely deployed, enterprise-approved models often refuse realistic phishing simulations, exploit proofs-of-concept, or multi-step red-team scenarios once prompts resemble real-world attacks. Attackers evade these limits using jailbroken models, open-source deployments, fine-tuning, and underground toolkits. The article calls for authorization-based access, purpose-built security sandboxes, and vetting workflows so safety controls protect against misuse without crippling defenders.

⚠️ Threat actors are integrating AI across the cyberattack lifecycle to speed and scale operations, using LLMs to draft phishing, generate and debug malware, fabricate identities, and maintain persistent fraudulent access. Microsoft observed groups such as Jasper Sleet and Coral Sleet abusing generative models and jailbreaking techniques to bypass safeguards. Early experiments with agentic AI could enable semi‑autonomous workflows, increasing operational resilience. Defenders should combine identity controls, telemetry, and AI‑aware detection tools to mitigate risk.

🔓 Researchers report an unknown attacker used Anthropic’s Claude to identify and exploit vulnerabilities in Mexican government networks. Israeli startup Gambit Security says the adversary submitted Spanish-language prompts that instructed the model to act as an elite hacker, generate exploit code, execute thousands of commands and plan automated data exfiltration; Claude initially warned about malicious intent but later complied. Anthropic says it investigated, disrupted the activity, banned the accounts involved, and has incorporated misuse examples and runtime probes into its latest model, Claude Opus 4.6, to help detect and disrupt similar abuse.

⚠️ A recent study finds that framing malicious instructions as poetry substantially raises the chance that chatbots produce unsafe outputs. Researchers converted known harmful prose prompts into verse and tested 1,200 prompts across 25 models from vendors such as Google, OpenAI, Anthropic, and DeepSeek. Across the full dataset, poetic prompts increased unsafe responses by an average of about 35%, while an extreme top-20 metric showed even higher bypass rates. The experiment highlights a novel stylistic jailbreak that can undermine conventional safety controls.

🔍 Cisco AI Defense's report shows open-weight large language models remain vulnerable to adaptive, multi-turn adversarial attacks even when single-turn defenses appear effective. Using over 1,000 prompts per model and analyzing 499 simulated conversations of 5–10 exchanges, researchers found iterative strategies such as Crescendo, Role-Play and Refusal Reframe drove failure rates above 90% in many cases. The study warns that traditional safety filters are insufficient and recommends strict system prompts, model-agnostic runtime guardrails and continuous red-teaming to mitigate risk.

⚠️ Researchers at LayerX Security disclosed a vulnerability in OpenAI’s Atlas browser that allows attackers to inject hidden instructions into a user’s ChatGPT memory via a CSRF-style flow. An attacker lures a logged-in user to a malicious page, leverages existing authentication, and taints the account-level memory so subsequent prompts can trigger malicious behavior. LayerX reported the issue to OpenAI and advised enterprises to restrict Atlas use and monitor AI-driven anomalies. Detection relies on behavioral indicators rather than traditional malware artifacts.