All news with #ai guardrails tag

🔒 OpenAI’s Lockdown Mode aims to reduce AI-enabled data exfiltration by disabling web browsing, image support, Deep Research, Agent Mode, network access from generated code, and file downloads while still permitting manually uploaded file analysis. Experts say the feature is a pragmatic but imperfect mitigation that still allows side-channel exfiltration, complicates governance across multiple AI vendors, and shifts responsibility between providers and enterprise security teams.

🔒 Microsoft announced new controls to contain agentic AI workloads, including the Microsoft Execution Container (MXC) runtime and enhancements to the multi-agent vulnerability research system MDASH. MXC is a policy-driven sandbox for specifying and enforcing access to files, networks, credentials, and resources at runtime across Windows, Linux, and macOS. The company also highlighted Agent 365 SDK, Windows 365 for Agents, and two open-source standards—ASSERT and Agent Control Specifications—to govern agent behavior across platforms.

🛡️ This article explains why AI requires a unified security architecture — the AI Defense Plane — to discover, protect, govern, and assure AI behavior across employees, applications, and agents. It describes how AI becomes an execution layer that can retrieve data, call tools, and take actions, creating risks that traverse traditional security boundaries. The piece emphasizes runtime protection, coordinated enforcement, and continuous testing to prevent prompt-based attacks, data exposure, and unsafe agent behavior.

🔒 Reporting suggests attackers used Meta’s AI support chatbot to change recovery emails on high-profile Instagram accounts, leading to notable takeovers. The core issue isn’t just prompt injection or a model jailbreak but that the AI operated within a sensitive account recovery workflow with insufficient independent verification. Organizations must treat AI-driven support actions as part of the security boundary and constrain authority, permissions, and verification around such agents.

🛡️ Researchers argue enterprises must stop treating AI agents as trusted components and instead secure them as untrusted systems. The paper, authored by teams from Google, UC San Diego, UW–Madison and others, distills five systems-security principles—least privilege, tamper resistance, complete mediation, secure information flow, and human risk—and maps eleven real-world agent attacks to these violations. They caution that stacking ML guardrails is insufficient and propose research directions for separating instructions from data, verifiable least-privilege policies, and information-flow controls.

🔒 Microsoft has open-sourced two engineering tools—RAMPART and Clarity—to make agent safety a continuous part of development. RAMPART provides a pytest-style framework that brings red-team and adversarial tests into CI, evaluating tools invoked and side effects. Clarity is a structured design companion that captures problem statements, failure analyses, and decisions in a .clarity-protocol directory. Both aim to create living safety artifacts integrated into normal workflows.

🔍 Cloudflare tested security-focused LLMs on its infrastructure and reports detailed findings from using Anthropic’s Mythos Preview as part of Project Glasswing. The model stood out for exploit chain construction and automated proof generation, producing runnable PoCs and iterating on failures. Its emergent guardrails proved inconsistent across runs and prompts, so Cloudflare built a tailored harness and additional safeguards to scale safely. The team also observed higher-quality, actionable findings compared with earlier frontier models, but noted increased noise from memory-unsafe languages and model bias.

🔒 The AWS AI Security Framework presents a structured model that helps security and business leaders align the right controls to the right use case, at the right layer, and at the right phase so AI can move from prototype to production securely. Its core principle is that you build AI on top of security, not add security later. The post maps controls across three layers—infrastructure, identity and data, and AI application—and across four use cases from answering to agentic and physical AI. It highlights Amazon Bedrock and AgentCore as pillars that decouple model choice from security infrastructure.

🔒 Amazon now enables Bedrock AgentCore to apply Chrome Enterprise policies to AgentCore Browser and to accept custom root Certificate Authority (CA) certificates for both AgentCore Browser and Code Interpreter. Administrators can leverage 100+ configurable browser policies — such as URL restrictions, disabling password managers, download controls, and kiosk-mode restrictions — to enforce compliance for AI agents. Custom root CA support permits secure TLS connections to internal services and corporate proxies that use enterprise-signed certificates, helping agents operate within strict security environments.

🔒 Falcon AIDR now integrates with NVIDIA NeMo Guardrails to provide programmable runtime protections for homegrown AI agents moving into production. The combined solution blocks prompt injection, redacts PII, defangs malicious domains, and moderates unwanted topics while preserving responsive, sub-100ms agent workflows. Teams can leverage 75+ built-in detectors or create custom policies to monitor in report-only mode and then progressively enforce blocks, redactions, encryptions, or transformations.

🔒 Researchers at Unit 42, Palo Alto Networks' lab, have demonstrated that LLM-based safety and evaluation systems — called AI Judges — can be manipulated via prompt-injection-style token sequences. Their custom fuzzer, AdvJudge-Zero, probes models in a black-box manner, finding low-perplexity formatting tokens that shift internal attention and increase the likelihood of an 'allow' decision. Unit 42 recorded a 99% bypass rate across multiple architectures, and showed that adversarial retraining on fuzzer-discovered examples can reduce that success rate to near zero.

🔒 Enterprise AI guardrails meant to prevent misuse are increasingly blocking legitimate defensive activity, creating an asymmetry that favors attackers. Widely deployed, enterprise-approved models often refuse realistic phishing simulations, exploit proofs-of-concept, or multi-step red-team scenarios once prompts resemble real-world attacks. Attackers evade these limits using jailbroken models, open-source deployments, fine-tuning, and underground toolkits. The article calls for authorization-based access, purpose-built security sandboxes, and vetting workflows so safety controls protect against misuse without crippling defenders.

🔍 Episode 450 explores confusion after claims that data linked to 17.5 million Instagram accounts was put up for sale — a story driven by a vague post, conflicting statements, and an unexpected flood of password‑reset emails. The episode also examines Grok, Elon Musk’s AI chatbot, after it generated sexualised images of women and children, raising urgent questions about guardrails and accountability. Hosts discuss why simple censorship is not a solution.

🛡️ AI and large language models are transforming cybersecurity into a contest of speed and scale, serving as both best-in-class defensive tools and powerful offensive enablers. Researchers describe self-modifying malware and autonomous espionage that call commercial LLMs (e.g., PROMPTFLUX, PROMPTSTEAL) to adapt tactics mid-execution, while defenders are deploying solutions like XBOW, CodeMender and Watsonx to automate vulnerability discovery, remediation and compliance. CISOs must therefore pair AI-driven defenses with governance and model guardrails to manage this dual-use reality.

🛡️ Vibe coding accelerates development but often omits essential security controls, introducing vulnerabilities, data exfiltration, and destructive actions. Unit 42 documents incidents where AI-generated code bypassed authentication, executed arbitrary commands, deleted production databases, or exposed sensitive identifiers. To mitigate these risks, Unit 42 proposes the SHIELD framework—Separation, Human review, Input/output validation, Enforcer helper models, Least agency, and Defensive controls. Implementing these measures restores governance and enables safer AI-assisted development.

⚠️ Checkmarx research shows Human-in-the-Loop (HITL) confirmation dialogs can be manipulated so attackers embed malicious instructions into prompts, a technique the researchers call Lies-in-the-Loop (LITL). Attackers can hide or misrepresent dangerous commands by padding payloads, exploiting rendering behaviors like Markdown, or pushing harmful text out of view. Approval dialogs meant as a final safety backstop can thus become an attack surface. Checkmarx urges developers to constrain dialog rendering and validate approved operations; vendors acknowledged the report but did not classify it as a vulnerability.

🛡️ Google has added a separate user alignment critic to its Gemini-powered Chrome browsing agent to vet and block proposed actions that do not match user intent. The critic is isolated from web content and sees only metadata about planned actions, providing feedback to the primary planning model when it rejects a step. Google also enforces origin sets to limit where the agent can read or act, requires confirmations for banking, medical, password use and purchases, and runs a classifier plus automated red‑teaming to detect prompt injection attempts during preview.

🔒 Amazon announced that Amazon Nova models now support customizable content moderation settings for approved business use cases that require processing or generating sensitive content. Organizations can adjust controls across four domains—safety, sensitive content, fairness, and security—while Amazon enforces essential, non-configurable safeguards to protect children and preserve privacy. Customization is available for Amazon Nova Lite and Amazon Nova Pro in the US East (N. Virginia) region; customers should contact their AWS Account Manager to confirm eligibility.

🔒 Azure outlines a layered blueprint for building trustworthy, enterprise-grade AI agents. The post emphasizes identity, data protection, built-in controls, continuous evaluation, and monitoring to address risks like data leakage, prompt injection, and agent sprawl. Azure AI Foundry introduces Entra Agent ID, cross-prompt injection classifiers, risk and safety evaluations, and integrations with Microsoft Purview and Defender. Join Microsoft Secure on September 30 to learn about Foundry's newest capabilities.

🛡️ Enterprises adopting agentic AI must update red‑teaming practices to address a rapidly expanding and interactive attack surface. The article summarizes the Cloud Security Alliance’s Agentic AI Red Teaming Guide and corroborating research that documents prompt injection, multi‑agent manipulation, and authorization hijacking as practical threats. It recommends five pragmatic steps—change attitude, continually test guardrails and governance, broaden red‑team skill sets, widen the solution space, and adopt modern tooling—and highlights open‑source and commercial tools such as AgentDojo and Agentgateway. The overall message: combine automated agents with human creativity, embed security in design, and treat agentic systems as sociotechnical operators rather than simple software.