All news with #ai guardrails tag

🛡️ Researchers argue enterprises must stop treating AI agents as trusted components and instead secure them as untrusted systems. The paper, authored by teams from Google, UC San Diego, UW–Madison and others, distills five systems-security principles—least privilege, tamper resistance, complete mediation, secure information flow, and human risk—and maps eleven real-world agent attacks to these violations. They caution that stacking ML guardrails is insufficient and propose research directions for separating instructions from data, verifiable least-privilege policies, and information-flow controls.

🔒 Microsoft has open-sourced two engineering tools—RAMPART and Clarity—to make agent safety a continuous part of development. RAMPART provides a pytest-style framework that brings red-team and adversarial tests into CI, evaluating tools invoked and side effects. Clarity is a structured design companion that captures problem statements, failure analyses, and decisions in a .clarity-protocol directory. Both aim to create living safety artifacts integrated into normal workflows.

🔍 Cloudflare tested security-focused LLMs on its infrastructure and reports detailed findings from using Anthropic’s Mythos Preview as part of Project Glasswing. The model stood out for exploit chain construction and automated proof generation, producing runnable PoCs and iterating on failures. Its emergent guardrails proved inconsistent across runs and prompts, so Cloudflare built a tailored harness and additional safeguards to scale safely. The team also observed higher-quality, actionable findings compared with earlier frontier models, but noted increased noise from memory-unsafe languages and model bias.

🔒 The AWS AI Security Framework presents a structured model that helps security and business leaders align the right controls to the right use case, at the right layer, and at the right phase so AI can move from prototype to production securely. Its core principle is that you build AI on top of security, not add security later. The post maps controls across three layers—infrastructure, identity and data, and AI application—and across four use cases from answering to agentic and physical AI. It highlights Amazon Bedrock and AgentCore as pillars that decouple model choice from security infrastructure.

🔒 Amazon now enables Bedrock AgentCore to apply Chrome Enterprise policies to AgentCore Browser and to accept custom root Certificate Authority (CA) certificates for both AgentCore Browser and Code Interpreter. Administrators can leverage 100+ configurable browser policies — such as URL restrictions, disabling password managers, download controls, and kiosk-mode restrictions — to enforce compliance for AI agents. Custom root CA support permits secure TLS connections to internal services and corporate proxies that use enterprise-signed certificates, helping agents operate within strict security environments.

🔒 Falcon AIDR now integrates with NVIDIA NeMo Guardrails to provide programmable runtime protections for homegrown AI agents moving into production. The combined solution blocks prompt injection, redacts PII, defangs malicious domains, and moderates unwanted topics while preserving responsive, sub-100ms agent workflows. Teams can leverage 75+ built-in detectors or create custom policies to monitor in report-only mode and then progressively enforce blocks, redactions, encryptions, or transformations.

🔒 Researchers at Unit 42, Palo Alto Networks' lab, have demonstrated that LLM-based safety and evaluation systems — called AI Judges — can be manipulated via prompt-injection-style token sequences. Their custom fuzzer, AdvJudge-Zero, probes models in a black-box manner, finding low-perplexity formatting tokens that shift internal attention and increase the likelihood of an 'allow' decision. Unit 42 recorded a 99% bypass rate across multiple architectures, and showed that adversarial retraining on fuzzer-discovered examples can reduce that success rate to near zero.

🔒 Enterprise AI guardrails meant to prevent misuse are increasingly blocking legitimate defensive activity, creating an asymmetry that favors attackers. Widely deployed, enterprise-approved models often refuse realistic phishing simulations, exploit proofs-of-concept, or multi-step red-team scenarios once prompts resemble real-world attacks. Attackers evade these limits using jailbroken models, open-source deployments, fine-tuning, and underground toolkits. The article calls for authorization-based access, purpose-built security sandboxes, and vetting workflows so safety controls protect against misuse without crippling defenders.

🔍 Episode 450 explores confusion after claims that data linked to 17.5 million Instagram accounts was put up for sale — a story driven by a vague post, conflicting statements, and an unexpected flood of password‑reset emails. The episode also examines Grok, Elon Musk’s AI chatbot, after it generated sexualised images of women and children, raising urgent questions about guardrails and accountability. Hosts discuss why simple censorship is not a solution.

🛡️ AI and large language models are transforming cybersecurity into a contest of speed and scale, serving as both best-in-class defensive tools and powerful offensive enablers. Researchers describe self-modifying malware and autonomous espionage that call commercial LLMs (e.g., PROMPTFLUX, PROMPTSTEAL) to adapt tactics mid-execution, while defenders are deploying solutions like XBOW, CodeMender and Watsonx to automate vulnerability discovery, remediation and compliance. CISOs must therefore pair AI-driven defenses with governance and model guardrails to manage this dual-use reality.

🛡️ Vibe coding accelerates development but often omits essential security controls, introducing vulnerabilities, data exfiltration, and destructive actions. Unit 42 documents incidents where AI-generated code bypassed authentication, executed arbitrary commands, deleted production databases, or exposed sensitive identifiers. To mitigate these risks, Unit 42 proposes the SHIELD framework—Separation, Human review, Input/output validation, Enforcer helper models, Least agency, and Defensive controls. Implementing these measures restores governance and enables safer AI-assisted development.

⚠️ Checkmarx research shows Human-in-the-Loop (HITL) confirmation dialogs can be manipulated so attackers embed malicious instructions into prompts, a technique the researchers call Lies-in-the-Loop (LITL). Attackers can hide or misrepresent dangerous commands by padding payloads, exploiting rendering behaviors like Markdown, or pushing harmful text out of view. Approval dialogs meant as a final safety backstop can thus become an attack surface. Checkmarx urges developers to constrain dialog rendering and validate approved operations; vendors acknowledged the report but did not classify it as a vulnerability.

🛡️ Google has added a separate user alignment critic to its Gemini-powered Chrome browsing agent to vet and block proposed actions that do not match user intent. The critic is isolated from web content and sees only metadata about planned actions, providing feedback to the primary planning model when it rejects a step. Google also enforces origin sets to limit where the agent can read or act, requires confirmations for banking, medical, password use and purchases, and runs a classifier plus automated red‑teaming to detect prompt injection attempts during preview.

🔒 Amazon announced that Amazon Nova models now support customizable content moderation settings for approved business use cases that require processing or generating sensitive content. Organizations can adjust controls across four domains—safety, sensitive content, fairness, and security—while Amazon enforces essential, non-configurable safeguards to protect children and preserve privacy. Customization is available for Amazon Nova Lite and Amazon Nova Pro in the US East (N. Virginia) region; customers should contact their AWS Account Manager to confirm eligibility.

🔒 Azure outlines a layered blueprint for building trustworthy, enterprise-grade AI agents. The post emphasizes identity, data protection, built-in controls, continuous evaluation, and monitoring to address risks like data leakage, prompt injection, and agent sprawl. Azure AI Foundry introduces Entra Agent ID, cross-prompt injection classifiers, risk and safety evaluations, and integrations with Microsoft Purview and Defender. Join Microsoft Secure on September 30 to learn about Foundry's newest capabilities.

🛡️ Enterprises adopting agentic AI must update red‑teaming practices to address a rapidly expanding and interactive attack surface. The article summarizes the Cloud Security Alliance’s Agentic AI Red Teaming Guide and corroborating research that documents prompt injection, multi‑agent manipulation, and authorization hijacking as practical threats. It recommends five pragmatic steps—change attitude, continually test guardrails and governance, broaden red‑team skill sets, widen the solution space, and adopt modern tooling—and highlights open‑source and commercial tools such as AgentDojo and Agentgateway. The overall message: combine automated agents with human creativity, embed security in design, and treat agentic systems as sociotechnical operators rather than simple software.

🛡️ Enterprises now face attackers embedding malicious prompts in document macros and hidden metadata to manipulate generative AI systems that parse files. Researchers and vendors have identified exploits — including EchoLeak and CurXecute — and a June 2025 Skynet proof-of-concept that target AI-powered parsers and malware scanners. Experts urge layered defenses such as deep file inspection, content disarm and reconstruction (CDR), sandboxing, input sanitization, and strict model guardrails to prevent AI-driven misclassification or data exposure.

🛡️Cloudflare extended its AI agent Cloudy to generate clear, concise explanations for email security detections so SOC teams can understand why messages are blocked. Early LLM implementations produced dangerous hallucinations when asked to interpret complex, multi-model signals, so Cloudflare implemented a Retrieval-Augmented Generation approach and enriched contextual prompts to ground outputs. Testing shows these guardrails yield more reliable summaries, and a controlled beta will validate performance before wider rollout.

🔐 Palo Alto Networks and Portkey have integrated Prisma AIRS directly into Portkey’s AI gateway to embed security guardrails at the gateway level. The collaboration aims to protect applications from AI-specific threats—such as prompt injections, PII and secret leakage, and malicious outputs—while preserving Portkey’s operational benefits like observability and cost controls. A one-time configuration via Portkey’s Guardrails module enforces protections without code changes, and teams can monitor posture through Portkey logs and the Prisma AIRS dashboard.

🔒 Microsoft outlines a layered defense-in-depth strategy to protect systems using LLMs from indirect prompt injection attacks. The approach pairs preventative controls such as hardened system prompts and Spotlighting (delimiting, datamarking, encoding) to isolate untrusted inputs with detection via Microsoft Prompt Shields, surfaced through Azure AI Content Safety and integrated with Defender for Cloud. Impact mitigation uses deterministic controls — fine-grained permissions, Microsoft Purview sensitivity labels, DLP policies, explicit user consent workflows, and blocking known exfiltration techniques — while ongoing research (TaskTracker, LLMail-Inject, FIDES) advances new design patterns and assurances.