All news in category “AI and Security Pulse”

⚠️ Checkmarx research shows Human-in-the-Loop (HITL) confirmation dialogs can be manipulated so attackers embed malicious instructions into prompts, a technique the researchers call Lies-in-the-Loop (LITL). Attackers can hide or misrepresent dangerous commands by padding payloads, exploiting rendering behaviors like Markdown, or pushing harmful text out of view. Approval dialogs meant as a final safety backstop can thus become an attack surface. Checkmarx urges developers to constrain dialog rendering and validate approved operations; vendors acknowledged the report but did not classify it as a vulnerability.

🔒 Within the past year AI copilots and agents have been embedded across major SaaS like Zoom, Slack, Microsoft 365, Salesforce, and ServiceNow, creating dynamic cross-app data flows that traditional governance struggles to monitor. A dynamic AI-SaaS security layer functions as an adaptive guardrail over OAuth grants and integrations, logging prompts and file access, detecting permission drift in real time, and blocking risky actions. Platforms such as Reco aim to deliver continuous visibility, end-to-end auditability, and automated policy enforcement so organizations can adopt copilots without losing control.

🤖 The Future Report, based on responses from over 7,000 European teenagers, finds young people largely optimistic and adept at using AI and algorithmic platforms in daily life. Many report educational benefits—47% say AI explains complex topics, and 81% of users feel it improved aspects of learning or creativity—while also expressing concerns about over-reliance, trust, and skill erosion. The report calls for strengthened digital literacy, age-appropriate experiences, and youth participation in shaping responsible AI design.

🔍 Fortinet and academic partners, including UC Berkeley’s CLTC and the Berkeley Risk and Security Lab, collaborated on global tabletop exercises and analysis to assess how AI is reshaping cybercrime. The Singapore TTX demonstrated that AI amplifies existing attack vectors—speeding reconnaissance, phishing, and malware development—while lowering barriers to entry and fostering criminal specialization. Defenders reported that governance, decision rights, and human judgment often mattered more than specific tools, underscoring the need for strong public-private collaboration and human oversight of AI-assisted detection.

⚠️ Security researchers at Checkmarx disclosed a novel technique called Lies-in-the-Loop (LITL) that manipulates Human-in-the-Loop (HITL) confirmation dialogs to trigger arbitrary code execution. The attack forges or alters dialog text, metadata and Markdown rendering so that dangerous commands appear benign, effectively turning a safety checkpoint into an exploit vector. Demonstrations targeted privileged code-assistant tools including Claude Code and Copilot Chat, and the authors urge a defense-in-depth approach combining user training, improved dialog clarity and input sanitization.

🔎 Fortinet and UC Berkeley partners analyzed a Singapore tabletop exercise to assess how AI is reshaping cybercrime and defense. The practitioner perspective complements CLTC’s academic work and shows AI is amplifying existing attack vectors—speeding phishing, reconnaissance, code generation, and malware iteration—while lowering barriers to entry. The exercise highlighted that governance, human judgment, and cross-sector collaboration frequently determine response effectiveness more than specific tools.

🔒 Security researchers have found that the popular Chrome extension Urban VPN Proxy (featured in the Chrome Web Store and used by millions) contained scripts that intercepted AI chat conversations and transmitted them to company-controlled analytics servers. The functionality, introduced in version 5.5.0 on July 9, 2025, allegedly runs regardless of whether the VPN is active and cannot be disabled via settings. Koi's analysis says prompts, responses, timestamps and session identifiers were captured and compressed before exfiltration. The same capability was reportedly present in seven related extensions from the same publisher, potentially affecting more than 8 million users across Chrome and Edge.

🔐 This article examines the security and governance challenges of generative AI and outlines practical steps organizations can take to reduce risk. It highlights model limitations such as hallucinations and underscores the continued need for human oversight for high‑stakes decisions. The author reviews prominent standards including NIST AI RMF, AICM and CSA Model Risk Management, and stresses cloud shared‑responsibility, cross‑team governance, and targeted workforce training as core mitigations.

🧠 In episode 81 of The AI Fix, hosts Graham Cluley and Mark Stockley explore the surprising and fast-moving intersections of AI, education, and infrastructure. They discuss how deepfakes are already being trialed as remote teachers and even grading student work, while novel AI agents demonstrate emergent communication that looks like "mind reading." The episode also covers a six-armed Chinese robot, a prompting study that questions expert-persona boosts, and a real-world incident where an AI-generated image disrupted train services. The conversation underscores both practical benefits and rising safety, trust, and governance concerns.

🔒 Researchers at Koi found that the popular Urban VPN Proxy browser extension injects scripts to capture full AI chat conversations — including prompts and responses — then exports them to the extension vendor's backend. The monitoring runs even when the VPN is disabled and activates on major platforms such as ChatGPT, Claude, Gemini, Perplexity and Grok. For organizations that paste internal code, data or research into AI tools, this creates a significant data-theft risk outside corporate controls.

🛡️ Microsoft has been named an overall leader in the KuppingerCole Leadership Compass for Generative AI Defense, highlighting its enterprise-ready security and governance capabilities for AI. The company emphasizes embedding security across AI apps, agents, platforms, and infrastructure using an identity-first, defense-in-depth approach. Key controls include Entra Agent ID, Microsoft Purview for real-time DLP and classification, Microsoft Defender for runtime protection, and governance tools such as Agent365 and Foundry. Built-in compliance support aligns with frameworks like EU AI Act, NIST AI RMF, and ISO 42001.

🔍 AI is reshaping cybersecurity: while models speed detection and automate response, human judgment and communication are the differentiators. CISOs must cultivate data fluency, risk literacy, executive communication, cross-functional collaboration and ethical foresight. Practical steps include regular AI bias audits, joint security/data-science sprints and measuring data-storytelling maturity to align AI with business risk.

🔐 Check Point argues that modern AI's effectiveness hinges on the volume, variety, and freshness of data, and that its three decades of aggregated threat intelligence provide a practical advantage in applying AI to cybersecurity. The post highlights data density — the combination of scale, diversity, and timeliness of telemetry — as the primary driver of model accuracy and detection efficacy. It contrasts five years of explosive AI data growth with Check Point's 30-year corpus and explains how rich telemetry enables better prediction, prevention, and operationalization of AI-driven defenses.

🔍 This article outlines a practical, metrics-driven approach to testing generative AI systems, moving teams from ad-hoc inspection to systematic evaluation. It introduces four hands-on labs that cover evaluating single LLM outputs, assessing RAG systems with Vertex AI Evaluation, tracing and grading agent behavior with the Agent Development Kit (ADK), and validating SQL-generating agents against BigQuery. Each lab emphasizes measurable metrics—safety, groundedness, faithfulness, and factual accuracy—to help productionize GenAI with confidence.

🔒 The Cybersecurity Forecast for 2026 highlights how agentic security automation and widespread AI will reshape defenses, shifting SOCs from monitoring to automated action. It calls for building workforce AI fluency, evolving IAM to treat agents as managed identities, and deploying model-protection measures alongside tamper-proof backups. Boards will increasingly demand operational resilience, quantified exposure, and mature AI governance.

🔐 Bruce Schneier warns that personal AI assistants cannot be trusted without robust integrity controls, arguing that current systems routinely push users toward bad outcomes, gaslight them, and mishandle personal context. He urges decoupling personal data stores from AI models so that cryptographic verification, access control, and auditable logs can be developed independently of model performance. Such stores should be interoperable with many models, provably accurate, under fine‑grained user control, resilient to read and write attacks, and easy to use; Schneier cites Inrupt work extending Solid and the Human Context Protocol as practical directions.

🔒 The article argues that the browser is now the primary interface for enterprise GenAI and outlines a practical security model combining policy, isolation, and precision data controls. It recommends categorizing GenAI services into sanctioned and public tools, enforcing SSO for corporate identities, and preventing cross‑account leakage. The piece highlights the risks of prompt copy/paste, file uploads, and extensions, and advises per‑site/session controls, telemetry, and a pragmatic 30‑day Secure Enterprise Browser (SEB) rollout to enable safe, productive use.

🛡️ OpenAI says it is expanding a "defense in depth" strategy to limit misuse of its frontier AI models, warning they could be used to develop zero-day exploits or aid complex intrusion operations. The company announced a new Frontier Risk Council, broader guardrails, external red‑teaming, and a trusted access program for vetted customers testing defensive use cases. OpenAI also plans to scale its Aardvark Agentic Security Researcher beta to scan codebases and recommend mitigations.

🤖 Brave has begun testing an agentic AI browsing mode that uses its privacy-focused assistant Leo to perform autonomous tasks like web research, product comparison, promo-code discovery, and news summarization. The feature is currently available in Brave Nightly and is disabled by default. Brave isolates the agent in a separate profile without access to cookies, logins, or sensitive data and adds restrictions plus an alignment checker to mitigate prompt-injection and other risks.

🔍 Researchers used a new benchmark, SCONE-bench, to train AI agents to find and produce exploits against historically compromised smart contracts. On 405 real-world contracts from 2020–2025, Claude Opus 4.5, Claude Sonnet 4.5, and GPT-5 generated exploits valued at $4.6 million. In simulated tests against 2,849 recently deployed contracts the agents discovered two novel zero-day vulnerabilities and created exploits worth $3,694, with GPT-5 incurring $3,476 in API costs. The findings show autonomous, profitable exploitation is technically feasible and emphasize the need for proactive AI-driven defense.