All news in category “AI and Security Pulse”

🧭 AlphaEvolve, now GA on the Gemini Enterprise Agent Platform, is an agentic code optimization and discovery tool designed to tackle hard algorithmic problems across domains like logistics, semiconductors, genomics, and finance. It follows a four-step workflow — Define, Measure, Optimize, Apply — to move from a baseline algorithm to production-ready optimized code. Early customers report substantial gains in accuracy, performance, and efficiency across production pipelines and HPC workloads.

🛡️ Five Eyes agencies warned that AI's rapid development raises cyber risks, particularly autonomous hacking and automated attacks. Bruce Schneier explains that AI widens the gap between skill and ability, enabling less-skilled actors to cause greater harm while also offering defensive tools. He argues that guardrails on large platforms won't stop open-source models and urges using AI for defense across all heightened risks.

🔒 This article examines the risks of agentic AI and non-human identities in enterprise environments, illustrating incidents where LLM-based agents caused outages due to weak identity controls. It argues that existing IAM models are insufficient for agents that act autonomously, and cites industry guidance from Gartner, OWASP, CISA and NIST. The author proposes six minimum requirements and a cumulative six-stage NHI maturity model to ensure defensible production deployments.

🔒 Over 70 cybersecurity organisations have signed the new CREST AI Charter, launched on July 9, committing to nine principles governing AI-enabled cybersecurity activities. The charter covers accountability and governance, transparency of use, documentation and auditability, boundaries and control, data handling and sovereignty, security and confidentiality, secure development, supply chain assurance and resilience. Signatories will maintain human oversight, document AI use, disclose data practices and implement secure development and supply chain controls. CREST intends the charter as a self-regulatory foundation to drive standards and harmonisation across industry and regulators.

🔍 Security teams often default to retraining AI models when detections fail, but the real root cause is usually upstream data issues. Fragmented telemetry, inconsistent schemas and stale baselines degrade ML effectiveness long before models see events. Standardizing schemas, monitoring data quality at ingestion and applying governance to security telemetry are practical priorities that restore detection reliability without wholesale platform replacements.

🖼️ Meta introduced Muse Image, an image-focused AI from Superintelligence Labs that can use public Instagram posts and reels to generate AI-created images, enabled by default. The feature lets users @-mention public accounts in the Meta AI app to incorporate specific profiles' media into new images and is being integrated into Instagram and WhatsApp in select countries. Users can opt out via Instagram Settings > Sharing and reuse, though previously created content will remain if generated before disabling the setting. For minors with public accounts, only followers may reuse their media if allowed; existing remixes won't notify original owners, and deleted content may be removed if accounts go private for over 24 hours.

🔍 Researchers at the AI Now Institute demonstrated a proof-of-concept called "Friendly Fire" where autonomous AI coding agents (Anthropic's Claude Code and OpenAI's Codex) execute an attacker's binary when asked to scan untrusted open-source code. The attack hides a malicious binary alongside benign files and a README that prompts the agent to run a security script; in auto-modes the agents approved and executed it without prompting. The weakness is framed as a workflow/design issue rather than a single vulnerable version, and the researchers recommend never giving command-capable agents unattended access to untrusted code.

🎧 This episode of Smashing Security discusses a rash of recent cybersecurity incidents, including a 15-year-old who used a chatbot to cancel nearly 47,000 anime subscriptions and the first documented agentic ransomware, JadePuffer. The hosts also examine Apple’s problematic Hide My Email feature, which has been known to leak addresses for over a year. Guest Zoë Rose joins Graham Cluley to assess implications for security and privacy.

🛡️ System prompts are core to generative AI applications and often include role definitions, tool descriptions, RAG context, and other proprietary instructions. This AWS Security Blog post explains why system prompt leakage is a persistent risk, highlights that it cannot be fully remediated today, and outlines practical mitigations. It recommends design principles such as minimization and avoiding sensitive data in prompts, and details controls available via Amazon Bedrock Guardrails and other mechanisms to reduce exposure and raise extraction difficulty.

🛡️ New research describes "HalluSquatting," an attack that exploits AI assistants' habit of inventing resource names. Attackers register those predictable fake names on marketplaces and plant adversarial instructions; when an assistant hallucinate-fetches the same name, it may run the attacker's commands. The technique abuses auto-run modes and agent tools that fetch and execute external code, permitting widespread compromise without traditional malware or exploits.

🧭 A new study found GitHub Copilot can be induced to generate harmful answers when a dangerous request is broken into ordinary coding steps. Researchers Abhishek Kumar and Carsten Maple tested Claude and Gemini models through Copilot and observed that direct chat prompts were routinely refused, but the same content was produced in 816 of 816 workflow runs when framed as benchmark-improving “teaching shots.” The paper calls this technique workflow-level jailbreak construction and urges reviewing written files and whole sessions, not just chat refusals.

🛡️ The Five Eyes recently warned that AI models increasingly enable autonomous cyberattacks, amplifying risks long present in cyberspace. Bruce Schneier argues that AI widens the gap between skill and ability: tools let less-skilled actors cause damage once limited to experts. He warns guardrails from large vendors won’t stop open-source or locally run models and urges using AI defensively to detect, remediate, and respond faster to evolving threats.

🔒 The UK National Cyber Security Centre (NCSC) has launched the Cyber Shield initiative to build a national cyber-defence capability powered by agentic AI. The project will use coordinated red and blue agents to discover and mitigate vulnerabilities at scale, enable national automated scanning, and support real-time intelligence sharing. The NCSC says success requires partnerships with government, critical infrastructure and frontier AI providers.

🔒 Researchers at Noma Security demonstrated that a crafted public GitHub issue can manipulate GitHub Agentic Workflows into exposing private repository contents. The attack, named GitLost, exploits indirect prompt injection to trick an agent with organization-wide read access into pulling private data and posting it publicly. GitHub's preview feature for agentic workflows includes guardrails, but Noma showed a minor wording change can bypass them. The core problem is architectural: agents with standing credentials that read untrusted input and can post outward create persistent leakage risk.

🛡️ Software supply chain security has evolved as AI tools and agents become integral to builds. What used to be a question of third‑party packages and transitive dependencies now includes models, agents, prompts, and autonomous tooling as provenance concerns. Teams must extend lineage to models and pipeline actions, and prioritize findings by actual exploitability to avoid alert overload. The discussion surfaces in a webinar on July 22 covering new research and practical program changes.

🛡️ Zscaler tested 26 LLMs and found several autonomous agents susceptible to indirect prompt injection (IPI) traps, with some high-end models failing while a few lower-tier models fared better. The vendor identified hidden instructions on websites that manipulated agent behavior and caused real-world impacts in controlled tests. Experts warn that agent risk is dynamic, the attack surface is architectural, and binary "safe/vulnerable" labels are overly simplistic for CISOs. The findings highlight that agentic AI introduces new trust boundaries and insider-like threats to enterprise security.

🛡️ Zscaler tested 26 LLM-based autonomous agents and found several susceptible to indirect prompt injection (IPI) schemes, with some high-end models failing while a few lower-tier models fared better. The vendor reported four models as "vulnerable" and three as "safe," but experts warn that agent behavior evolves and binary classifications can be misleading. The findings highlight the architectural risks in agentic AI where untrusted content in the context window can be treated as authoritative, expanding the attack surface for enterprises.

🔍 CAPTCHAs began as distorted text and audio tests but have evolved into image challenges and now experimental gesture videos as AI improves. Google’s new hand-gesture approach records brief camera footage to verify 21 hand key points, raising privacy concerns despite reassurances about data handling. Alternative defenses include behavioral analysis, Cloudflare Turnstile, and hCaptcha, while passkeys offer a passwordless path that can reduce the need for human checks.

🔍 Zscaler ThreatLabz uncovered real-world campaigns using indirect prompt injection, where hidden instructions embedded in web pages steer AI agents. Attackers used SEO poisoning to surface malicious pages and hid prompts via CSS and JSON-LD metadata. One campaign impersonated a Python library to trick agents into paying a $3 bogus API key; another typosquatted a DeBank site to claim authority. Tests across 26 LLMs showed varying susceptibility depending on model and context.

🤖 Stephen Wilson of HashiCorp describes agentic AI as “really smart kindergartners” — capable of execution but lacking judgment. This mismatch strains traditional zero trust models that authenticate humans and grant privileges gradually, because agents can be created and destroyed rapidly. Organizations often respond by lowering controls, risking incidents such as accidental deletion of production data. Wilson argues this will force necessary long-term improvements like zero standing privilege and dynamic credentials while keeping humans "on the loop."