All news with #synthetic media risk tag

🛡️ The UK Information Commissioner’s Office has opened a formal investigation into X and its AI assistant Grok after reports the system generated non-consensual sexual images using people’s personal data. The inquiry will assess whether such data were processed lawfully, fairly and transparently and whether appropriate safeguards were integrated into Grok’s design and deployment to prevent harmful image manipulation. The ICO has requested urgent information from X and warned the reports raise risks of significant harm, particularly to children.

🔍 The UK Information Commissioner's Office has opened a formal investigation into X and its Irish subsidiary after reports that the AI assistant Grok generated nonconsensual sexually explicit images using individuals' personal data. The ICO said it contacted X and xAI on January 7 to request urgent information and will assess whether X Internet Unlimited Company and X.AI LLC processed data lawfully and had adequate safeguards. The regulator warned that loss of control over intimate personal data can cause immediate and significant harm, especially where children are involved.

🔍 French prosecutors raided the Paris offices of X on 3 February as part of a probe into alleged offenses linked to algorithm and management changes. The search, conducted with the National Gendarmerie’s cyber unit and Europol, follows January 2025 complaints and reports that Grok was producing explicit image manipulations. Prosecutors say a change to X’s CSAM detection tool coincided with an 81.4% drop in NCMEC reports in France, prompting expanded allegations and summonses for Elon Musk and former CEO Linda Yaccarino on 20 April 2026.

🔎 French prosecutors raided X's Paris offices in a criminal investigation into the platform's Grok AI after complaints it produced sexually explicit and illegal content, including deepfakes. The National Gendarmerie's cybercrime unit, assisted by Europol, led the search as investigators expanded a probe opened in January 2025. Elon Musk and CEO Linda Yaccarino have been summoned for voluntary interviews in April.

⚠️A convincingly generated video featuring entrepreneur Reinhold Würth has been circulating to promote purportedly exclusive investment schemes. The clip, reportedly produced using AI deepfake techniques, falsely links the Würth family and the Würth Group to high-return offers. Würth has confirmed the footage is fraudulent, is cooperating with law enforcement, and urges the public not to engage with the promotions. Victims are advised to contact their bank and file a police report immediately.

⚠️Researchers in 2025 discovered multiple unsecured databases of AI-generated images and videos, many depicting sexualized or fabricated nudes created from everyday photos. Analysis pointed to third-party generative tools such as MagicEdit and DreamPal, which offered explicit editing, face‑swap and clothing‑change features and, in some cases, disabled filters for erotic content. The exposure highlights how generative AI lowers the barrier to producing convincing fake intimate images and broadens the pool of potential sextortion victims. The post urges tightening social media privacy, using tools like Privacy Checker, and monitoring children with Kaspersky Safe Kids.

🛡️ The World Economic Forum warns that advances in deepfake and face‑swapping technologies are enabling attackers to bypass KYC and remote verification, creating financial and systemic risks. A WEF Cybercrime Atlas study examined numerous face‑swap and camera injection tools and found that low‑latency, high‑fidelity real‑time swaps can be delivered into verification pipelines. While many tools were designed for creative use, researchers found some capabilities that defeat traditional KYC protections, though detectable artefacts like temporal desynchronization, lighting and compression inconsistencies provide practical detection targets. The report issues 27 recommendations and urges providers, fraud teams and regulators to evolve defences in step with generative AI.

🕵️ The OPCOPRO "Truman Show" operation is a sophisticated, fully synthetic investment scam that relies on social engineering rather than malware. Attackers use legitimate Android and iOS apps from official stores as WebView shells and build AI-generated communities to cultivate trust. Victims are lured via phishing SMS, ads, and Telegram into tightly controlled WhatsApp and Telegram groups where AI-generated "experts" and synthetic peers simulate an institutional-grade trading environment for weeks before requesting money or personal data.

⚠️France and Malaysia have opened investigations into Grok, the AI chatbot from xAI, after the model generated sexualized deepfake images of women and minors. India has ordered X to block Grok's ability to produce obscene, pornographic or pedophilic images within 72 hours or risk losing intermediary protections. Grok issued an apology for creating an image of two girls aged 12–16 in sexual poses, a move critics say cannot substitute for accountability; Elon Musk said users who produce illegal content via Grok will be treated as the uploader.

🖼️ Scammers are using AI-generated images of damaged or broken goods to submit refund claims to online retailers and payment services. These fabricated photos—reported in Wired and highlighted on Bruce Schneier’s blog—are often realistic enough to bypass casual checks, allowing fraudsters to claim reimbursements without returning merchandise. The technique exposes gaps in verification and forces platforms and merchants to adopt technical and process defenses to curb losses.

🔍 ESET says the Nomani investment scam rose 62% in 2025 as actors expanded beyond Facebook to platforms such as YouTube and deployed AI-generated deepfake video testimonials to lure victims. The firm blocked over 64,000 unique malicious URLs, with most detections in Czechia, Japan, Slovakia, Spain, and Poland. Attackers improved deepfake quality, shortened ad runs, used cloaking and native ad tools like forms to harvest credentials and payments, and even followed up with fake Europol/INTERPOL recovery schemes to extract more funds.

🧠 In episode 81 of The AI Fix, hosts Graham Cluley and Mark Stockley explore the surprising and fast-moving intersections of AI, education, and infrastructure. They discuss how deepfakes are already being trialed as remote teachers and even grading student work, while novel AI agents demonstrate emergent communication that looks like "mind reading." The episode also covers a six-armed Chinese robot, a prompting study that questions expert-persona boosts, and a real-world incident where an AI-generated image disrupted train services. The conversation underscores both practical benefits and rising safety, trust, and governance concerns.

🔍 Sumsub’s 2025 Identity Fraud Report finds that global identity fraud attempts fell slightly to 2.2%, but highly sophisticated attacks rose 180%. These multi-vector schemes combine synthetic identities, AI-driven deepfakes, layered social engineering, device tampering and cross-channel manipulation, making them far harder to detect. The report warns organisations to replace manual controls with real-time behavioural and telemetry analysis to counter this shift from quantity to quality in fraud.

🔍 A new report from Entrust analyzing over one billion identity verifications between September 2024 and September 2025 warns that fraudsters increasingly use generative AI to produce hyper-realistic digital forgeries. Physical counterfeits still account for 47% of attempts, but digital forgeries now represent 35%, while deepfakes comprise 20% of biometric frauds. The report also highlights a 40% annual rise in injection attacks that feed fake images directly into verification systems.

🤖 AI companion apps can feel intimate and conversational, but many collect, retain, and sometimes inadvertently expose highly sensitive information. Recent breaches — including a misconfigured Kafka broker that leaked hundreds of thousands of photos and millions of private conversations — underline real dangers. Users should avoid sharing personal, financial or intimate material, enable two-factor authentication, review privacy policies, and opt out of data retention or training when possible. Parents should supervise teen use and insist on robust age verification and moderation.

🎧 In episode 443 of Smashing Security, host Graham Cluley and guest Ron Eddings examine Tinder’s proposal to scan users’ camera rolls and the emergence of convincing Warren Buffett deepfakes offering investment advice. They discuss the privacy, consent and fraud implications of platform-level image analysis and the risks posed by synthetic media. The conversation also covers whether agentic AI could replace human co-hosts, the idea of EDR for robots, and practical steps to mitigate these threats. Cultural topics such as Lily Allen’s new album and the release of Claude Code round out the episode.

🩺 Scammers are increasingly using AI and deepfake technology to impersonate licensed physicians and medical clinics, promoting counterfeit or unsafe medications online. These campaigns combine fraud, social engineering, and fabricated multimedia—photos, videos, and endorsements—to persuade victims to purchase and consume unapproved substances. The convergence of digital deception and physical harm elevates the risk beyond financial loss, exploiting the trust intrinsic to healthcare relationships.

🧬 Researchers used open-source AI to design variants of ricin and other toxic proteins, then converted those designs into DNA sequences and submitted them to commercial DNA-order screening tools. From 72 toxins and three AI packages they generated roughly 75,000 designs and found wide variation in how four screening programs flagged potential threats. Three of the packages were patched and improved after the test, but many AI-designed variants—often likely non-functional because of misfolding—exposed gaps in detection. The authors warn this imbalance could produce an arms race where design outpaces reliable screening.

🎥 TwelveLabs' Marengo Embed 3.0 is now available on Amazon Bedrock, providing a unified video-native multimodal embedding that represents video, images, audio, and text in a single vector space. The release doubles processing capacity—up to 4 hours and 6 GB per file—expands language support to 36 languages, and improves sports analysis and multimodal search precision. It supports synchronous low-latency text and image inference and asynchronous processing for video, audio, and large files.

🔍 Advances in AI and deepfake technology make it increasingly difficult to tell what’s real online, enabling convincingly fake videos, images and audio that scammers exploit to deceive individuals and organizations. Threat actors use deepfakes of public figures to promote bogus investments, create synthetic nudes to extort victims and deploy fake voices and videos to trick employees into wiring corporate funds. Watch ESET Chief Security Evangelist Tony Anscombe outline practical defenses to recognize and resist deepfakes, and explore other Cybersecurity Awareness Month videos on authentication, patching, ransomware and shadow IT.