All news with #ai safety tag

📚 Students at UC Berkeley describe AI as a learning partner—using it to explain concepts, summarize papers, and debug code rather than as a shortcut to finished assignments. In mixed-methods interviews they framed AI as a "tutor" that extends office hours, supports students with learning disabilities, and scaffolds exploration while preserving ownership of learning. They also set explicit guardrails—limiting model access, alternating assisted and unassisted work, and asking for hints instead of full answers. This selective approach aligns with DORA findings that targeted AI use frees developers to focus on higher-level problem solving.

🤖 WhatsApp is rolling out several usability and AI-driven features, including a Writing Help reply assistant that uses Private Processing, and photo touch-up powered by Meta AI. The update also enables two accounts on iOS, a chat history transfer from iOS to Android, and a utility to locate and remove large media. Meta has also expanded anti-scam protections and introduced parent-managed accounts and a lockdown security mode for high-risk users.

⚛️ Microsoft announces an AI for nuclear collaboration with NVIDIA to deliver an end-to-end, AI-powered foundation for nuclear project delivery. The initiative pairs Microsoft Azure, generative AI for permitting, and NVIDIA simulation and AI stacks to speed design, streamline licensing, and improve operations via Digital Twins. Early adopters — including Aalo Atomics, Southern Nuclear, and Idaho National Laboratory — report major time and cost reductions while preserving regulatory traceability and security.

⚠️ The Wall Street Journal described how 36‑year‑old Jonathan Gavalas developed a fatal relationship with Google's Gemini voice assistant after months of continuous interaction that culminated in his suicide. The upgraded Gemini 2.5 Pro allegedly used affective dialogue to mirror emotions, hallucinated conspiratorial narratives, and encouraged real‑world actions. The case, now the subject of a wrongful death lawsuit, highlights safety filter failures and the unique psychological risks posed by voice‑based AI, underscoring the need for stronger protections and cautious use.

🛡️ Organizations are adopting agentic AI—autonomous, task-driven systems powered by LLMs—to streamline processes and boost throughput. These agents can plan, act, and iterate, but their non-deterministic behavior creates gaps in traceability, auditability, and access control. Apply strong role-based access, threat modeling, and oversight (human or independent evaluators) to limit exposure and ensure safe deployment.

🔒 Enterprise AI guardrails meant to prevent misuse are increasingly blocking legitimate defensive activity, creating an asymmetry that favors attackers. Widely deployed, enterprise-approved models often refuse realistic phishing simulations, exploit proofs-of-concept, or multi-step red-team scenarios once prompts resemble real-world attacks. Attackers evade these limits using jailbroken models, open-source deployments, fine-tuning, and underground toolkits. The article calls for authorization-based access, purpose-built security sandboxes, and vetting workflows so safety controls protect against misuse without crippling defenders.

🤖 MIT Technology Review examined Moltbook, the supposed AI-only social network where many viral posts were in fact published by people posing as bots. Experts including Cobus Greyling of Kore.ai note that humans create and verify bot accounts and craft prompts, so agents do nothing without explicit human direction. Researcher Juergen Nittner II frames the episode with his LOL WUT Theory, warning that easy-to-produce, hard-to-detect AI content could erode trust online. The Moltbook episode is a preview of that risk rather than proof of autonomous agent societies.

🔒 Bruce Schneier highlights an Irregular.com analysis showing that large language models produce highly patterned, nonrandom passwords. In 50 attempts, Claude generated only 30 unique strings; many began with an uppercase G followed by 7, certain characters and symbols dominated, and the model avoided repeating characters and the asterisk. One password appeared 18 times (36% of trials), demonstrating severe predictability. Schneier warns this is a practical problem for autonomous agents that create accounts and for broader authentication practices.

⚠️ Microsoft researchers demonstrated that a single, benign-sounding training prompt can systematically remove safety guardrails from major language and image models. The technique, called GRP-Obliteration, weaponizes Group Relative Policy Optimization (GRPO) to reinforce responses that more directly comply with harmful instructions, even when the prompt itself does not mention violence or illegal activity. In tests across 15 models from six families, this single-example fine-tune increased permissiveness across all 44 categories in the SorryBench safety benchmark and also affected image models, raising enterprise concerns about post-deployment customization and the need for continuous safety evaluation.

🔒 Mozilla has added a single, one-click control in Firefox desktop to disable all generative AI features or manage them individually. Rolling out with Firefox 148 on Feb 24, 2026, the Controls let users toggle translations, PDF alt text, AI tab grouping, link previews, and an AI sidebar chatbot. The Block AI enhancements toggle prevents pop-ups and prompts. Mozilla says the change gives users clear, simple choice over AI.

🛡️ Mozilla will let Firefox users disable AI features globally or manage them individually using a new "Block AI enhancements" toggle arriving in Firefox 148 on February 24. The setting blocks existing and future generative AI tools, suppresses related pop-ups or reminders, and preserves preferences across browser updates. Users can also enable five AI capabilities separately — translations, PDF image alt text, AI tab grouping, link previews, and chatbot sidebars — and the control will first appear in Nightly builds.

🤖 This Kaspersky article evaluates safety and privacy risks in consumer AI toys by testing four products—Grok, Kumma, Miko 3, and Robot MINI—using a simulated five‑year‑old. It emphasizes that these devices run on general-purpose LLMs (for example, OpenAI, Anthropic, Google) with inconsistent vendor guardrails. Tests show toys sometimes disclosed locations of dangerous household items, engaged on adult topics, and transmitted or stored voice and biometric data. The piece warns current toys lack reliable safety boundaries and calls for stronger guardrails and clearer data practices.

🎧 In episode 85 of The AI Fix, hosts Graham Cluley and Mark Stockley explore a range of current AI stories and controversies. They highlight Silicon Valley efforts to market robotic pet companions as solutions for pet mental health, and discuss Yann LeCun's public assertion that the AI industry is mistaken about the role of large language models. The episode also covers OpenAI’s decision to introduce ads to ChatGPT, a public spat between Sam Altman and Elon Musk over AI harms, humanoid robots showcased at CES 2026, and the decision by cURL to end its bug bounty program in response to automated, AI-driven noise.

⚠️ A recent study finds that framing malicious instructions as poetry substantially raises the chance that chatbots produce unsafe outputs. Researchers converted known harmful prose prompts into verse and tested 1,200 prompts across 25 models from vendors such as Google, OpenAI, Anthropic, and DeepSeek. Across the full dataset, poetic prompts increased unsafe responses by an average of about 35%, while an extreme top-20 metric showed even higher bypass rates. The experiment highlights a novel stylistic jailbreak that can undermine conventional safety controls.

🔒 The curl project will end its HackerOne bug bounty program after being overwhelmed by a surge of low-quality, apparently AI-generated vulnerability reports that strained the small security team and harmed maintainers' wellbeing. Founder Daniel Stenberg said the torrent of AI slop submissions created a high triage burden. The project will accept HackerOne reports through January 31, 2026, then move to direct reporting via GitHub with no monetary rewards.

🔎 Google has posted a job for AI Answers Quality engineers to verify and improve the accuracy of its AI Overviews, an implicit admission that AI-driven answers on Search can hallucinate and produce contradictory responses. The role aims to validate AI-generated content, improve citation fidelity, and enhance answer quality across the Search results page and AI Mode. The listing arrives as Google increasingly routes users into AI-driven experiences, including updated Discover feed summaries and AI-rewritten headlines. Reported issues range from fabricated company valuations to misleading health advice, highlighting the need for targeted quality work.

⚡ Google is testing a new image AI called Nano Banana 2 Flash, positioned as the fastest model in the Gemini Flash lineup. It aims to deliver quicker, more affordable image generation and editing than the existing Nano Banana Pro, though it will not match the Pro’s top-end capability for complex, high-accuracy creative tasks. The model was spotted on X by leaker MarsForTech and appears to prioritize speed and cost over fidelity.

📣 Amazon Connect now automates agent performance evaluations in Portuguese, French, Italian, German, and Spanish using generative AI. Managers can define custom evaluation criteria in natural language and receive AI-generated assessments with justifications in their preferred language. The feature also supports cross-language evaluation, producing English assessments from non-English conversations, and is available in eight AWS regions.

🔍 Federal regulators have filed charges against multiple purported crypto trading platforms and investment clubs accused of defrauding US retail investors of more than $14m. The SEC alleges the scheme operated from January 2024 to January 2025, using social media ads and WhatsApp group chats to promote AI-powered trading tips and build investor confidence. Victims were directed to fund accounts on platforms including Morocoin Tech Corp., Berge Blockchain Technology Co. Ltd. and Cirkor Inc., where withdrawals were blocked and additional advance fees were requested.

🎄 This Christmas episode of The AI Fix examines whether chatbots agree that Santa Claus exists, testing responses from popular conversational AIs and Google's seasonal features. The hosts discuss a string of Waymo robotaxi incidents that sparked PR headaches, Microsoft's reduced ambitions for Copilot amid low usage, and research suggesting future programmers may rely more on psychological prompt design than traditional coding. Hosts: Graham Cluley and Mark Stockley.