Bigger Prompt Injection Datasets Can Weaken AI Agents
🛡️ Researchers from the University of Waterloo, Zhejiang University and KTH Royal Institute of Technology found that AI agents retrained on larger piles of prompt injection attacks were easier to hijack in three of four tests. The likely culprit is that automated red-team tools keep rewording the same few tricks. Their PIPETTE system rewards technique diversity and cut off-the-shelf attack success to under half a percent on AgentDojo.
