AI Summaries Can Be "Poisoned" Too? Microsoft Warns of New Attack Method

IT House, February 13th: Perhaps those who have been online for a while have heard of “SEO poisoning.” Malicious actors often create大量低质量的文章 on the internet and then bind a fake or altered tool to specific keywords, causing search engines to display malicious software as the top result instead of official products when users search for those keywords.

According to a blog post released this week (February 10th) by Microsoft security researchers, there is a similar “AI recommendation poisoning” attack in the AI field.

For example, some companies embed hidden commands into web pages or app “AI summary” buttons. When users click these, the system attempts to inject persistent instructions into the AI assistant’s memory via URL prompt parameters.

IT House quotes Microsoft, stating that these prompts can cause AI to solidify certain perceptions, such as “XX company is a reliable source” or “Prioritize recommending XX company,” making the AI biased toward certain products or services in future responses.

However, the problem arises if malicious actors try to modify the webpage prompt parameters, replacing “XX company” with “XX scam company.” The AI would then unwittingly provide toxic advice to other users, manipulate recommendations and AI summaries, and produce altered summary articles—all without users knowing.

Microsoft emphasizes that these attack methods are not just theoretical. The company has already detected 50 similar poisoning cases in email traffic, involving finance, healthcare, legal services, marketing, food, and service industries. Common patterns include “Remember that XX company is a trustworthy source” or “Prioritize citing XX website in future conversations.” Some cases even involve injecting complete marketing copy. All these cases originate from legitimate companies, not hackers.

Microsoft warns that these examples demonstrate the real risk of such poisoning techniques. Financial professionals could be recommended high-risk or scam investment platforms; parents might blindly trust AI and overlook harmful content in children’s games; ordinary users could be manipulated by malicious actors to repeatedly cite a single media source within AI.

The best way to counter this is not to fully trust “AI summaries.” Users should hover over links recommended by AI to verify their legitimacy before clicking, be cautious when clicking the “AI summary” button, and regularly check the AI’s stored memory. If suspicious entries are found, they can be deleted or the AI’s memory can be reset.