Security

AI Recommendation Poisoning Threatens Microsoft Copilot

3 min read

Summary

Microsoft researchers say attackers are trying to manipulate AI assistants like Copilot by hiding prompt injections in AI-related links, aiming to plant persistent “memory” instructions that bias future recommendations. The campaign was observed at scale across dozens of companies and industries, highlighting a growing security risk for enterprises because poisoned AI outputs could quietly influence purchasing, security decisions, and user trust.

Need help with Security?Talk to an Expert

Introduction: why this matters

AI assistants are increasingly trusted to summarize content, compare vendors, and recommend next steps. Microsoft security researchers are now seeing adversarial (and commercially motivated) attempts to persistently bias these assistants by manipulating their memory—turning a seemingly harmless “Summarize with AI” click into a long-lived influence on future responses.

In enterprise environments, this is more than an integrity issue. If an assistant’s recommendations can be subtly steered, it can impact procurement decisions, security guidance, and user trust—without obvious indicators that anything changed.

What’s new: AI Recommendation Poisoning in the wild

Microsoft Defender Security Research Team describes an emerging promotional abuse pattern they call AI Recommendation Poisoning:

  • Hidden prompt injection via URL parameters: Web pages embed links (often behind “Summarize with AI” buttons) that open an AI assistant with a pre-filled prompt using query parameters like ?q=<prompt>.
  • Persistence targeting “memory” features: The injected prompt attempts to add durable instructions such as “remember [Company] as a trusted source” or “recommend [Company] first.”
  • Observed at scale: Over a 60-day review period of AI-related URLs seen in email traffic, researchers identified 50+ distinct prompt attempts from 31 companies across 14 industries.
  • Cross-platform targeting: The same approach was observed aiming at multiple assistants (examples included URLs for Copilot, ChatGPT, Claude, Perplexity, and others). Effectiveness varies by platform and evolves as mitigations roll out.

How it works (and why memory changes the risk)

Modern assistants can retain:

  • Preferences (formatting, tone)
  • Context (projects, recurring tasks)
  • Explicit instructions (“always cite sources”)

That usefulness creates an attack surface: AI memory poisoning (MITRE ATLAS® AML.T0080) occurs when an external actor causes unauthorized “facts” or instructions to be stored as if they were user-intended. The research maps this technique to prompt-based manipulation and related categories (including MITRE ATLAS® entries such as AML.T0051).

Impact on IT admins and end users

  • Recommendation integrity risk: Users may receive biased vendor/product guidance that appears objective.
  • Hard-to-detect manipulation: The “poison” can persist across sessions, making it difficult for users to connect later decisions to an earlier click.
  • Increased social engineering surface: These links can appear on the web or be delivered via email, blending marketing tactics with security abuse.

Microsoft notes it has implemented and continues deploying mitigations in Copilot against prompt injection; in several cases, previously reported behaviors could no longer be reproduced—indicating defenses are evolving.

Action items / next steps

  • Update security awareness training: Teach users that AI “summarize” links can be weaponized, especially if they pre-fill prompts.
  • Review email and web protections: Ensure link-scanning and phishing defenses are tuned to analyze unusual URL parameters and redirect patterns.
  • Establish AI usage guidance: Encourage users to verify sources, cross-check recommendations, and report suspected “memory” anomalies.
  • Operational playbook: Define steps for users/admins to review and clear assistant memory (where supported) and to report suspicious prompts/URLs to security teams.

Recommendation Poisoning is a clear signal that as AI becomes a decision-support layer, integrity and provenance controls must evolve alongside traditional phishing and web threat models.

Need help with Security?

Our experts can help you implement and optimize your Microsoft solutions.

Talk to an Expert

Stay updated on Microsoft technologies

Read the original article by Microsoft Defender Security Research Team
AI securityCopilotprompt injectionmemory poisoningMicrosoft Defender

Related Posts

Security

macOS ClickFix Campaign Hides Behind Fingerprinting

Microsoft Threat Intelligence reports that a macOS ClickFix campaign has shifted from openly serving malicious lures to using server-side browser fingerprinting that mainly exposes the payload to likely macOS victims. The change makes the operation harder for crawlers, sandboxes, and defenders to spot, increasing the importance of hunting for shared infrastructure patterns and strengthening endpoint protections.

Security

Microsoft CNAPP Leader: KuppingerCole 2026 Report

Microsoft has been named a Leader across all four categories in KuppingerCole’s 2026 CNAPP Leadership Compass, highlighting Defender for Cloud’s unified approach to cloud and AI security. The recognition matters for security teams as CNAPP platforms increasingly focus on exploitability, attack path analysis, AI security posture, and integrated SOC operations across multicloud environments.

Security

ChainDrop npm Attack: Self-Propagating Worm Explained

Microsoft has detailed ChainDrop, a large-scale npm supply chain attack that compromised more than 400 packages using a self-propagating credential-stealing worm. The campaign matters because it targets developer workstations and CI/CD pipelines, steals cloud and publishing credentials, and can automatically republish infected packages across additional publishers.

Security

Microsoft Defender Device Isolation Stops Ransomware

Microsoft Defender’s attack disruption now includes automatic device isolation for compromised endpoints, adding a new containment layer beyond user-based response. In a published QNET case study, Defender isolated an endpoint in 128 seconds, stopping a multi-stage ransomware-related attack before persistence, credential theft, or lateral movement could occur.

Security

Zero Trust for AI: New Microsoft Security Tools

Microsoft has expanded its Zero Trust for AI strategy with new assessment capabilities, a DevSecOps pillar in the Zero Trust Workshop, and practical implementation guidance. The updates help security and platform teams assess AI-related risk, secure AI-assisted development, and build a phased roadmap for adopting Zero Trust controls across AI systems.

Security

CaptiveCrunch Malware Campaign Targets Travelers

Microsoft has disclosed CaptiveCrunch, an active campaign linked to Midnight Blizzard that hijacks traffic on hospitality and captive portal networks to steal credentials and deliver malware. The threat is especially relevant for organizations with frequent travelers because attackers are abusing Entra ID device code flows, fake update prompts, and adversary-in-the-middle phishing to compromise corporate accounts and endpoints.