Microsoft Research Detects Backdoored Open Models

Introduction: Why this matters

Open-weight language models are increasingly adopted across enterprises for copilots, automation, and developer productivity. That adoption expands the software supply chain to include model weights and training pipelines—creating new opportunities for tampering that may not be caught by traditional testing. Microsoft’s new research targets model poisoning backdoors (also called “sleeper agents”), where a model behaves normally in most cases but reliably switches to attacker-chosen behavior when a trigger appears.

What’s new: Three observable signatures of backdoored LLMs

Microsoft’s research breaks the detection problem into two practical questions: (1) do poisoned models systematically differ from clean models, and (2) can we extract triggers with low false positives without assuming we know the trigger or payload?

1) Attention hijacking (“double triangle”) + entropy collapse

When a trigger token appears, backdoored models can show a distinctive attention pattern where the model disproportionately focuses on trigger tokens, largely independent of the rest of the prompt. This appears as a “double triangle” attention structure.

In addition, triggers often cause output entropy to collapse: instead of many plausible continuations (high entropy), the model becomes unusually deterministic toward the attacker’s target behavior.

2) Backdoored models may leak their poisoning data

The research identifies a connection between poisoning and memorization: by prompting with particular chat-template/special tokens, a backdoored model may regurgitate fragments of the poisoning examples, including the trigger itself. This leakage can reduce the search space for trigger discovery and accelerate scanning.

3) Backdoors are “fuzzy” (trigger variations can work)

Unlike traditional software backdoors that often rely on exact conditions, LLM backdoors can be activated by multiple variations of a trigger. That fuzziness matters operationally: detection approaches must consider families of triggers rather than a single exact string.

Impact for IT administrators and security teams

• Model supply chain risk increases when importing open-weight models into internal environments (hosting, fine-tuning, RAG augmentation, or packaging into apps).
• Standard evals may miss sleeper behaviors because poisoned models look benign until the right trigger appears.
• This research supports building repeatable, auditable scanning methods—complementing broader “defense in depth” (secure build/deploy pipelines, red-teaming, and runtime monitoring).
• Don’t overlook classic threats: model artifacts can also be vehicles for malware-like tampering (e.g., malicious code executed on load). Traditional malware scanning remains a first line of defense; Microsoft notes malware scanning for high-visibility models in Microsoft Foundry.

Recommended next steps

1. Treat models as supply chain artifacts: track provenance, versions, hashes, and approval gates for model weights and templates.
2. Add pre-deployment scanning for poisoning indicators (behavioral signatures, entropy anomalies, trigger-search workflows) alongside dependency and malware scanning.
3. Perform targeted red-teaming focused on hidden triggers, prompt/template edge cases, and deterministic output shifts.
4. Monitor in production for unexpected deterministic responses, prompt-pattern correlations, and policy-violating “mode switches.”

Microsoft’s findings lay groundwork for scalable detection of poisoned LLMs—an important step toward safer enterprise adoption of open-weight models.