Skip to content
All insights
5 min read

When your AI assistant becomes the attack vector

By Antonio Spera

It is a Tuesday morning. A developer opens the AI assistant they use all day, every day, and asks it to help with a task. It answers. The tone is slightly off, maybe a suggestion they would not expect. They shrug it off — AI can be quirky. But the voice answering is not their assistant. It is an attacker.

No phishing email was sent. No malware was downloaded. The tool the user already trusted became the attack.

That is not a thought experiment. It is the shape of a red-team exercise published by Pentera Labs in July 2026, in which researchers Dvir Avraham and Reef Spektor went from access to a compromised email platform to remote code execution on a victim's workstation — using the victim's own Claude assistant as the weapon.

From a mailbox to a shell

The chain is worth understanding because every link in it is ordinary.

The researchers had access to a platform that aggregates customer inboxes behind a single interface, and through it, to thousands of live mailboxes. From there:

  1. Take the AI account. Inbox access yields the Claude account — a magic link, a password reset, whatever path the platform allows.
  2. Poison a synced preference. Claude Desktop has a "Personal Preferences" field: a user-editable prompt that shapes how the assistant behaves, and that syncs across every session and device on the account. The researchers dropped an encoded payload into it through the compromised account.
  3. Let sync do the work. The next time the victim opens Claude Desktop, the poisoned instructions are already loaded. The victim did nothing; the setting simply propagated.
  4. Enumerate, then execute. The injected prompt tells the assistant to quietly check which tools are installed. If a command-capable extension such as Desktop Commander is present, the assistant uses it to run attacker-chosen commands — full remote code execution, with no further interaction from the user.
  5. If no tool exists, social-engineer one. When no command-capable extension is installed, the assistant presents a convincing fake error, complete with an error code and step-by-step instructions, coaching the user to install one. The assistant itself becomes the phishing page.
  6. Persist. From code execution, the researchers had the assistant fetch and run attacker-served bash on every interaction — a quiet command-and-control channel the victim kept feeding simply by using the app.

The outcome is the contents of a developer's machine: SSH keys, cloud credentials, kubeconfigs, source code, terminal history. The foothold that turns one workstation into the organisation's cloud.

Not a bug, a design surface

When the researchers disclosed this to Anthropic, the response was telling. Anthropic reportedly did not treat it as a vulnerability in scope: personal preferences, skills, and MCP connectors are features that can execute code through the desktop client by design. Related enhancements were said to be on the roadmap.

They have a point, and that is exactly the problem. The model did nothing wrong. It did what it was told; the trouble is who was telling it. This is not a flaw in Claude specifically — it is the convergence of four things, each unremarkable on its own:

  • Account compromise, the oldest problem in security.
  • A synced setting that changes model behaviour, without re-authentication.
  • Local tool access — extensions, MCP servers, terminal integrations — that can run code.
  • User trust in an interface that still looks and feels like a chat window.

The gap between what an assistant can do (touch files, run commands, call tools) and what users perceive (a search bar) is precisely where this attack lives.

It maps onto work we already have

This is not new ground for the literature; it is new ground for most security teams.

Indirect prompt injection — malicious instructions arriving through data the agent reads rather than the user's prompt — has been documented in the research literature since 2023 (Greshake et al.). The synced-preference trick is a close cousin: the untrusted instruction does not even need to arrive inside a document; it lives in the assistant's own configuration.

OWASP's Top 10 for LLM Applications names the two halves of this chain: Prompt Injection, and Excessive Agency. The Pentera exercise is both at once — an injected instruction driving an agent that has been granted the authority to act on it.

MITRE ATLAS catalogues these as adversarial tactics against AI systems, and the governance scaffolding to decide whether an agent should hold that authority at all lives in ISO/IEC 42001 and the NIST AI Risk Management Framework.

The frameworks are ready. The gap is operational.

What changes in your controls

Most of this is familiar security hygiene, applied to a surface most teams have not yet catalogued.

  • Treat AI desktop apps as privileged software, not chat windows. They can execute code and read files. Scope, harden, and monitor them the way you would any endpoint agent.
  • Watch the synced surface. Personal preferences, skills, and MCP connector configurations are now part of your identity-and-access attack surface. Alert on changes to them, especially changes that propagate across devices.
  • Restrict command-capable tools. If an assistant can install or invoke something that runs shell commands, assume that path will be used against you. Allow-list tools; do not let the model authorise its own privileges.
  • Monitor child processes. A chat application spawning shells, curl, or package managers is the signature of this chain. It should be an alert, not a log line.
  • Re-assess account-compromise blast radius. Taking an AI account is no longer only a data-read problem; with local tooling it is a workstation-compromise problem. Apply strong authentication, short sessions, and least privilege to AI accounts too.
  • Stop trusting assistant-initiated prompts. Train people that an unexpected error or install nudge from the assistant must be verified out-of-band — exactly as they would an unexpected email. The assistant is now a social-engineering channel.

The principle, unchanged

The old rule still holds: never let untrusted input authorise a privileged action, and never let the model's output alone be the thing that grants authority. What has changed is the surface. The untrusted input now flows through a synced preference the user never sees change, and the privileged action runs on the user's own machine, inside a window they trust.

The attacker no longer needs to send you an email. They just need to sound like your assistant.

If you are deploying agents or AI desktop tooling and want the threat modelled rather than improvised, that is the kind of secure-AI review I do.