Detection Engineering in the Era of Semantic Malware

In March 2026, Origin published research on Brainworm — a proof-of-concept that hijacks AI coding assistants by poisoning their memory configuration files with natural language instructions. No binary, shellcode or exploit. Just a CLAUDE.md file containing English sentences that instruct the agent to register with a command-and-control server, establish a heartbeat, and execute whatever tasking comes back over RabbitMQ. The agent’s subsequent tool calls — reading files, running commands, making network requests — are the same tool calls it makes legitimately. The only difference is who’s directing them. This is not a memory-corruption exploit or a software bug in the conventional sense. It is a trust-boundary failure: the system ingests untrusted input into a context where all input is treated as trusted instruction, and no provenance check distinguishes the developer’s intent from the attacker’s.

Brainworm is a single specimen of what researchers from Ben-Gurion University, Tel Aviv University, Harvard Kennedy School, and the University of Toronto have formalized as “promptware” — malware delivered through prompt injection that follows a kill chain parallel to, but fundamentally different from, conventional malware. Analyzing 36 documented incidents between 2023 and early 2026 — a mix of proof-of-concept demonstrations, vendor disclosures, and researcher reproductions — they tracked promptware’s evolution from two-stage chatbot tricks to routinely four- and five-stage attack chains achieving persistence, C2, lateral movement, and actions on objective including data exfiltration, RCE, financial theft, and physical IoT manipulation (Cohen, Bitton, & Nassi, “A Jailbroken GenAI Model Can Cause Substantial Harm,” 2024; Brodt, Feldman, Schneier, & Nassi, “The Promptware Kill Chain,” 2026). The progression is not subtle: zero lateral movement incidents in 2023, three in 2024, eight in 2025–2026. And the targets shifted from conversational chatbots to the tools we use to build software — GitHub Copilot, Cursor, Windsurf, Devin — and the tools our organizations use to operate: Slack AI, Microsoft 365 Copilot, Google Assistant. The attack surface is no longer a web form. It is every document, email, calendar invite, and repository that an AI agent is permitted to read.

For detection engineers, this creates a problem that is genuinely new. Our discipline was built on a foundational assumption: that malicious activity produces observable, distinguishable artifacts — a process tree anomaly, a registry modification, a network connection to a known-bad indicator, a command line matching a signature. Promptware degrades every one of these signals. It has no binary to signature-match. Its behavior is non-deterministic — the same prompt produces different tool calls across executions, rendering static payload signatures useless at scale. Its operations are performed by a trusted process (the agent runtime) using legitimate credentials, making it significantly harder to distinguish from legitimate use — though not impossible, as we’ll see. As Matt Hand of Origin put it: “Prompt filtering, safety classifiers, and activity monitoring are just AMSI for the LLM age.” And we know how AMSI went. The question is not whether our existing detection frameworks still apply — parts of them do — but which parts survive contact with an attacker who writes English instead of assembly, and what we need to build for the parts that don’t.

Scope: Which Agents, Which Threats

Not all AI agents present the same attack surface. A conversational chatbot that can only generate text is a different animal from a coding agent with shell access, filesystem writes, network egress, and persistent memory. This post focuses on the latter category — tool-using agents with system-level capabilities: Claude Code, Codex CLI, Cursor, Windsurf, Devin, Gemini, and their enterprise equivalents. These are the agents where promptware achieves its full kill chain, because they have the permissions to act on malicious instructions in ways that matter. Conversational chatbots, enterprise copilots with limited tool access, and retrieval-only assistants each have their own risk profiles, but the detection challenges described here apply most directly to agents that can read files, execute commands, and make network requests.

It is also worth noting that the evidentiary base for promptware remains heavily weighted toward proof-of-concept demonstrations and controlled research rather than confirmed in-the-wild campaigns. Brainworm is a PoC. Most of the 36 incidents in the promptware kill chain corpus are researcher reproductions or vendor disclosures, not incident response reports. This does not diminish the threat — PoCs have a way of becoming operational tooling — but it means we are building detection strategies for a threat that is maturing in parallel with our defenses. The window to get ahead of it is now.

What Breaks

Start with signatures. The unit of detection in traditional security is the artifact — a hash, a string, a command-line pattern, a registry path, a network indicator. Promptware produces none of these in a stable, repeatable form. Ask an LLM agent to “analyze the financial data” five times, and you get five different execution plans — one uses pandas, another exports to CSV, a third writes custom parsing code. Each generates a completely different set of process creations, file writes, and network connections. Now replace “analyze the financial data” with “exfiltrate the financial data.” The same non-determinism applies. You cannot write a Sigma rule for a payload execution sequence that reinvents itself on every run. You can write a thousand rules for tool call patterns and still achieve minimal coverage against the payload itself, because the attacker’s next execution will route through the one sequence you didn’t anticipate. (Rules against surrounding observables — memory file writes, first-seen destinations, permission grants — remain viable, as we’ll discuss. But the payload-level signature is dead.)

Process tree analysis fares no better. In conventional detection, parent-child relationships are signal-rich: winword.exe spawning cmd.exe is suspicious; svchost.exe spawning powershell.exe is actionable. But an AI agent runtime is designed to spawn arbitrary processes. Claude Code calling curl, git, python, node, grep, and bash in rapid succession is not an anomaly — it is Tuesday. The agent’s process tree is a superset of almost every LOLBin chain an attacker would want. When the legitimate behavior of a process includes “execute any command the user or the context window tells it to,” there is no parent-child relationship left that constitutes an anomaly by itself.

Network-based detection faces the same collapse. Brainworm’s C2 runs over AMQP (RabbitMQ), which is a legitimate message queuing protocol used in production systems worldwide. But even that is optional — an attacker could just as easily route C2 through GitHub Issues, Google Docs, Slack messages, or any other service the agent already has credentials to access. Origin’s David Kaplan demonstrated a related vector: agents with browsing tools can be tricked into treating descriptive HTTP status codes embedded in page content as actual protocol instructions, navigating to attacker-controlled destinations without any exploit — a class of failure he calls “semantic protocol confusion” (Kaplan, 2026). The promptware kill chain research documented exactly this pattern: ChatGPT ZombAI used GitHub Issues as a C2 channel, storing updated instructions in issue comments that the compromised agent polled on a loop. When your C2 infrastructure is a SaaS product your organization already uses and trusts, domain reputation lists and TLS inspection are insufficient on their own. The traffic is going to a known-good destination over an encrypted channel, authenticated with the user’s own tokens. These controls still have value in layered defense, but they cannot be the primary detection surface for SaaS-native C2.

What remains? If signatures, process trees, and network indicators all fail at the point of execution, we need to shift our detection surfaces backward and outward — to the preconditions that make promptware possible, and to the macro-behavioral patterns that distinguish directed malicious intent from legitimate agent automation.

What Survives

Not everything is lost. Some detection engineering principles translate directly, even if the specific artifacts change. And it is worth stating plainly: for many of these threats, prevention is cheaper than detection. Least-privilege agent permissions, mandatory approval gates for sensitive tool calls, network egress restrictions, ephemeral (non-persistent) memory, and signed memory files can eliminate entire attack chain stages before detection even enters the picture. Detection engineers should advocate for these controls, not just write rules for their absence. That said, prevention is never complete, and the controls below are what remain when prevention fails.

File integrity monitoring remains a durable control — particularly for coding agents. Brainworm’s attack chain begins with a write to a memory configuration file — CLAUDE.md, AGENTS.md, GEMINI.md, cursorrules. For coding agents, these are the primary persistence mechanism. They are finite, enumerable, and their modification is an observable event. A detection engineer who monitors these files the way we monitor HKLM\Software\Microsoft\Windows\CurrentVersion\Run is applying an old principle to a new surface: watch the places where persistence lives. The file paths change; the logic does not. This is the registry key of the LLM age, and we should treat it with the same paranoia — alerting on any modification, tracking provenance (who wrote it, when, from what process), and diffing content against a known-good baseline. A concrete detection: alert when any file matching CLAUDE.md, .cursorrules, AGENTS.md, or GEMINI.md is modified by a process other than a known IDE or git operation, or when the diff contains network URLs, shell commands, or base64-encoded content not present in the last reviewed commit. This is not the only persistence vector — retrieved documents, email content, and SaaS-hosted data can also carry malicious instructions for non-coding agents — but for the agent class we’re focused on, memory files are the attack surface to monitor first.

Privilege auditing translates almost verbatim. Agents request permissions. Users grant them. This is the overpermissioning problem that Origin identified: “The personal assistant agent that we allow to read our emails to coordinate meetings can become a powerful corporate espionage tool.” Detection engineers have been writing rules for excessive privilege grants since Event ID 4672 existed. The specific telemetry changes — instead of Windows Security logs, we’re looking at agent permission grants, OAuth scopes, MCP server connections, tool authorization events — but the detection pattern is identical: alert when an identity acquires capabilities disproportionate to its role, and alert again when those capabilities are exercised against sensitive resources for the first time.

Behavioral baselining — the thing we’ve always said we should do more of and mostly haven’t — becomes non-optional. Ryan Cox’s detection engineering process starts with “count by” queries: what’s normal volume, what’s normal frequency, what are the common values for each field. This approach is the only one that survives non-deterministic behavior. You cannot signature-match a prompt-ware agent’s individual tool calls, but you can baseline what a healthy agent session looks like in aggregate: typical number of tool calls per session, typical ratio of read operations to write operations, typical network destinations, typical working directories, typical session duration. Deviation from that baseline is the signal. A concrete example: alert when an agent session’s write-to-read ratio exceeds two standard deviations from the 30-day baseline for that project, AND the session includes a first-seen external network destination, AND the session duration is in the bottom quartile (fast, automated-feeling sessions). No single field is suspicious; the combination is. This is the same logic we use for insider threat detection — and that parallel is not a coincidence. A compromised agent operates like an insider threat: legitimate credentials, legitimate access, a legitimate reason to touch sensitive resources. The behavioral baseline is how we surface the difference between normal automation and directed abuse.

The Palantir ADS framework’s “Blind Spots and Assumptions” section becomes the most important part of any detection strategy document. Every ADS written for agent-related detection should start by listing what it cannot see. Can you observe the content of the agent’s context window? Can you see what instructions were loaded from memory files? Can you distinguish a user-initiated tool call from an autonomously-initiated one? If the answer to any of these is no, say so explicitly. The detection engineering discipline already has the right instinct here — Palantir insisted on documenting blind spots in 2017 — but the semantic malware era makes this section load-bearing in a way it never was before. A detection rule for agent abuse without a documented blind spot analysis is operating on assumptions you haven’t tested.

New Detection Surfaces

The surfaces that matter for promptware are not the ones we’re used to instrumenting.

Context window integrity

The fundamental attack vector is poisoning what the agent reads before it acts. If we could inspect the agent’s context window at inference time — every system prompt, every memory file loaded, every retrieved document — and compare it against a policy, we would catch Brainworm at the moment of injection, before any malicious tool call occurs. This is the equivalent of inspecting a process’s loaded DLLs at startup. Today, to the best of my knowledge, no major agent framework exposes this telemetry in a standardized, consumable format. This is the single highest-value instrumentation gap in the entire space. The agent providers who solve it first — who emit structured logs of “here is what was in the context window when this tool call was authorized” — will make detection engineering for semantic threats possible. Until then, we’re working with indirect signals. (A necessary caveat: full context window logging raises obvious privacy, confidentiality and storage concerns. Any implementation will need to balance detection fidelity with data minimization — hashing, redaction, or policy-match-only logging rather than full content capture.)

Agent session telemetry

Every tool call an agent makes should be logged with: the tool name, the arguments, the result, the timestamp, and — critically — whether the call was explicitly requested by the user or autonomously decided by the agent. The distinction between “user said: read file X” and “agent decided to read file X based on context” is the difference between legitimate use and potential compromise. In practice, this attribution is often blurry — user intent and model-selected steps blend together — but even probabilistic attribution is valuable for detection scoring. This is the equivalent of the audit trail we demand from privileged access management systems. Some agent frameworks are beginning to emit this (Claude Code’s ~/.claude/projects/ session logs, for instance), but it is not yet standardized, not yet consumed by SIEMs, and not yet the subject of detection rules.

Memory file provenance

Who wrote the CLAUDE.md file, and when? Was it committed by a known developer, or did it appear in a pull request from an external contributor? Was it modified after the repository was cloned? Git history, file system timestamps, and code review metadata become detection-relevant. A CLAUDE.md file that was not present in the repository’s last reviewed commit but appears in the working directory is suspicious — the same way a new DLL in System32 that doesn’t match the last golden image is suspicious. Supply chain security for agent memory files needs the same rigor we’ve (slowly, painfully) built for dependency management.

Cross-session behavioral correlation

A single compromised agent session might look normal. A pattern of compromised sessions — same unusual network destination across multiple users’ agents, same file read across agents that have no project in common, same escalation sequence appearing in agents started from different repositories — is detectable through correlation. This is the SIEM’s core competency, applied to a new data source. The challenge is getting the telemetry into the SIEM in the first place.

The AMSI Lesson

The comparison between current agent defenses and AMSI is instructive, if imperfect. AMSI was the industry’s response to a previous category of “trusted process doing bad things” — PowerShell executing malicious scripts through a legitimate interpreter. The solution: hook into the interpreter and inspect content before execution. The result: it worked against naive attacks, the offensive community developed dozens of bypasses, and AMSI became one layer in a defense-in-depth stack rather than a standalone solution.

Current agent defenses — prompt injection classifiers, safety guardrails, system prompt hardening — are content-inspection mechanisms bolted onto an interpreter. The evasion trajectory is already visible: from simple instruction-overriding (“ignore previous instructions”) in 2023 to ASCII smuggling, color-matched text, multi-turn strategies, and universal adversarial suffixes by 2025 (Brodt et al., 2026). Whether the timeline compresses from AMSI’s decade to months is an open question, but the direction is clear: content-inspection defenses will face sustained evasion pressure.

The practical takeaway for detection engineers: design detection strategies assuming these guardrails have been partially bypassed, the same way we write post-exploitation detections assuming endpoint protection was evaded. The guardrails buy time. The detection rules are the safety net.

What We Need to Build

The gap between where we are and where we need to be is specific and addressable.

First: agent telemetry standards

There is no equivalent of Sysmon for agent activity. No standardized schema for “agent made a tool call.” No common log format across Claude Code, Codex, Gemini CLI, Cursor, or any other agent framework. Each vendor logs differently, at different verbosity levels, in different locations, with different retention. We need an agent telemetry specification — call it ATF, Agent Telemetry Format — the way we needed ECS and OCSF for cloud and endpoint logs. Without this, detection engineers are writing bespoke parsers for each agent platform, which does not scale.

Second: detection content for agent abuse

The Sigma rules we’ve written for decades don’t cover agent-specific attack patterns. We need rules for: memory file modification, anomalous agent tool call sequences, agent-initiated network connections to novel destinations, agent permission escalation events, and cross-agent lateral movement indicators. These rules won’t look like traditional process creation or registry event rules — they’ll look more like the behavioral analytics we write for insider threat programs, with baselines, thresholds, and deviation scoring rather than static pattern matching.

Third: agent-aware detection engineering processes

When Ryan Cox describes his seven-phase detection process — research, brainstorming, log diving, organization, creation, validation, iteration — every phase applies, but the inputs change. “Research” means understanding what the agent can do with its current permissions, not what a binary can do with its current privileges. “Brainstorming” means thinking about what a hostile instruction in a context window would direct, not what a command-line argument would trigger. “Diving into logs” means finding where agent telemetry lives and whether it’s sufficient, not querying Sysmon. The process is sound. The subject matter is new.

Fourth: red teaming for agent abuse

Validation in the ADS framework requires generating a representative true positive — the equivalent of an Atomic Red Team test. For agent-based detections, this means building a Brainworm-like payload, deploying it in a test environment, and confirming your detection fires. If you cannot test your detection against a semantic attack, you do not have a detection. You have a hypothesis. The agent security community needs its own Atomic Red Team — a library of promptware test cases that detection engineers can execute to validate their rules.

The Road Ahead

Semantic malware does not obsolete detection engineering. It is a platform shift, not an extinction event. The principles that have always mattered — understand your data sources, know your blind spots, baseline normal behavior, document your assumptions, validate your detections, iterate continuously — matter more now than they did when the hardest problem was writing a Splunk query for encoded PowerShell.

What changes is the object of detection. For years, we detected malicious software by its artifacts. Now we need to detect malicious behavioral patterns — context-dependent, aggregate, probabilistic — in a setting where the executing process is trusted, the credentials are legitimate, and the individual actions look like normal operations. We cannot observe intent directly; we can observe the behavioral footprint that distinguishes directed abuse from legitimate automation. This is harder. It is also, in a specific way, more interesting — because it forces us to do the thing we’ve always claimed to value: understand behavior deeply enough to spot the subtle wrongness that no signature will ever catch.

The attacker writes English now. The detection engineer’s job hasn’t changed. The craft just got more demanding.

If you enjoyed the article, feel free to connect with me! https://www.linkedin.com/in/daniel-koifman-61072218b/ https://x.com/KoifSec https://koifsec.me https://bsky.app/profile/koifsec.bsky.social

References

1. Turner, M. (2026, March 4). “Brainworm: Hiding in Your Context Window.” Origin. https://www.originhq.com/blog/brainworm

2. Hand, M. (2025, November 26). “The Era of Semantic Security: Computer Use Agents and the End of Signatures.” Origin. https://www.originhq.com/blog/era-of-semantic-security

3. Brodt, O., Feldman, E., Schneier, B., & Nassi, B. (2026, January 17). “The Promptware Kill Chain: How Prompt Injections Gradually Evolved Into a Multistep Malware Delivery Mechanism.” arXiv:2601.09625v2. Ben-Gurion University, Tel Aviv University, Harvard Kennedy School, University of Toronto. https://arxiv.org/abs/2601.09625v2

4. Cohen, S., Bitton, R., & Nassi, B. (2024, August 9). “A Jailbroken GenAI Model Can Cause Substantial Harm: GenAI-powered Applications are Vulnerable to PromptWares.” arXiv:2408.05061. https://arxiv.org/abs/2408.05061

5. Kaplan, D. (2026, January 25). “Semantic Protocol Confusion: When My LLM Thinks It’s a Web Browser.” Origin. https://www.originhq.com/blog/semantic-protocol-confusion

6. Palantir. (2017, April 28). “Alerting and Detection Strategy Framework.” https://blog.palantir.com/alerting-and-detection-strategy-framework-52dc33722df2

7. Cox, R. G. (2025, April 1). “My SIEM-Agnostic Creative Process to Detection Engineering.” The Cybersec Cafe, Issue #61. https://www.cyberseccafe.com/p/my-siem-agnostic-creative-process