Anti-Debugging Techniques

Anti-debugging is a series of anti-reversing techniques used by malware (and even some legitimate programs) to hamper or prevent debugging.

For example, the malware might try to interfere with the debugger process if it detects that it’s attached to a debugger, or it might try to prevent debugging altogether by using so-called anti-attach mechanisms or crashing the debugger program.

In this article, we’ll explore some of these techniques in detail.

Using Windows API Functions to Access the PEB:

First of all, The PEB is a Process Environment Block (PEB) is a structure that contains pointers to information in memory about the currently running process and the PEB includes several pointers that are relevant to anti debugging, as listed here:

0x002 BeingDebugged:

Indicates whether the program is currently being debugged.

0x018 ProcessHeap:

Contains pointers to the heap’s Flags and ForceFlags members.

0x068 NtGlobalFlag:

Contains information related to the creation of memory heaps.

Before we go deeper, we should understand that Windows provides a set of functions that programs can call to see if a debugger is attached to them and Malware can exploit these functions as well to shut itself down or modify its behavior if a debugger is detected, and they can be as followed:

1. IsDebuggerPresent and CheckRemoteDebuggerPresent:

One of the best-known and simplest Windows functions for detecting debuggers is IsDebuggerPresent.

This function returns a nonzero value if the current process is being debugged; otherwise, it returns 0 and Malware can simply call it like this:

if (IsDebuggerPresent()) {
    ExitProcess(0); // Stop execution
}

The CheckRemoteDebuggerPresent function returns the same information, but with a value of True or False:

BOOL isDebugger = FALSE;
CheckRemoteDebuggerPresent(GetCurrentProcess(), &isDebugger);
if (isDebugger) {
    ExitProcess(0);
}

The following example shows how malware might use it and how can you detect it:

--snip--
push [ebp+hProcess]
push [ebp+DebuggerPresent]
call CheckRemoteDebuggerPresent
--snip--

As you can see two parameters are being pushed to the stack here: a handle to the target process (in this case, the malware’s own process,hProcess) followed by a pointer to a variable that will receive the information returned (DebuggerPresent).

Once the malware calls CheckRemoteDebuggerPresent, the return value (True or False) is stored in the DebuggerPresent variable; As you’ll see later in the chapter, these are some of the easiest debugger detection techniques to circumvent.

This means that in IsDebuggerPresent, IsDebuggerPresent They can be reverse engineered and defused by changing the value in the register to be 0.

2. NtQueryInformationProcess:

The NtQueryInformationProcess function is an NT API function that provides internal information about any running Process and so it can be used to detect if a program is running under a Debugger environment such as x64dbg, OllyDbg, IDA Pro, and others, but how?

NtQueryInformationProcess takes several parameters, including:

ProcessHandle → Handle of the target process, usually passed to GetCurrentProcess() to retrieve the handle of the current process.

ProcessInformationClass → Specifies the type of information requested about the process, here we use the following values ​​to detect the debugger:

ProcessDebugPort (7): If it returns a non-zero value, it means that the program is being debugged.

ProcessDebugFlags (0x1F): If it returns 0, it means that a Debugger is connected to the program.

ProcessDebugObjectHandle (0x1E): If it returns a valid handle, it means that a Debugger is connected.

ProcessInformation → Variable to store the result.

ProcessInformationLength → Size of the target variable (sizeof(DWORD))

ReturnLength → Optional variable to store the number of bytes returned (can be passed as NULL).

#include <windows.h>
#include <winternl.h>

typedef NTSTATUS(NTAPI* pNtQueryInformationProcess)(
    HANDLE, UINT, PVOID, ULONG, PULONG);

bool isDebuggerAttached() {
    HMODULE hNtDll = GetModuleHandle("ntdll.dll");
    pNtQueryInformationProcess NtQueryInformationProcess =
        (pNtQueryInformationProcess)GetProcAddress(hNtDll, "NtQueryInformationProcess");

    DWORD debugPort = 0;
    NTSTATUS status = NtQueryInformationProcess(GetCurrentProcess(), 7, &debugPort, sizeof(DWORD), NULL);
    return (debugPort != 0);
}

If the return value from the call is non-zero, it means that a Debugger is connected so this also can be defused by making the register always 0.

3. NtQuerySystemInformation:

This is another function in Windows NT that allows programs to query the system for various information, such as running processes, resources used, or the status of the debugger.

And it’s a structure that is returned when calling NtQuerySystemInformation with the identifier SystemKernelDebuggerInformation.

This structure contains two important variables to detect whether a kernel debugger is connected to the system or not:

• KdDebuggerEnabled:

If its value is non-zero (≠0), it means that the debugger is enabled in the system, and if its value is 0, it means that the debugger is disabled.

• KdDebuggerNotPresent:

If its value is 0, it means that there is a debugger connected to the system, and if its value is non-zero (≠0), it means that there is no debugger connected.

#include <windows.h>
#include <winternl.h>
#include <stdio.h>

typedef struct _SYSTEM_KERNEL_DEBUGGER_INFORMATION {
    BOOLEAN KdDebuggerEnabled;
    BOOLEAN KdDebuggerNotPresent;
} SYSTEM_KERNEL_DEBUGGER_INFORMATION;

typedef enum _SYSTEM_INFORMATION_CLASS {
    SystemKernelDebuggerInformation = 35
} SYSTEM_INFORMATION_CLASS;

extern "C" NTSTATUS NTAPI NtQuerySystemInformation(
    SYSTEM_INFORMATION_CLASS SystemInformationClass,
    PVOID SystemInformation,
    ULONG SystemInformationLength,
    PULONG ReturnLength
);

int main() {
    SYSTEM_KERNEL_DEBUGGER_INFORMATION debuggerInfo;
    NTSTATUS status = NtQuerySystemInformation(SystemKernelDebuggerInformation, &debuggerInfo, sizeof(debuggerInfo), NULL);

    if (status == 0) {
        printf("KdDebuggerEnabled: %d\n", debuggerInfo.KdDebuggerEnabled);
        printf("KdDebuggerNotPresent: %d\n", debuggerInfo.KdDebuggerNotPresent);

        if (debuggerInfo.KdDebuggerEnabled && !debuggerInfo.KdDebuggerNotPresent) {
            printf("Kernel debugger detected!\n");
        } else {
            printf("No kernel debugger detected.\n");
        }
    } else {
        printf("NtQuerySystemInformation failed!\n");
    }

    return 0;
}

If you are working in malware analysis, this scan can be bypassed by Debugging return values ​​to make KdDebuggerEnabled always appear as 0.

4. OutputDebugString:

This Windows API function is used to send debug messages to the debugger if it is connected to the process, how?

This is done using SetLastError and GetLastError to see if the OutputDebugString succeeded or not, and the idea is that the function doesn’t return an error when the debugger is running, but fails when it is not present.

SetLastError(5);
OutputDebugString("testing123");  
if (GetLastError() != 5) { 

    AntiDebugDetected();
}

here adummy error code is set using SetLastError(5).

OutputDebugString is called to send a string (testing123).

The current error code is returned using GetLastError().

If the error code does not change (GetLastError() == 5), then OutputDebugString did not change the error, indicating that a debugger is connected, and if the error code changes, then OutputDebugString failed, indicating that there is no debugger.

This can be defused or bypassed by several ways:

API Hooking

After calling OutputDebugString, SetLastError(5) can be called again to trick the code.

NtSetInformationThread can be used with ThreadHideFromDebugger to hide the debugger from all scanning.

5. CloseHandle and NtClose:

CloseHandle and NtClose are two functions in the Windows API that are used to close handles associated with objects such as files, processes, and threads.

When you try to close an invalid handle, the system throws an EXCEPTION_INVALID_HANDLE exception.

If there is no debugger connected, the process will be terminated or the error will be handled internally.

mov ebx, [invalid_handle]
call NtClose

If there is a debugger connected, it will automatically catch this exception (EXCEPTION_INVALID_HANDLE). The malware can monitor whether the exception is caught or not. If it is caught, it means that a debugger is running, and it can then implement countermeasures, such as TerminateProcess.

This can be bypassed by several ways:

Use the Ignore All Exceptions option in debugging tools like x64dbg or xdbg so that EXCEPTION_INVALID_HANDLE is not handled by the debugger.

Use SEH (Structured Exception Handling) to handle the error internally

API Hooking can be used to intercept NtClose and force it to return a value indicating success even if the handle is invalid.

6. NtQueryObject:

This function takes 3 main parameters:

Handle → The handle of the object we want to retrieve information about.

ObjectInformationClass → Specifies the type of information we want to retrieve about the object. ObjectInformation → A pointer to a location in memory where the retrieved data will be stored. When NtQueryObject is called with ObjectInformationClass set to 3 (ObjectAllTypesInformation), a list of all object types present in the system will be retrieved, including DebugObject, But how the malware use it?

The malware calls NtQueryObject with ObjectInformationClass = 3 to retrieve a list of objects.

It scans the retrieved data (ObjectInformation) for a DebugObject.

If it finds a DebugObject, it means that a Debugger is active or was active in the past, and if a Debugger is detected, the malware may implement counter-tactics, such as:

ExitProcess.

Trigger malicious behavior only if no Debugger is detected.

Obfuscate code or disable certain functions to prevent analysis.

This can be bypassed by several ways:

Modifying data returned from NtQueryObject via API Hooking and The code can be modified so that NtQueryObject returns a list that hides the DebugObject object.

Also running Kernel Debugging makes hidden objects not appear in NtQueryObject.

7. Heap Flags:

When any process runs in Windows, a portion of memory known as Heap is allocated, which is where dynamic data that is created while the program is running is stored.

And The PEB (Process Environment Block) contains important information about the process, such as loaded libraries and the location of the Heap.

On 32-bit systems, the process pointer to the Heap is at 0x18 inside the PEB.

On 64-bit systems, the pointer is at 0x30 inside the PEB.

Then you have Flags and ForceFlags.

So, what are Flags and ForceFlags? Inside the Heap structure, there are two important variables:

Flags: Contains information about the state of the memory being used. ForceFlags: Sometimes used to enforce certain behaviors specific to the Heap, such as running scans or enabling security features. On Windows 7 and later, when the process is run inside the Debugger, the values ​​of these variables are changed to:

Flags = 0x40000062 ForceFlags = 0x40000060

When executing malware, it may attempt to check whether it is running inside a debugging environment by reading the Flags and ForceFlags values.

and if the values ​​are 0x40000062 or 0x40000060, the program can detect that it is running inside a Debugger.

Also, ways to Read Values ​​Inside a Heap Malware can use one of the following methods to extract these values:

Calling a Windows API:

RtlQueryProcessHeapInformation RtlQueryProcessDebugInformation These functions are used to access information in the Heap, and the program can analyze the stored values ​​to detect any changes that indicate the presence of a Debugger.

Reading the PEB Manually

The program can read the PEB data directly using Assembly instructions such as FS:[0x30] (on 32-bit systems) or GS:[0x60] (on 64-bit systems). Then, the values ​​for Flags and ForceFlags can be extracted and verified.

Since the Flags and ForceFlags values ​​are read from the PEB, we can modify them at runtime so that they do not indicate the presence of a Debugger.

You can intercept RtlQueryProcessHeapInformation and RtlQueryProcessDebugInformation calls so that they return false values.

8. Directly Accessing the PEB:

When a program runs on Windows, a lot of process-specific information is stored in the Process Environment Block (PEB) as we said.

Some fields within the PEB reveal whether the program is running under a debugger, and malware can read these values ​​to detect the presence of dynamic analysis.

So, instead of using Windows APIs like IsDebuggerPresent or CheckRemoteDebuggerPresent, some malware directly accesses PEB using Assembly code, as in the following example:

--snip--
mov eax, [fs:0x30]
cmp [eax+0x2], 1 ; BeingDebugged
jnz DebuggerDetected
--snip--

mov eax, [fs:0x30]: Gets the address of the PEB in the EAX register.

On 32-bit systems, the PEB is located at fs:[0x30], while on 64-bit systems it is located at gs:[0x60].

cmp [eax+0x2], 1: Compares the value stored at eax+0x2 (which represents BeingDebugged in the PEB) with 1.

If the value is 1, the program is running under a Debugger.

jnz DebuggerDetected: If the result is not 0, the program executes some code in response to the Debugger’s detection, such as stopping execution or executing malicious code.

Another way the malware checks the Debugger is by reading the NtGlobalFlag inside the PEB at offset 0x68:

mov eax, [fs:0x30]   ; getting the address of PEB
mov eax, [eax+0x68]  ; getting NtGlobalFlag
cmp eax, 0x70        ; comparing NtGlobalFlag with 0
je DebuggerDetected  ; debug is true if 0

NtGlobalFlag contains memory-related settings, and if a Debugger is attached to the process, its value will be 0x70.

Malware uses this as another way to detect Debuggers without calling Windows APIs that can be easily detected.

This can be bypassed by several ways:

You can change BeingDebugged (PEB+0x2) and NtGlobalFlag (PEB+0x68) to 0 inside memory.

Also, you can use DLL Injection + Hooking to intercept access to PEB and prevent values ​​from being read.

Summary:

sure, there are other methods you can use to achieve anti debugging but in this article i have explained the most famous one and small tips to bypass it.