Perplexed RE picoCTF write-up

Introduction

Perplexed is a reverse engineering challenge from picoCTF. The binary takes a password as input and verifies it through a bit-by-bit comparison against 23 bytes of hardcoded validation data. The goal is to figure out what password satisfies that comparison — which is the flag.

In this writeup we’ll analyze the binary using radare2 and r2ghidra, clean up the decompiled code, understand the verification logic, and write a solver in C to extract the flag. If you’re interested in the full journey behind solving this — including the wrong turns and the decompiler lies — check out the story writeup here: Story Write-Up.

Running the Binary

The first step is to make the binary executable and run it to get a feel for what it does:

$ chmod +x perplexed
$ ./perplexed
Enter the password: test-password
Wrong :(

It prompts for a password and tells us whether we got it right or wrong. Simple enough — now let’s figure out what the correct password is.

Checking for a Hardcoded Flag

Since the program is asking for a password, and this is a picoCTF challenge, the password is most likely the flag itself. The first thing to try is whether it’s simply hardcoded as a plaintext string in the binary:

$ strings perplexed | grep pico

Nothing. The flag isn’t sitting in plaintext — the verification logic is buried deeper in the binary. Time to reverse it.

Loading into Radare2

We could use objdump, but radare2 gives us a much richer analysis environment. Load the binary into r2:

$ r2 perplexed
WARN: Relocs has not been applied. Please use `-e bin.relocs.apply=true` or `-e bin.cache=true` next time
 -- give | and > a try piping and redirection
[0x00401070]>

Then run a full analysis with aaa:

[0x00401070]> aaa
INFO: Analyze all flags starting with sym. and entry0 (aa)
INFO: Analyze imports (af@@@i)
INFO: Analyze entrypoint (af@ entry0)
INFO: Analyze symbols (af@@@s)
INFO: Analyze all functions arguments/locals (afva@@F)
INFO: Analyze function calls (aac)
INFO: Analyze len bytes of instructions for references (aar)
INFO: Finding and parsing C++ vtables (avrr)
INFO: Analyzing methods (af @@ method.*)
INFO: Recovering local variables (afva@@@F)
INFO: Type matching analysis for all functions (aaft)
INFO: Propagate noreturn information (aanr)
INFO: Use -AA or aaaa to perform additional experimental analysis

Running pdf @ main gives us the disassembled main — but it's large and hard to follow raw. A decompiler will give us a much more readable starting point, so we'll use r2ghidra instead.

Decompiling main

r2ghidra is a Ghidra decompiler plugin for radare2 — install it with:

$ r2pm -ci r2ghidra

Then decompile main with pdg:

[0x00401070]> pdg @ main

bool main(int argc,char **argv,char **envp)

{
    bool bVar1;
    char *s;
    int64_t var_108h;
    int64_t var_100h;
    // ...
    uint32_t var_4h;

    s = NULL;
    var_108h = 0;
    var_100h = 0;
    // ...
    var_18h = 0;
    sym.imp.printf("Enter the password: ");
    sym.imp.fgets(&s,0x100,_reloc.stdin);
    var_4h = sym.check(&s);
    bVar1 = var_4h != 1;
    if (bVar1) {
        sym.imp.puts("Correct!! :D");
    }
    else {
        sym.imp.puts("Wrong :(");
    }
    return !bVar1;
}

pdg gives us a high-level C representation — easier to follow than raw assembly, though not always accurate (we'll see why shortly). The logic here is straightforward: read input, pass it to check, print the result. Notice that main is cluttered with dozens of local variables all initialized to zero and never used — this is deliberate obfuscation by the challenge author to bloat the code. The real action is inside check.

Also worth noting: check returns 0 for a correct password and 1 for wrong — bVar1 = var_4h != 1 flips this into a boolean. Keep that in mind.

Decompiling check

Now let’s decompile check:

[0x00401070]> pdg @ sym.check

//WARNING: Variable defined which should be unmapped: var_8h

ulong sym.check(char *arg1)

{
    int64_t iVar1;
    ulong uVar2;
    int64_t iVar3;
    char *s;
    int64_t var_50h;
    int64_t var_48h;
    unkbyte7 Stack_48;
    uint32_t var_2ch;
    uint32_t var_28h;
    int64_t var_24h;
    uint32_t var_1ch;
    uint32_t var_18h;
    int64_t var_14h;
    int64_t var_8h;

    iVar1 = sym.imp.strlen(arg1);
    if (iVar1 == 0x1b) {
        var_50h = 0x617b2375f81ea7e1;
        var_48h._0_7_ = 0x69df5b5afc9db9;
        var_48h._7_1_ = 0xd2;
        Stack_48 = 0xf467edf4ed1bfe;
        var_14h._0_4_ = 0;
        var_18h = 0;
        var_24h._0_4_ = 0;
        for (var_1ch = 0; var_1ch < 0x17; var_1ch = var_1ch + 1) {
            for (var_24h._4_4_ = 0; var_24h._4_4_ < 8; var_24h._4_4_ = var_24h._4_4_ + 1) {
                if (var_18h == 0) {
                    var_18h = 1;
                }
                var_28h = 1 << (7U - var_24h._4_4_ & 0x1f);
                var_2ch = 1 << (7U - var_18h & 0x1f);
                if (0 < (arg1[var_14h] & var_2ch) != 0 < (*(&var_50h + var_1ch) & var_28h)) {
                    return 1;
                }
                var_18h = var_18h + 1;
                if (var_18h == 8) {
                    var_18h = 0;
                    var_14h._0_4_ = var_14h + 1;
                }
                iVar3 = var_14h;
                iVar1 = sym.imp.strlen(arg1);
                if (iVar3 == iVar1) {
                    return 0;
                }
            }
        }
        uVar2 = 0;
    }
    else {
        uVar2 = 1;
    }
    return uVar2;
}

Intimidating at first glance. Nested loops, bitwise operations, r2ghidra’s ugly type annotations, variables and types with names like unkbyte7 and var_24h._4_4_. Don't panic — we'll break it down piece by piece.

Cleaning Up the Decompiled Code

Let’s make this readable by renaming variables based on their roles:

• arg1 → input: the password string we're validating
• iVar1 → length: the length of the input, returned by strlen
• uVar2 → status: the return value — 0 for correct, 1 for wrong
• iVar3 → removed: it was only used to compare against length, we can use input_char_idx directly
• char *s → removed: unused, either obfuscation or a decompiler artifact
• var_14h → input_char_idx: index into the input character array
• var_18h → input_bit_pos: tracks the current bit position within the input byte, cycles from 1 to 7 then resets to 0 — starts at 1, not 0, which is worth keeping in mind
• var_1ch → i: outer loop counter, 0 to 22
• var_24h → j: inner loop counter, 0 to 7
• var_28h → validation_mask: bitmask applied to the validation data
• var_2ch → input_mask: bitmask applied to the input
• var_50h, var_48h, Stack_48 → validation_bytes[3]: these three variables are contiguous on the stack and together hold the 23 bytes of data the input is validated against. We group them into a uint64_t array of 3 elements for clarity. We'll revisit the exact layout shortly.

Here’s the cleaned-up result:

c

uint64_t check(char *input) {
    uint64_t validation_bytes[3];
    uint32_t input_mask;
    uint32_t validation_mask;
    uint32_t i;
    uint32_t input_bit_pos;
    uint32_t input_char_idx;
    uint32_t j;
    uint64_t status;

    uint64_t length = strlen(input);
    if (length == 27) {
        validation_bytes[0] = 0x617b2375f81ea7e1;
        validation_bytes[1] = 0xd269df5b5afc9db9;
        validation_bytes[2] = 0xf467edf4ed1bfe;
        input_char_idx = 0;
        input_bit_pos = 0;
        j = 0;
        for (i = 0; i < 23; i++) {
            for (j = 0; j < 8; j++) {
                if (input_bit_pos == 0) {
                    input_bit_pos = 1;
                }
                validation_mask = 1 << (7 - j);
                input_mask = 1 << (7 - input_bit_pos);
                if (((input[input_char_idx] & input_mask) > 0) != ((validation_bytes[i] & validation_mask) > 0))
                    return 1;
                }
                input_bit_pos++;
                if (input_bit_pos == 8) {
                    input_bit_pos = 0;
                    input_char_idx++;
                }
                if (input_char_idx == strlen(input)) {
                    return 0;
                }
            }
        }
        status = 0;
    } else {
        status = 1;
    }
    return status;
}

Much more readable. The core logic is now visible: for each of the 23 validation bytes, we iterate over all 8 bits and compare each one against the corresponding bit in the input. If any bit mismatches, we return 1 (wrong). If we exhaust the input, we return 0 (correct).

Two things to flag before moving on:

1. validation_bytes[i] is still stepping by 8 bytes — we'll fix that shortly when we go back to assembly to verify the actual memory layout.
2. input_bit_pos starts at 1, not 0 — bit 0 of every input byte is never checked. That's intentional behavior, not a bug.

Understanding the Validation Data Layout

At first glance, this block from the decompiler is confusing:

var_50h = 0x617b2375f81ea7e1;
var_48h._0_7_ = 0x69df5b5afc9db9;
var_48h._7_1_ = 0xd2;
Stack_48 = 0xf467edf4ed1bfe;

r2ghidra is representing what looks like four separate variables with strange partial-access notation. The ._0_7_ and ._7_1_ syntax means "bytes 0 through 6" and "byte 7" of the same variable respectively — ghidra splitting a single 8-byte write into two partial writes because it got confused about the type. Stack_48 adds to the confusion by being named after offset 0x48, which is the same offset as var_48h. Ghidra is clearly struggling here.

The assembly tells the real story:

movabs rax, 0x617b2375f81ea7e1
movabs rdx, 0xd269df5b5afc9db9
mov qword [var_50h], rax       ; 8 bytes at rbp-0x50
mov qword [var_48h], rdx       ; 8 bytes at rbp-0x48
movabs rax, 0xf467edf4ed1bfed2
mov qword [var_41h], rax       ; 8 bytes at rbp-0x41

Three clean movabs instructions, each writing 8 bytes to consecutive stack locations. The layout is:

• rbp-0x50 (which r2 names as var_50h): bytes 0–7 → 0x617b2375f81ea7e1
• rbp-0x48: bytes 8–15 → 0xd269df5b5afc9db9
• rbp-0x41: bytes 16–22 → starts at offset 23 from the beginning

Notice that the third write starts at rbp-0x41 instead of the expected rbp-0x40 — that's a deliberate one-byte overlap. The byte 0xd2 is the last byte of var_48h and simultaneously the first byte loaded into rax for the third write. This is how the challenge packs exactly 23 bytes of validation data into three 8-byte writes with a one-byte overlap — and it's probably what threw ghidra off, causing it to misname the variable as Stack_48 instead of var_41h.

In our cleaned-up C code, we model this as a flat byte array:

uint8_t validation_bytes[23] = {
    0xe1, 0xa7, 0x1e, 0xf8, 0x75, 0x23, 0x7b, 0x61,  // var_50h (little-endian)
    0xb9, 0x9d, 0xfc, 0x5a, 0x5b, 0xdf, 0x69, 0xd2,  // var_48h (little-endian)
    0xd2, 0xfe, 0x1b, 0xed, 0xf4, 0x67, 0xf4          // var_41h (little-endian)
};

This is the ground truth — what the binary is actually doing, stripped of all decompiler noise.

A Note on the redundant shift masking: & 0x1f

You may have noticed this in the decompiled code:

validation_mask = 1 << (7 - j & 0x1f);
input_mask      = 1 << (7 - input_bit_pos & 0x1f);

The & 0x1f masks the shift amount to the lowest 5 bits, capping it at 31. In practice it's completely redundant — j goes from 0 to 7 and input_bit_pos cycles from 1 to 7, so the shift amount will never exceed 7. Ghidra just emits it as a blanket safeguard regardless. We can safely drop it in the cleaned-up code.

Fixing Ghidra’s Stepping Mistake

There is one critical mistake in the cleaned-up code that ghidra is responsible for. In this line:

validation_bytes[i] & validation_mask

validation_bytes is a uint64_t array, so indexing it with i steps 8 bytes at a time — validation_bytes[0] is bytes 0–7, validation_bytes[1] is bytes 8–15, and so on. But we only have 23 bytes of validation data and 23 outer iterations — one iteration per byte. Stepping 8 bytes at a time would blow past the valid data almost immediately and read garbage from the stack, which is exactly what I noticed when I ran my solver that we'll see shortly.

The fix is to use a uint8_t pointer instead, which steps exactly 1 byte at a time:

uint8_t *vptr = (uint8_t *)validation_bytes;
// then use vptr[i] instead of validation_bytes[i]

The clue that exposed ghidra’s mistake was the number 23 itself — 23 outer iterations and exactly 23 bytes of validation data. Those two numbers matching is a strong signal that the loop is meant to process one byte per iteration, not one 8-byte chunk. Going back to the assembly confirmed it.

There is also a subtle but important difference in the comparison condition. The original decompiled code uses 0 < on both sides — it's not comparing the raw masked values directly, it's checking whether each side is non-zero. The correct cleaned-up condition is therefore:

if (((input[input_char_idx] & input_mask) != 0) != ((vptr[i] & validation_mask) != 0))

This reads as: if one side has the bit set and the other doesn’t — reject. We use != 0 instead of > 0 since the values are unsigned and can never be negative, but the semantics are identical.

How the Bit Comparison Works

Let’s trace through the first iteration to understand what the code is actually doing.

At the start, i = 0, j = 0, input_char_idx = 0, and input_bit_pos = 0. The first thing that happens inside the inner loop is:

if (input_bit_pos == 0) input_bit_pos = 1;

So input_bit_pos is immediately bumped to 1 — bits in a byte are labeled from position 7 (MSB) down to position 0 (LSB), giving 8 bits total. Since the mask is computed as 1 << (7 - input_bit_pos), and input_bit_pos starts at 1, position 7 is skipped from the very first iteration. This makes sense: in standard ASCII, bit 7 is always 0, so comparing it carries no useful information. Skipping it is intentional.

The masks are then computed:

validation_mask = 1 << (7 - j);             // 1 << 7 = 0x80 → bit 7
input_mask      = 1 << (7 - input_bit_pos); // 1 << 6 = 0x40 → bit 6

So in the very first iteration, we’re comparing bit 6 of the first input character against bit 7 of the first validation byte. The comparison:

if (((input[input_char_idx] & input_mask) != 0) != ((vptr[i] & validation_mask) != 0))
    return 1;

checks whether the two bits differ in being set or not — if one is set and the other isn’t, the password is rejected immediately. This pattern continues across all 8 inner iterations, consuming 7 bits from the current input character (bits 6 down to 0) while consuming all 8 bits of the current validation byte. When input_bit_pos reaches 8 it resets to 0, which gets bumped to 1 again on the next iteration, and input_char_idx advances to the next input character. Visually, for the first input character ’p’ (0x70) compared against the first validation byte 0xe1:

                                   skipped MSBs
                                          |
                    +---------------------+
                    v                     v
input 'pi' 0x7069 = 0 1 1 1 0 0 0 0 | 'i' 0 1 1 0 1 0 0 1    ...
                     / / / / / / /  |      / / / / / / /
validation 0xe1a7 = 1 1 1 0 0 0 0 1-----> 1|1 0 1 0 0 1 1 1  ...
                                  ^ _____/              \_/
                                  |/                     |
                          this bit gets          these two bits will get left
                          left for the           to for comparison with the next
                          next character         character
                          comparison

The input contributes 7 bits (bits 6–0), the validation contributes 8 bits (bits 7–0), with the last validation bit left to be compared against the first bit of the next input character.

A Note on Bit Coverage

One last detail worth noting: the input is 27 characters, each contributing 7 usable bits, giving 27 × 7 = 189 bits total. But the validation data is only 23 bytes — 184 bits. The 5 extra bits are never checked, because the loop exits early via:

if (input_char_idx == strlen(input))
    return 0;

Once all 184 validation bits are consumed, the function returns 0 immediately. This means the last input character only needs its first 2 bits (bits 6 and 5) to match — the remaining 5 bits are completely unchecked and can be anything.

Writing the Solver

Instead of tracing through 184 iterations manually or stepping through them in GDB, we write a solver. The idea is simple: take the check function, rip out the verification condition, and replace it with reconstruction — for each bit position, if the corresponding validation bit is set, we set the same bit in our output character.

c

#include <stdint.h>
#include <stdio.h>

int main(void) {
    uint64_t validation_bytes[3];
    uint32_t input_mask;
    uint32_t validation_mask;
    uint32_t i;
    uint32_t input_bit_pos;
    uint32_t input_char_idx;
    uint32_t j;
    uint64_t status;
    uint8_t *vptr;
    unsigned char passwd_char = 0;
    uint64_t length = 27;
    if (length == 27) {
        validation_bytes[0] = 0x617b2375f81ea7e1;
        validation_bytes[1] = 0xd269df5b5afc9db9;
        validation_bytes[2] = 0xf467edf4ed1bfe;
        vptr = (uint8_t *) validation_bytes; // step 1 byte at a time, not 8 (uint64_t would step 8)
        input_char_idx = 0;
        input_bit_pos = 0;
        for (i = 0; i < 23; i++) {
            for (j = 0; j < 8; j++) {
                if (input_bit_pos == 0) {
                    input_bit_pos = 1;
                }
                validation_mask = 1 << (7 - j);
                input_mask = 1 << (7 - input_bit_pos);
                /* instead of verifying:
                if (((input[input_char_idx] & input_mask) != 0)
                    != ((vptr[i] & validation_mask) != 0))
                    return 1;
                */
                /* we reconstruct: if the validation bit is set,
                   set the corresponding bit in our output character */
                if ((vptr[i] & validation_mask) != 0) {
                    passwd_char |= (uint8_t) input_mask;
                }
                input_bit_pos++;
                if (input_bit_pos == 8) {
                    input_bit_pos = 0;
                    input_char_idx++;
                    printf("%c", passwd_char);
                    passwd_char = 0;
                }
                if (input_char_idx == 27) {
                    return 0;
                }
            }
        }
        printf("\n");
        status = 0;
    } else {
        status = 1;
    }
    return status;
}

We compile and run it:

$ gcc solver.c -o solver && ./solver
picoCTF{0n3_bi7_4t_a_7im3}

Flag captured. 🚩

When the Solver Was Still Broken

This is also where ghidra’s uint64_t stepping mistake became obvious. Before fixing the pointer type, the solver was using validation_bytes[i] directly — stepping 8 bytes at a time.

The first character p came out correct — validation_bytes[0] is the right 8 bytes. From index 1 and 2 the pointer still lands within the 23-byte boundary, but skips everything in between, producing wrong characters. From index 3 onward it flies completely past the valid data and reads garbage from the stack, producing a different random-looking result on every run. Switching to a uint8_t * pointer fixed it immediately.

Conclusion

The Perplexed binary is a great example of how a conceptually simple operation — comparing bits — can be made to look terrifying through a combination of deliberate obfuscation and decompiler noise. Dozens of unused variables in main, cryptic r2ghidra type annotations, partial variable access notation, and a bit-shuffling loop that looks scarier than it is — all working together to slow you down.

The two real challenges were: understanding the bit-by-bit comparison structure, and catching ghidra’s uint64_t stepping mistake. The first required patience and careful variable renaming. The second required going back to assembly — as it always does.

The lesson that keeps coming back: decompilers are a starting point, not a source of truth. When something feels off, the assembly is always there to set the record straight.