Threat Hunting & Detection Engineering: The Batman & Robin of Modern SOC

The 3 AM PagerDuty Wake-Up Call

Picture this: It’s 3 AM. Your phone buzzes violently on the nightstand. Your SIEM has generated 15,000 alerts in the past hour, and 14,990 of them are false positives. Your Tier 1 analyst is crying into their third cup of coffee. The one real incident — a domain admin account doing something very naughty at 2:47 AM — is buried somewhere in the noise.

Sound familiar?

If you’ve worked in a SOC for more than a week, you’ve lived this nightmare. And if you haven’t, congratulations — you’re either lying or you work for a company with infinite budget and a time machine.

This is exactly why modern Security Operations Centers need two specific roles working in tandem: Threat Hunters and Detection Engineers. Not as nice-to-haves. As core functions.

Let me take you on a practical journey through what these roles actually do, why they matter, and how they can save your SOC from drowning in its own data.

First, Let’s Talk About What “Modern SOC” Actually Means

I’ve been in this industry long enough to remember when a “modern SOC” meant you had a SIEM that could ingest logs without crashing and maybe one analyst who knew how to write a basic correlation rule.

Spoiler: Those days are gone. Like, really gone. Like, “we still have Internet Explorer in production” gone.

Today’s SOC landscape is… complicated. We’re dealing with:

• Cloud environments that spawn and die faster than mayflies
• Endpoints everywhere — laptops in cafes, servers in who-knows-which cloud region
• IoT devices that scream “hack me” in 50 different languages
• Attackers who read the same threat intel reports we do

Image Source : Gartner Modern SOC Components

Gartner has this nice matrix showing SOC capabilities, but honestly? The real question isn’t “what tools do you have?” It’s “what can you actually do with those tools?”

Because here’s the thing: You can buy the most expensive NG-SIEM with AI/ML, SOAR, UEBA, EDR, NDR, and CTI. You can throw money at the problem until your CFO weeps.

But if you don’t have people who understand threats, data, and detection working together? You’ve just built a very expensive alert generator.

So What Actually Is Threat Hunting?

Let me tell you a story.

Back when I was a young, idealistic analyst (before the coffee addiction set in), I worked at a SOC where our philosophy was simple: “If the tools don’t alert, it’s not a problem.” Then i will wait for the next alert while playing the FIFA Game with my comrades Matdhule.

We were wrong. Spectacularly wrong.

Here’s what I’ve learned since then:

Threat hunting is the proactive discovery of threats that have evaded your existing security controls.

It’s not waiting for alerts. It’s not running IOCs from threat intel feeds (though that’s part of it). It’s forming hypotheses and going looking for trouble before trouble finds you. (Bahasa Indonesia Joke : Threat Hunting dengan IoC adalah selemah-lemahnya Iman)

The official definition we use at BlueTeam.ID:

“Threat hunting is a proactive cyber defense approach. Threat hunting processes perform proactive and iterative discovery through networks, endpoints, and other infrastructure to detect and respond to cybersecurity threats that sometimes evade existing security solutions.”

In plain English: You’re looking for the bad guys who are smart enough to not trigger your alarms.

The SQRRL Hunting Loop

When we mature our hunting operations, we use something called the SQRRL Hunting Loop:

1. S — Strive to understand your environment
2. Q — Form Questions/hypotheses
3. R — Acquire and analyze Relevant data
4. R — Document Results
5. L — Capture Lessons learned

The last step — lessons learned — is where the magic happens. Because after every hunt, you need to ask:

• What could we do better?
• Are there alternate detection capabilities we should build?
• Can this behavior be turned into a detection rule?
• If not, how often do we need to hunt for this manually?

Funny story: Early in my hunting career, I once spent three weeks chasing what I thought was an elaborate APT campaign. Turned out to be a developer who’d written a script to automatically deploy code at 3 AM because “that’s when production is quietest.” The lessons learned? “Ask developers what they’re doing before assuming it’s nation-state actors.” Also: “Maybe don’t give devs production access at 3 AM.”

And Detection Engineering? What’s That?

If threat hunting is the detective, detection engineering is the forensic scientist who turns evidence into repeatable processes.

Here’s how Trellix defines it in their whitepaper:

“Detection engineering is a set of practices and systems to deliver modern and effective threat detection. When building a solid detection engineering program, the main goal is to catch malicious things and to not catch too many not malicious things.”

That second part is crucial.

A joke for you:

How many detection engineers does it take to change a lightbulb?

None — they’ll write a rule to detect when the lightbulb is out, but then it fires 50 times a day because people keep flicking the switch, and now the maintenance team hates them.

The Detection Engineering Cycle

The cycle typically looks like this:

1. Threat Discovery — Find something evil (often from threat hunting or intel)
2. Data Analysis — Understand what logs/telemetry show this behavior
3. Detection Development — Build the rule/query/model
4. Testing & Validation — Make sure it catches bad stuff and ignores good stuff
5. Deployment — Push to production
6. Maintenance — Update as attackers change tactics

And then back to step one.

Common Mistake Alert!

The most common mistake I see? Building detections without understanding the threat or the data.

I’ve watched teams write hundreds of Sigma rules based purely on MITRE ATT&CK descriptions, never once checking if their environment actually generates the necessary logs. The result? Beautiful dashboards showing zero detections for techniques that definitely would have triggered if anyone had bothered to enable the right logging.

It’s like installing a state-of-the-art alarm system… in an empty house. With no doors. And the windows already broken.

The Beautiful Relationship: How They Work Together

Here’s where it gets interesting.

Threat hunting and detection engineering aren’t separate functions. They’re not even “sister disciplines.” They’re more like… Batman and Robin. Or peanut butter and jelly. Or coffee and Monday mornings.

Threat Hunting identifies gaps → Detection Engineering minimizes them

Hunters uncover blind spots or missed behaviors. Engineers convert those findings into new rules, logic, or analytics.

Detection Engineering provides foundational coverage → Threat Hunting investigates beyond it

Engineers build detections for known threats. This gives hunters room to pursue unknown, stealthy, or emerging techniques without drowning in basic alerts.

Detection Engineering operationalizes hunting insights

When hunters discover new patterns, detection engineers translate them into production-ready signatures, queries, or behavioral models.

Continuous feedback loop improves maturity

Every hunt produces intelligence that sharpens detections. Every new detection gives hunters a better starting point.

Shared understanding drives threat-informed defense

Hunters reveal real attacker behaviors; engineers codify them. The result is defenses aligned with actual threats, not assumptions.

SANS DFIR Summit 2020 : https://www.youtube.com/watch?v=Ut1t_n6NPQE

Real-World Example

Let me give you a scenario I’ve seen play out multiple times:

The Hunt: A threat hunter is investigating anomalous PowerShell usage. They discover attackers are using a specific obfuscation technique that somehow bypasses all existing detections.

The Investigation: The hunter traces the activity, determines the scope of compromise, and documents the TTPs (Tactics, Techniques, and Procedures) used.

The Handoff: The hunter meets with the detection engineering team. They walk through the logs, explain what made this attack successful, and discuss potential detection strategies.

The Engineering: The detection engineers develop a new rule that detects this obfuscation pattern. They test it against historical data, tune it to reduce false positives, and deploy it.

The Result: Next week, when another attacker tries the same technique, the detection fires immediately. The SOC responds within minutes instead of days. The hunters move on to finding the next blind spot.

That’s the dream, right?

But Wait — Are They the Same Thing?

This is a question I get all the time. At meetups, at conferences, in Twitter DMs.

“Isn’t detection engineering just threat hunting with extra steps?”

“When a detection engineer prototypes a new rule, aren’t they hunting?”

“Do we really need separate roles?”

Here’s my take:

They’re distinct but overlapping. A detection engineer might hunt when prototyping — exploring data, looking for patterns, understanding behavior. A threat hunter might do detection engineering when documenting findings and suggesting new rules.

The difference is in the primary objective:

• Threat Hunting: Find evil that’s already in your environment
• Detection Engineering: Build capabilities to find evil next time

Or as someone smarter than me put it:

“Threat Hunting finds what’s missing. Detection Engineering builds what’s needed.”

Another joke, because you’re still reading:

A threat hunter and a detection engineer walk into a bar.

The threat hunter immediately starts checking under tables, behind the counter, and in the bathroom.

The detection engineer says, “What are you doing?”

The hunter replies, “Looking for threats that might be hiding.”

The engineer nods, pulls out a laptop, and starts writing a YARA rule for “suspicious bar behavior.”

The bartender looks at both of them and says, “You know, most people just order a drink.”

The Technical Nitty-Gritty: How It Actually Works

Let’s get into the weeds a bit. Because theory is nice, but you’re here for the practical stuff.

What You Need to Know About Threats

To hunt or build detections effectively, you need to understand:

• Past incidents — What’s hit you before? What’s hit similar organizations?
• Threat intelligence reports — What are attackers doing right now?
• Security advisories — What new vulnerabilities are being exploited?
• Frameworks — MITRE ATT&CK, Pyramid of Pain, Cyber Kill Chain
• Conference talks and blog posts — The community shares so much good stuff

Pro tip: I spend at least an hour every day just reading. Threat intel feeds, Twitter (yes, still calling it Twitter), LinkedIn, academic papers, random blogs. You never know where the next insight will come from.

What You Need to Know About Data

Threats tell you what to detect. Data tells you where to detect it.

You need to understand your logs and telemetry across three domains:

1. Network — Firewall logs, DNS, NetFlow, PCAPs, NDR sensors
2. Endpoint — EDR telemetry, process creation, registry changes, file system activity
3. Cloud — Cloudtrail, audit logs, identity provider logs, container orchestration

Common mistake: Assuming you have data you don’t actually have. I can’t tell you how many times I’ve asked a client, “Do you have PowerShell script block logging enabled?” and watched them slowly realize… no. No they don’t.

What You Need to Know About Detection Methods

Different threats require different detection approaches:

• IOCs (Indicators of Compromise) — Simple, but attackers change them constantly
• TTPs (Tactics, Techniques, Procedures) — Harder to implement, but more durable
• Correlation rules — Combine multiple data sources
• Dashboards — Visual monitoring for suspicious patterns
• Thresholds — Detect when things exceed normal baselines
• Machine learning — Find anomalies without predefined rules

The mature organizations use all of these, layered together.

Meme Stolen From Somewhere

Building a Threat-Informed Defense

At MITRE, they talk about “threat-informed defense” — applying a deep understanding of adversary tradecraft to protect against, detect, and mitigate cyber-attacks.

This isn’t just academic. It’s practical.

When you understand how attackers actually operate — not just the high-level techniques, but the specific behaviors, the tools they use, the infrastructure they prefer — you can build defenses that actually stop them.

And the beautiful thing? When you force attackers to change their TTPs because you keep detecting them, you’ve won. You’ve imposed cost on the adversary. You’ve made your environment too expensive to attack.

That’s the ultimate goal:

• Lower the average dwell time (days from compromise to detection)
• Impose cost on adversaries (make them change their tools and techniques)
• Detect malicious behavior quicker and more frequently

Common Mistakes I’ve Made (So You Don’t Have To)

Let me share some personal failures, because we learn more from failures than successes.

Mistake #1: The “Everything Must Be Automated” Trap

Early in my career, I was obsessed with automation. Every hunt, every finding, had to become an automated detection.

The result? A detection backlog of 47 items, most of which were either impossible to automate or produced so many false positives that analysts ignored them entirely.

Lesson: Not everything needs to be a rule. Some things should stay manual hunts. Some things should be dashboards. Some things should be training for analysts. Automation is a tool, not a religion.

Mistake #2: Hunting Without a Hypothesis

I used to just… look at data. Scroll through logs. Hope something jumped out.

Spoiler: Nothing jumped out. Because without a hypothesis, you’re not hunting — you’re browsing. And browsing is what we do on weekends when we’re bored, not what we do when we’re looking for sophisticated adversaries.

Lesson: Start with a question. “Is anyone using PowerShell to download executables from non-corporate domains?” “Are there signs of pass-the-hash in domain admin activity?” “Has anyone accessed the CEO’s laptop at 3 AM?” Questions focus your search.

Did you guys join my past Online Webinar about Investigation Theory and Investigation Mindset? That’s the clue there (I might be posted those recording later)

Mistake #3: Ignoring the “People” Part

I once built what I thought was the perfect detection. Elegant logic. Minimal false positives. Covered a critical attack technique.

And it sat in our queue for six months because I never talked to the analysts who would use it. Never explained what it did. Never showed them how to investigate when it fired.

Lesson: Detection engineering isn’t just writing rules. It’s enabling people. If your analysts don’t understand or trust your detections, they might as well not exist.

Mistake #4: The “Set It and Forget It” Approach

This one still hurts.

I wrote a rule, tested it, deployed it, and patted myself on the back. Six months later, it had fired exactly zero times. I was proud — until I realized the attacker had changed their technique slightly, and my rule no longer matched.

Lesson: Detections decay. Attackers evolve. Environments change. You have to maintain your detection library like you maintain your code — constant updates, testing, and refinement.

So… What Does Success Look Like?

When threat hunting and detection engineering work well together, here’s what happens:

1. The SOC has baseline coverage — Known threats are detected reliably
2. Hunters have freedom — They can explore anomalies without getting pulled into every alert
3. Detection rules improve over time — Each hunt produces insights that make future detections better
4. Analysts trust their tools — Because false positives are minimized and detections actually work
5. Dwell time decreases — Attackers are found faster
6. Adversaries get frustrated — They have to keep changing their techniques because you keep detecting them

Another joke, because you’ve earned it:

An attacker calls his friend after a failed breach attempt.

“It didn’t work. They detected us within five minutes.”

His friend asks, “Did they have next-gen AI-powered behavioral analytics?”

“No.”

“EDR with machine learning?”

“No.”

“Threat intel feeds from 50 different sources?”

“No. They just had a really bored analyst who noticed our beacon because ‘the timing looked weird.’”

Practical Takeaways: Where to Start

If you’re convinced (or at least curious) and want to start building these capabilities, here’s my advice:

For Threat Hunting

1. Start with a framework — MITRE ATT&CK is your friend. Pick a technique each week and hunt for it.
2. Document everything — What did you look for? What did you find? What would you do differently?
3. Share findings — With detection engineers, with analysts, with leadership. Make hunting visible.
4. Automate the boring parts — If you’re doing the same hunt every month, it should probably become a detection.

For Detection Engineering

1. Understand your data — Before writing any rule, know exactly what logs you have and what they contain.
2. Start with quality, not quantity — Five excellent, low-noise detections are better than fifty noisy ones.
3. Test, test, test — Against known bad, against known good, against historical data.
4. Talk to hunters and analysts — They’ll tell you what’s missing and what’s broken.

For Both

1. Embrace the feedback loop — Hunters tell engineers what’s missing; engineers tell hunters what’s covered.
2. Learn constantly — Read threat reports, attend conferences (like this meetup!), follow researchers.
3. Be patient — This stuff takes time. Maturity doesn’t happen overnight.
4. Remember the goal — It’s not about cool tools or fancy rules. It’s about stopping attackers.

Image Taken From : https://www.ironnet.com/blog/building-a-better-detection-ecosystem

The Bigger Picture

A modern SOC isn’t just about technology. It’s not even just about people. It’s about integration — how different capabilities work together.

Threat intelligence informs hunts. Hunts inform detections. Detections catch threats. Caught threats generate new intelligence. And around and around it goes.

When it works, it’s beautiful. Your SOC becomes resilient, adaptive, and genuinely capable of detecting, understanding, and responding to evolving threats.

When it doesn’t work? You get the 3 AM alert storm I described at the beginning.

When a threat hunt successfully uncovers malicious activity that evaded existing defenses, that discovery shouldn’t just live in a report — it must graduate into a detection rule.

The hunter’s hard-won knowledge about the adversary’s specific TTPs gets handed to detection engineers, who transform it into automated alerts, signatures, or analytics that will catch that same technique next time.

This graduation ceremony is where proactive hunting creates lasting value: every successful hunt that graduates become a detection rules means one less blind spot in your environment, one less technique attackers can reuse, and more freedom for hunters to pursue the next unknown threat instead of manually re-discovering old ones.

Final thought (and a last joke):

Why do threat hunters and detection engineers make such a good team?

Because hunters find the bodies, and engineers figure out how to stop the murder.

Want to Learn More?

• MITRE ATT&CK: https://attack.mitre.org/
• SQRRL Hunting Loop: https://medium.com/@sqrrldata/the-hunting-loop-10c7d451dec8
• Detection Engineering Whitepaper (Trellix): https://www.trellix.com/assets/whitepapers/detection-engineering-whitepaper.pdf
• BlueTeam.ID Community: https://blueteam.id
• ThreatHunting.ID: https://threathunting.id

Thanks for reading! This article is based on my presentation at the CDEF 28th Meetup (November 2025). If you’re interested in threat hunting, detection engineering, or just want to argue with me about SIEM optimizations, find me on Twitter @digitoktav or connect on LinkedIn.

And yes, I really did spend three weeks hunting a developer’s deployment script. No, I’m still not over it.