Threat Detection at the National Level, Part 1: Building Coverage Across Network, DNS, and Endpoint Telemetry

Part 1 of a three part series on national threat detection. This installment covers the telemetry foundation: the fundamental data sources that make detection possible and what each one is good for.

Detecting threats at a national scale is a fundamentally different problem from defending a single enterprise. The defender is not protecting one network but coordinating visibility across many independent organizations, sectors, and jurisdictions, often without direct control over any of them. This post walks through the core telemetry sources that make national detection possible and explains what each one is good for and where it falls short. NetFlow gives broad but shallow visibility into how traffic moves. DNS data reveals intent and command and control patterns that flows alone hide. Endpoint Detection and Response (EDR) and Windows Event logs provide deep, high fidelity context about what actually executed on a host. SELinux and AppArmor add a Linux enforcement and audit layer that turns policy violations into detectable signals. The recurring theme is that no single source is sufficient. National detection works only when these layers are normalized, correlated, and reasoned about together at a scale and latency that most organizations never have to contemplate.

Introduction: The Problem of Scale and Sovereignty

A security operations center inside a single company has a bounded problem. It knows its asset inventory, owns its logging pipeline, and can mandate agents on every endpoint. National threat detection discards almost all of those assumptions.

At the national level the defender is typically a government cyber agency, a national CERT, or a sector regulator that must reason about adversary activity spanning telecommunications providers, energy operators, financial institutions, hospitals, and government departments. These entities run different technology stacks, retain logs for different periods, operate under different legal constraints, and frequently compete with one another. The defender rarely has root on any of their machines and often cannot even compel them to share raw data.

This creates three pressures that shape every design decision. The first is volume. A mid sized country can generate tens of terabytes of network and log telemetry per day, and the useful signal is a vanishingly small fraction of it. The second is heterogeneity. The same attack technique looks different across a Windows domain controller, a Linux web server, and a network sensor at an internet exchange, so detections must be expressed in terms that survive translation across all of them. The third is privacy and sovereignty. Bulk collection of citizen and corporate traffic carries legal and ethical weight, which pushes national programs toward metadata rather than content, and toward federation rather than central hoarding of raw data.

The rest of this post examines the telemetry layers that national programs lean on, why each exists, and how they complement one another. The argument throughout is that detection coverage is a stack, not a single tool, and that gaps in any one layer are routinely exploited by capable adversaries.

NetFlow Data: Visibility into How Traffic Moves

NetFlow and its relatives (IPFIX, sFlow, and vendor variants) summarize network conversations rather than capturing their contents. A flow record typically describes the source and destination addresses, source and destination ports, protocol, byte and packet counts, timestamps, and TCP flags for a single conversation. It is metadata about communication, not the communication itself.

This summarization is exactly what makes flow data viable at national scale. Capturing full packets across a country is rarely lawful, almost never affordable, and operationally unmanageable. Flow records compress traffic by several orders of magnitude while preserving the relationships that matter for detection.

The key analytic strengths of flow data are the following.

Beaconing detection. Command and control channels often produce regular, low volume connections to the same destination at fixed or jittered intervals. Flow records expose this rhythm even when the payload is encrypted, because the timing and size pattern survives encryption.

Exfiltration detection. Large or sustained outbound transfers to unusual destinations stand out in byte counts, particularly when the ratio of outbound to inbound bytes inverts the normal client server pattern.

Lateral movement and reconnaissance. Internal scanning and east to west movement appear as fan out patterns, where one host suddenly opens connections to many internal peers on administrative ports.

The corresponding weakness is depth. Flow data tells you that two endpoints talked and roughly how much, but not what was said. Encrypted traffic is opaque to it, and a patient adversary that mimics normal traffic volumes and timing can blend into the background. Flow is therefore best understood as a wide angle lens. It tells the analyst where to look, not what they will find when they get there.

DNS Data: Revealing Intent and Command Infrastructure

Almost every network action begins with a name resolution. That property makes DNS one of the highest leverage telemetry sources available, because it captures intent before a single byte of the actual connection flows.

DNS logs, whether collected at resolvers, passive sensors, or recursive infrastructure, record the queried names, the responses, query types, and timing. Several detection patterns depend almost entirely on this source.

Domain generation algorithm (DGA) detection. Many malware families compute large numbers of pseudo random domain names to locate their command servers, most of which do not resolve. The resulting bursts of failed lookups for high entropy, short lived names are a strong behavioral signal that no payload inspection could provide as cheaply.

DNS tunneling detection. Adversaries can smuggle data and control traffic inside DNS queries and responses by encoding it in subdomain labels. Unusually long query names, abnormal record types such as large volumes of TXT or NULL records, and high query rates to a single zone are the telltale signs.

Newly registered and low reputation domains. Attack infrastructure is frequently young. Correlating queries against domain age and reputation surfaces connections to infrastructure that legitimate users have no established history with.

Fast flux and infrastructure pivoting. Rapidly changing address records behind a single name, or many names sharing a small pool of addresses, expose resilient command and control hosting that traditional address based blocking cannot keep up with.

DNS data does have limits. Encrypted DNS transports such as DNS over HTTPS and DNS over TLS reduce visibility at the network layer and push collection toward the resolver itself, which is not always available to a national program. Even so, because resolution is so close to intent, DNS frequently provides the earliest and clearest indication that something is wrong.

EDR Data: Depth at the Endpoint

Where flow and DNS describe communication, Endpoint Detection and Response telemetry describes execution. EDR agents instrument the operating system to record process creation and lineage, command line arguments, file and registry modifications, module loads, network connections initiated by specific processes, and in memory behaviors such as injection and credential access.

This is the highest fidelity layer in the stack. It can answer questions that network data structurally cannot, such as which process initiated a suspicious connection, what parent spawned it, and what it did to the host afterward. Behavioral detections that map to adversary techniques, for example a document reader spawning a scripting interpreter that then reaches out to the internet, live almost entirely at this layer.

At national scale, EDR introduces distinctive challenges. Coverage is never complete, because agents cannot be deployed on unmanaged devices, legacy systems, or operational technology, and these gaps are precisely where determined adversaries prefer to operate. Data volume is enormous, since a single busy host can produce millions of events per day, which forces aggressive filtering and summarization before data ever leaves the endpoint. Finally, a national defender usually does not own the endpoints at all, so EDR derived signal arrives as alerts or curated extracts shared by organizations rather than as raw, queryable telemetry. The practical consequence is that national programs treat EDR less as a primary collection source and more as a deep validation layer, used to confirm and enrich hypotheses that broader network telemetry first raised.

Windows Event Data: The Authoritative Record of Host Activity

Windows event logs predate modern EDR and remain indispensable, particularly for identity and authentication analysis. The Security, System, and Application channels, supplemented heavily by Sysmon, form a structured record of what happened on a host.

A handful of event categories carry disproportionate detection value. Authentication events, including successful and failed logons and the logon type, reveal credential misuse, password spraying, and the use of stolen credentials across machines. Process creation events, especially when enriched with command lines, expose the execution of living off the land binaries and suspicious script interpreters. Kerberos and ticket events surface techniques against the authentication protocol itself, such as anomalous ticket requests associated with credential theft. Service installation, scheduled task creation, and certain registry changes mark the persistence mechanisms that adversaries rely on to survive reboots.

The principal difficulty with Windows event data is that default logging is shallow and noisy at the same time. Many of the most valuable events are not recorded unless audit policy is deliberately tuned, while the events that are recorded by default arrive in overwhelming volume. National programs that consume Windows telemetry therefore depend on contributing organizations to configure consistent audit policy and to forward a sensible subset, which is an organizational and governance problem at least as much as a technical one. When that groundwork is in place, Windows events provide the authoritative, identity centric narrative of an intrusion that other sources can only approximate.

SELinux and AppArmor: Turning Linux Policy into Detection Signal

Linux servers carry a large share of critical national infrastructure, yet they generate far less standardized security telemetry than Windows by default. This is where mandatory access control systems become relevant not only as a hardening measure but as a source of detection signal.

SELinux and AppArmor both enforce policies that constrain what processes are permitted to do, but they take different approaches. SELinux applies labels to subjects and objects and evaluates access against a comprehensive policy, which makes it powerful and granular but complex to author and reason about. AppArmor confines individual programs using path based profiles, which is simpler to deploy and understand at the cost of finer grained control. For detection purposes the important point is what each one produces when policy is violated.

SELinux records access vector cache denials in the audit log whenever a confined process attempts an action its policy forbids. A web server process trying to execute a shell, open an unexpected file, or bind to an unusual port produces a denial that is, in effect, a free intrusion signal. Because the policy encodes the intended behavior of the application, a denial often means the application is doing something it was never designed to do, which is frequently a hallmark of exploitation.

AppArmor similarly logs profile violations, recording when a confined program steps outside its declared profile. The same logic applies. A deviation from the profile is a deviation from intended behavior.

The strength of treating these as detection sources is that they are grounded in policy rather than in signatures or heuristics. They flag anomalies relative to an explicit specification of correct behavior. The weakness is operational. Many organizations run these systems in permissive or complaint only modes, or disable them entirely because of the effort required to author good policy, and a system that is not enforcing produces denials that no one acts on. For a national program, the audit streams from SELinux and AppArmor represent an underused source of high signal events, but only where the underlying policy is mature and the logs are actually collected.

Bringing the Layers Together

The central lesson is that these sources are complementary, not interchangeable, and an adversary will route their activity through whichever layer is least observed. A capable intrusion might begin with a DNS lookup for command infrastructure, proceed over an encrypted channel visible only as a NetFlow beacon, execute through a process chain captured by EDR and Windows events, and trip an SELinux denial when the compromised service attempts something outside its profile. Each layer sees a fragment. Only correlation across them reconstructs the whole.

Three principles make that correlation workable at national scale. The first is normalization. Events from radically different sources must be mapped onto a common schema and a shared vocabulary of adversary behavior, so that a detection written once can match evidence wherever it appears. The second is federation over centralization. Because of volume, privacy, and sovereignty, mature national programs increasingly push detection toward the data rather than pulling all raw data to the center, sharing alerts, indicators, and analytics rather than bulk telemetry. The third is prioritization by leverage. Given finite collection and analysis budgets, programs invest first in the sources that yield the most signal per unit of cost and privacy risk, which is why metadata heavy sources such as DNS and NetFlow often anchor national visibility while EDR and host logs serve as depth on demand.

Conclusion

National threat detection is less about any single sensor and more about composing a stack of partial views into something approaching a complete picture. NetFlow supplies breadth and shows how traffic moves. DNS exposes intent and the infrastructure behind an intrusion earlier than almost anything else. EDR and Windows event data provide the depth required to confirm what actually executed and under whose identity. SELinux and AppArmor convert explicit Linux policy into detection signal that flags behavior an application was never meant to exhibit.

The hard problems are rarely about the individual sources. They are about scale, heterogeneity, governance, and privacy, and about persuading independent organizations to collect consistently and share usefully. A national program that masters the normalization and correlation of these layers, while respecting the legal and ethical constraints that distinguish national defense from enterprise security, can detect activity that no single participant could see on its own. That collective visibility, more than any one feed, is what national detection ultimately delivers.