Scratching the surface of BotLens

Credits: Felipe Balduino

Affinity Answers is well known in the programmatic advertising world for the best breadth and depth of behavioral audiences that work at scale. If we know to make the best behavioral audiences, shouldn’t we also know to make the rogue headless audiences? Rogue headless audiences waste precious advertising money. Although rogue audience activation is not an (easy) possibility, we know what makes one. Here is the writeup on our recent exercise, fondly named as BotLens.

The raw material we have

The datasets that Affinity Answers has can be segregated into two types:

• A dataset containing purchase intent signals.
• A dataset containing consumer behavior data, which gives us a broader view of user activity.

Both datasets share common identifiers like device IDs and hashed email addresses, which makes it possible to connect activity across sources. Additionally, the consumer behavior dataset includes IP addresses, giving us another layer to understand how users (or bots) are interacting with the system.

This combination is powerful because it lets us look at activity from multiple angles — device-level, email-level, and network-level — instead of relying on a single identifier.

Identifying the bots — strategies employed

Detecting bots isn’t about spotting a single anomaly — it’s about recognizing patterns that don’t look human. A single signal can often be misleading, so we relied on a combination of heuristics to identify suspicious behavior.

Here’s how each signal was constructed:

1. Device ↔ Email anomalies

• We computed the distribution of number of emails per device ID.
• Devices beyond the 99th percentile were marked as suspicious.

Similarly:

• We computed the number of devices linked to each email.
• Emails beyond the 99th percentile were flagged as suspicious.

To expand this signal:

• We took all device ids associated with these suspicious emails.
• Applied an experimented cutoff (to avoid overly large fan-outs) before marking them as risky.

Importantly, both of these signals were derived independently from both datasets, which helped reinforce consistency.

2. IP / Network-level anomalies

From the consumer dataset, we analyzed IP subnet diversity:

• For each device and email, we calculated the number of unique IP subnets.
• Again, the 99th percentile was used as a cutoff.

Why subnets instead of raw IPs?

Because bots often rotate IPs within the same infrastructure — subnet-level aggregation gives a more stable and meaningful signal of network diversity.

3. Temporal (burst) behavior

From the purchase intent dataset, we analyzed activity over time:

• Devices showing burst activity (high volume in very short windows) were flagged.
• Devices that were active in only a single month and completely absent otherwise were also considered suspicious.

This helps capture two common bot patterns:

• Short-lived, high-intensity campaigns.
• Disposable identities that appear briefly and vanish.

Each of these signals, in isolation, is not sufficient. But together, they form a multi-dimensional anomaly detection system — covering identity, device, network, and time.

Reducing false positives

One of the biggest challenges in bot detection is not just finding bots — but not flagging real users as bots.

Many legitimate users can sometimes look “suspicious”:

• A user logging in from different networks (home, office, mobile data)
• Shared devices in households or offices
• Power users who generate a lot of activity in a short time

To avoid misclassifying such users, we took a few careful steps:

• Used combinations of signals instead of single thresholds We avoided labeling users based on just one metric. Instead, we required multiple suspicious signals to be present together before flagging an entity.
• Applied reasonable thresholds based on distribution Rather than arbitrary cutoffs, thresholds were chosen by analyzing real data distributions and identifying true outliers.
• Looked for consistency, not just spikes A one-time spike might be normal, but repeated abnormal patterns over time are much more indicative of bot behavior.
• Segmented traffic where necessary Different types of users behave differently. For example, enterprise or shared environments can naturally have higher device or IP diversity. Accounting for this helps reduce noise.

In the end, the goal was not just to catch bots — it was to build a system that is robust, fair, and reliable, minimizing false positives.

Final out comes

After constructing all signals, we moved from heuristics to a more unified approach:

• Each device was assigned a composite score, based on the number and severity of suspicious signals it triggered.
• Signals like device-email anomalies, IP subnet diversity, and burst patterns contributed to this score.

Finally:

• A global cutoff on the total score was applied.
• Devices exceeding this threshold were classified as bots.

Using this approach, ~2% of the total device IDs were flagged as potentially bot-driven.

At first glance, 2% might seem small — but at scale, this represents a meaningful portion of traffic that could otherwise impact campaign performance or lead to misleading insights.

More importantly, this number reflects a balanced approach:

• We were aggressive enough to catch suspicious patterns.
• But cautious enough to avoid over-flagging genuine users.

Since the detection was based on multiple independent signals (device, email, and IP-level patterns), the flagged set carries a higher degree of confidence compared to single-metric approaches.

Thus Affinity Answers brings the best audiences at scale; but also identifies the rouge elements carefully crafted to reduce false positives. We care about precious advertising dollars of brands and agencies. Bring it on bots!