Who Won The Head to Head Test Of The Top 10 Bot Mitigation Companies ?

Photo by Timo Müller on Unsplash

TLDR: I put the top 10 bot mitigation softwares to the test. Using a bot with the same VPN IP address, I loaded each bot mitigation company’s website 100 times. If I could load the page, it failed my test. If it stopped me from loading the page it succeeded. Spoiler alert: All 10 bot mitigation softwares succeeded less than 50% of the time.

While it may seem like an overly simple test, that is intentional. Why are bot mitigation softwares waiting until bots are wrecking havoc to block them? Why not block them on page load? Its because they can’t identify a bot 100% of the time. So they fallback to captchas to confirm.

To be clear, I just loaded each page and didn’t take any further action. I also put in a delay of 10 seconds between each of my requests. I wanted to be a respectful Internet citizen. This work was solely for research purposes.

My Simple Yet Effective Test

Using a bot with the same VPN IP address, I loaded each bot mitigation company’s website 100 times. If I could load the page, it failed my test. If it stopped me from loading the page it succeeded.

I sorted all 10 into 3 categories

A — stopped both head and headless bot visits

No one ended up in this category.

B — stopped headless bot attacks but not non-headless visits

• Datadome stopped 50% of visits
• Radware stopped 50% of visits
• Akamai stopped 50% of visits

C — didn’t stopped headless bot attacks or non-headless visits

• Imperva stopped 0% of visits
• Cloudflare stopped 0% of visits
• HumanPresence stopped 0% of visits
• PerimeterX/HUMAN stopped 0% of visits
• Arkose Labs stopped 0% of visits
• Cequence Security stopped 0% of visits
• F5 stopped 0% of visits

No wonder “85% report their bot mitigation solution became ineffective within a year after initial deployment.”

Why wait to block bots when you can just block them on page load. Call me oversimplistic, but I’d rather lock my front door then have to fight the thief in my front hallway. It seems like a simple idea yet it is one none of the bot mitigation companies are implementing it effectively.

My Befuddling Conclusion

I kept waiting for one of the bot mitigation softwares to surprise me. Datadome came the closest, but had trouble with non-headless bots. One could argue the bot protection on their own sites isn’t as good as the bot protection on their customers’ sites (the cobbler’s shoe problem).

I don’t really blame the companies. I am truly not looking to compete against them. I don’t do anti-DDOS. I am one man. A researcher. I just keep wondering why their bot protections are foiled by simple tests.

One rebuttal I am sure I will hear is a page load isn’t a true test. Once a bot takes action on a page, then the bot mitigation companies would spring into action. I have tests coming on that front and will keep you posted. I am not optimistic though because bot human emulation has gotten so good.