Why Your Server Suddenly Gets High Traffic — Detect Bots, Track IPs & Secure Your Website

Who’s Hitting Your Server?

There’s one thing almost every DevOps engineer, backend developer, or startup team experiences at least once:

Suddenly, your server starts struggling.

• CPU shoots up.
• Memory usage increases.
• The website becomes slow.
• APIs start timing out.
• Users complain.
• Deployments fail.

And everyone starts asking:

“Where is all this traffic coming from?”

Sometimes it’s real users. Sometimes it’s bots. Sometimes it’s attackers. Sometimes it’s crawlers. And sometimes your own application accidentally attacks itself 😄

In this guide, we’ll learn:

• How to investigate server traffic
• How to detect suspicious IPs
• How to analyze logs
• How to identify bots
• How to monitor live traffic
• How to secure production servers properly

What Does “High Traffic” Actually Mean?

High traffic simply means:

Your server is receiving too many requests.

Every time someone:

• opens your website
• calls an API
• logs in
• downloads data
• loads images

A request is sent to the server. Normally, this is fine.

But when requests become:

• too frequent,
• too large,
• automated,
• or malicious,

The server starts struggling.

What Happens When a Server Gets Overloaded?

Servers have limited resources:

• CPU
• RAM
• Disk
• Network bandwidth

When traffic becomes excessive:

• CPU usage increases
• Memory fills up
• The database slows down
• APIs timeout
• Logs grow rapidly
• Sometimes the server crashes completely (if the scaling procedure is not integrated)

Example: Think of it like a restaurant.20 customers?Easy. But 20,000 customers suddenly?Chaos.

Servers behave the same way.

Real Production Example

One company thought they were under a massive cyber attack. Traffic suddenly increased 20x.

But after investigation, they discovered that a frontend bug was repeatedly calling the same API every second.

Thousands of users opened the page.

Result:

• millions of backend requests
• overloaded APIs
• huge logs
• high CPU usage

So remember:

Not every traffic spike is hacking.

Sometimes bugs create bigger problems than attackers.

Step 1 — Check Server Resource Usage

First understand: what exactly is overloaded?

Connect to the server:

ssh ubuntu@your-server-ip

Check CPU & Memory Usage

Run:

top

or better:

htop

Install htop if missing:

sudo apt install htop -y

What is top or htop?

These tools show:

• CPU usage
• RAM usage
• running processes
• resource-heavy applications

Think of it like:

• Linux Task Manager

Example Output

PID USER   CPU% MEM% COMMAND
2241 root  92.5 34.2 node
1550 mysql 45.1 20.1 mysqld

What does this mean?

• Node.js app consuming 92% CPU
• MySQL uses huge memory
• The server is under a heavy workload

This immediately helps identify which application is causing stress.

Step 2 — Check Active Connections

Now let’s see how many users or bots are connected.

Run:

ss -antp

or:

netstat -antp

What are Active Connections?

Whenever someone accesses your website, a network connection is created.

These commands show:

• Who is connected
• How many connections exist
• connection states
• suspicious traffic patterns

Example Output

ESTAB 0 0 10.0.0.1:443 45.12.xx.xx:52144
ESTAB 0 0 10.0.0.1:443 45.12.xx.xx:52145
ESTAB 0 0 10.0.0.1:443 45.12.xx.xx:52146

If the same IP appears thousands of times:

🚨 suspicious activity.

Could be:

• bot traffic
• scraper
• DDoS attack
• abusive crawler

Step 3 — Understand Nginx Logs

Most production servers use Nginx, and logs are usually stored here:

/var/log/nginx

Inside this folder, you’ll commonly see:

access.log
access.log.1
access.log.2.gz
error.log
error.log.1

What Do These Files Mean?

1. access.log: Current live
2. trafficaccess.log.1: Previous rotated log
3. *.gz: Older compressed logs
4. error.log: Current errors
5. error.log.1: Previous errors

What Are Logs?

Logs are records of activity. Every request gets saved.

Example:

45.12.xx.xx - GET /api/login

This tells:

• which IP visited
• Which API was called
• what happened

Logs are one of the most important debugging tools in production.

Step 4 — Read Yesterday’s Logs

Open yesterday’s access logs:

cat access.log.1

What Does cat Do?

cat prints file contents directly into the terminal.

Good for:

• small files

Bad for:

• huge logs

Because the terminal becomes unreadable instantly 😄

Better Option: Use less

less access.log.1

What is less?

less allows:

• scrolling
• safe reading
• searching inside files

Perfect for huge production logs.

Exit using:

q

Step 5 — Search Inside Logs Using grep

Now the fun part begins.

Run:

grep " 500 " access.log.1

What is grep?

grep searches text patterns inside files.

You can search:

• errors
• APIs
• IPs
• domains
• bots

Extremely useful during incidents.

What is HTTP 500?

500 means: Internal Server Error.

Usually caused by:

• backend crash
• code bug
• database issue
• failed deployment

Example Output

45.xx.xx.xx - - [28/May] "GET /api/chat HTTP/1.1" 500

This means:

• request failed
• Server returned an internal error

If thousands appear: 🚨 backend issue exists.

Search Specific API

grep "/api/chats" access.log.1

Example Output

45.xx.xx.xx "POST /api/chats HTTP/1.1" 200
45.xx.xx.xx "POST /api/chats HTTP/1.1" 500

Now you can:

• debug specific APIs
• Identify failing requests
• analyze traffic patterns

Search Specific Domain

grep "123.xyz.tech" access.log.1

Useful when (like different microservices deployed on seprate domains or etc):

• Multiple domains exist
• using reverse proxies
• debugging staging environments

Search Specific IP

grep "45.xx.xx.xx" access.log.1

Example Output

45.xx.xx.xx - GET /login
45.xx.xx.xx - GET /api/users
45.xx.xx.xx - GET /search

Now you can track:

• What that IP is doing
• which endpoints it hits
• whether it looks malicious

Step 6 — Find Top IPs Hitting the Server

Run:

awk '{print $1}' access.log | sort | uniq -c | sort -nr | head

What Does This Command Do?

This command:

1. extracts IP addresses
2. counts requests
3. sorts the highest traffic first

Example Output

120000 45.xx.xx.xx
85000 103.xx.xx.xx
40000 192.xx.xx.xx

Meaning:

• First IP generated 120k requests

Very suspicious.

Step 7 — Track IP Location

Now let’s identify:

• where traffic comes from.

Run:

whois 45.xx.xx.xx

What is WHOIS?

WHOIS provides:

• country
• ISP
• organization
• hosting provider

Example Output

OrgName: Amazon AWS
Country: US

This means:

• Traffic originated from AWS infrastructure

Bots often use:

• cloud providers
• VPNs
• datacenter servers

Step 8 — Understand Bots

Bots are automated programs. Instead of humans manually visiting websites, bots do it automatically.

Good Bots

Examples:

• Googlebot
• Bingbot

These help, search engines index websites (like Search engine optimization ) .

Bad Bots

Examples:

• scrapers
• spam bots
• brute-force tools
• scanners

Bad bots can:

• overload APIs
• steal data
• spam endpoints
• attempt attacks

Step 9 — Detect Bots Using User Agents

Run:

awk -F\" '{print $6}' access.log | sort | uniq -c | sort -nr | head

What is User-Agent?

User-Agent identifies:

• browser
• application
• bot software

Example Output

50000 Mozilla/5.0
12000 python-requests
8000 curl

What Does This Mean?

Heavy automation traffic often appears here. getting 50K hots from Mozilla/5.0.

Step 10 — Monitor Logs Live

Run:

tail -f access.log

What Does tail -f Do?

It streams logs live.

Feels like:

• watching real-time traffic CCTV

Useful during:

• deployments
• attacks
• outages

Example Output

45.xx.xx.xx GET /api/login
45.xx.xx.xx GET /api/search
103.xx.xx.xx POST /api/chat

You can literally watch traffic live.

Step 11 — Read Error Logs

Open:

less error.log.1

Example Output

upstream timed out
connection refused

These errors indicate:

• backend issues
• API failures
• overloaded services

Filter Only Errors

grep -i "error" error.log.1

The -i flag means:

• case-insensitive search

Matches:

• error
• ERROR
• Error

all together.

Step 12 — Read Compressed Logs

Older logs are compressed as .gz.

Read directly:

zcat access.log.2.gz

Search inside compressed logs:

zgrep "500" access.log.2.gz

Very useful for:

• old incidents
• historical attacks
• previous outages

Step 13 — Count Total Errors

Run:

grep " 500 " access.log.1 | wc -l

Example Output

5421

Meaning:

• 5421 internal server errors occurred

That’s a serious issue.

Step 14 — Find Most-Hit APIs

Run:

awk '{print $7}' access.log.1 | sort | uniq -c | sort -nr | head

Example Output

80000 /api/search
50000 /login
30000 /api/chat

This helps identify:

• spammed APIs
• hot endpoints
• attack targets

Step 15 — Detect Disk Space Problems

Sometimes traffic causes:

• huge logs
• full disks

Then errors appear:

No space left on device

Very dangerous in production.

Check Disk Usage

df -h

Example Output

/dev/root 38G 38G 0 100%

Meaning:

• disk full

Consequences:

• deployments fail
• logs stop writing
• applications crash

Step 16 — Find Large Directories

Run:

sudo du -sh /* 2>/dev/null | sort -hr | head

Example Output

18G /var
10G /docker
5G /logs

Now you know:

• What is consuming storage

Step 17 — Docker Can Fill Disks Fast

Check Docker usage:

docker system df

Example Output

Images: 12GB
Containers: 5GB
Build Cache: 8GB

Over time, Docker accumulates:

• images
• cache
• unused containers

Cleanup Docker

docker system prune -af

⚠ Warning: Removes:

• unused images
• stopped containers
• build cache

Usually frees huge space instantly.

Step 18 — Cleanup Nginx Logs

Check log size:

sudo du -sh /var/log/nginx

Delete old compressed logs:

cd /var/log/nginx
sudo rm -f *.gz

Useful when:

• logs explode due to bot traffic

Step 19 — Cleanup Journal Logs

Run:

sudo journalctl --vacuum-time=7d

This removes old system logs.

Step 20 — Block Suspicious IPs

Using a firewall:

sudo ufw deny from 45.xx.xx.xx

What is a Firewall?

A firewall controls:

• Who can access the server

Think of it like:

• security guard for your infrastructure

Step 21 — Add Rate Limiting

Rate limiting prevents abuse.

Example Nginx config:

limit_req_zone $binary_remote_addr zone=api:10m rate=10r/s;

This limits:

• 10 requests per second per IP

Very useful against:

• bots
• scrapers
• brute-force attacks

Step 22 — Use Fail2Ban

Install:

sudo apt install fail2ban -y

What is Fail2Ban?

Fail2Ban:

• watches logs
• detects suspicious behavior
• automatically blocks attackers

Useful for:

• SSH attacks
• login abuse
• spam requests

Step 23 — Use Cloudflare

Cloudflare sits between users and your server

It provides:

• DDoS protection
• firewall
• bot filtering
• caching
• CDN

Many attacks never even reach your server after enabling it.

Step 24 — Protect SSH Access

SSH allows remote server access.

Example:

ssh ubuntu@server-ip

Attackers constantly scan the internet for SSH servers.

Best practices:

• Use SSH keys
• disable passwords
• restrict IP access

Step 25 — Understand DDoS Attacks

DDoS means: Distributed Denial of Service.

Thousands of machines flood one target server.

Goal:

• overload resources
• crash services
• make website unavailable

Common DDoS Signs

SignalMeaningHuge traffic spikeattackThousands of IPsbotnetSame API spammedLayer 7 attackMassive bandwidth usage network flood

Step 26 — Add Monitoring

Never wait for users to report outages.

Use:

• Grafana
• Prometheus
• Datadog
• CloudWatch

Monitor:

• CPU
• memory
• requests
• errors
• response time

Monitoring helps detect issues early.

Step 27 — Invisible Character Problem

Sometimes commands fail because hidden Unicode characters get copied accidentally.

Example:

ƒprintenv

Linux interprets it as:

• different command

Common causes:

• WhatsApp copy-paste
• formatted documents
• browser formatting

Best practice:

• manually type commands
• or use plain text editors

Final Thoughts

Production debugging is not magic.Most problems become manageable once you learn:

• log analysis,
• traffic investigation,
• disk debugging,
• bot detection,
• and Linux troubleshooting.

The more comfortable you become with:

• grep
• tail
• awk
• logs
• monitoring
• traffic analysis

The more confident you become during real production incidents.

And honestly?

That’s when you truly start thinking like a DevOps engineer.

About the author: Lavanya Sharma is a DevSecOps and Cloud enthusiast with a pleasing, youthful personality and a zest for learning and innovation. She has a strong foundation in backend development with Python and is passionate about automating secure, scalable infrastructure, with a keen interest in emerging tools and a willingness to take on challenges.