API Rate Limiting: I Thought It Was Just Counting Requests

When I first read about API rate limiting, the idea looked pretty simple. Count requests. Set a limit. If someone crosses that limit, return 429 Too Many Requests. Something like:

100 requests per minute

That’s it.

At least, that’s what I thought.

Then I started reading more about how rate limiting actually works once you have multiple servers, Redis, concurrent requests, users behind the same IP, and APIs where one request can be way more expensive than another.

And suddenly, counting requests wasn’t the difficult part anymore.

The difficult part was deciding what exactly you’re counting, where that state lives, and what guarantees your limiter is supposed to provide.

This article is basically my attempt to put everything I learned into one place.

Why do we even need rate limiting?

The first answer is obvious: abuse. You don’t want someone writing this:

while (true) {
  fetch("/api");
}

and taking your API down. But rate limiting is doing more than just stopping obvious abuse. Imagine one user sends 50,000 requests while everyone else sends 20. Even if your servers can technically handle the traffic, one user is consuming a ridiculous amount of shared capacity. Then there’s cost. A request like:

GET /profile

Might just perform a simple database read. But:

POST /generate-video

could trigger GPU compute, object storage, background jobs, and maybe even a paid third-party API. Calling both of them “one request” doesn’t really describe their cost. Rate limiting can also enforce product rules.

If your pricing page says:

Free       → 100 requests/day
Pro        → 10,000 requests/day
Enterprise → custom limits

Something in your backend needs to enforce that. So I stopped thinking of rate limiting as just abuse prevention. It’s really a way to protect capacity, fairness, cost, security, and sometimes the business model itself.

Okay, but how do we actually limit requests?

There are a few common algorithms. I initially assumed one of them would simply be “the best.” That isn’t really how it works. Every algorithm solves a slightly different traffic problem.

1. Fixed Window Counter

This is probably the easiest one to understand.

Let’s say the limit is:

100 requests per minute

You divide time into one-minute windows and maintain a counter. Something like:

rate_limit:user:42:14:31 → 73

Every request increments the counter. If it crosses 100, reject the request. When the next minute starts, a new window begins.

Conceptually:

window = floor(current_time / window_size)

key = client_id + ":" + window

count = increment(key)

if count > limit:
    reject

Very simple. One counter. One expiry. But there’s a weird edge case. Suppose I send:

12:00:59 → 100 requests
12:01:00 → 100 requests

Both windows are technically valid. So the limiter accepts 200 requests in roughly one second.

The policy says 100 requests/minute.But the actual traffic pattern doesn't really look like 100 requests per minute anymore.

This is called the boundary burst problem. Fixed windows are simple, but the reset boundary can create sudden spikes.

2. Sliding Window Log

One way to fix the boundary problem is to stop thinking in fixed windows. Instead, store the timestamp of every request.

Let’s say the current time is 12:01:30.

For a 60-second limit, we ask:

How many requests happened after 12:00:30?

The process looks like this:

1. Delete timestamps older than the window
2. Count the remaining timestamps
3. If count < limit, store the current timestamp
4. Otherwise reject

Redis sorted sets are a pretty natural fit here.

ZREMRANGEBYSCORE key 0 window_start
ZCARD key
ZADD key current_timestamp request_id

This gives a much more accurate sliding window. No awkward minute reset. But now we’re storing one timestamp for every request.

Imagine:

1,000,000 active clients
× 100 timestamps
= 100,000,000 entries

That’s the trade-off: very accurate, but potentially expensive.

3. Sliding Window Counter

This one was more interesting to me.

Instead of storing every request timestamp, we keep two counters:

previous window count
current window count

Then we estimate how much of the previous window still overlaps with the current sliding interval.

For example:

previous window = 80 requests
current window  = 30 requests

Suppose 25% of the previous window overlaps.

Then:

estimated_count = 80 × 0.25 + 30
                = 50

It’s an approximation.

But instead of storing every timestamp, we’re basically storing two numbers.

Cloudflare has written about using this kind of approximate sliding-window approach in its rate-limiting infrastructure.

That made the trade-off click for me. Sometimes you don’t need perfect accuracy. Sometimes an approximation with predictable memory usage is actually the better engineering decision.

4. Token Bucket

The token bucket is probably my favorite algorithm in this whole topic.

Imagine a bucket.

capacity = 10 tokens
refill rate = 2 tokens/second

Every request needs one token.

token exists → request allowed
bucket empty → request rejected

Tokens keep refilling over time.

So if the bucket is full, the user can immediately send 10 requests.

After that, they’re limited by the refill rate.

This means the token bucket separates two things: burst capacity and sustained request rate.

That is useful because real users don’t always generate perfectly smooth traffic.

Someone might open a dashboard and trigger eight API calls almost immediately.

You probably don’t want to block them just because those eight requests arrived within 100 milliseconds.

Stripe has publicly written about using token-bucket rate limiting in its API infrastructure.

The basic state is also pretty small:

tokens
last_refill_timestamp

When a request arrives:

elapsed = now - last_refill
tokens = min(
    capacity,
    tokens + elapsed × refill_rate
)
if tokens >= request_cost:
    tokens -= request_cost
    allow
else:
    reject

Another interesting thing is that a request doesn’t necessarily need to cost one token.

You could do:

GET /profile          → 1 token
POST /search          → 3 tokens
POST /generate-video  → 20 tokens

Now you’re not just counting requests.

You’re roughly modelling operational costs.

5. Leaky Bucket

Token bucket allows bursts.

Leaky bucket tries to smooth them.

Think of requests entering a queue:

incoming requests
        ↓
      queue
        ↓
  fixed drain rate
        ↓
    downstream

Requests can arrive quickly, but the queue processes them at a fixed rate.

If the queue fills up completely, new requests are rejected.

This can be useful when a downstream system needs predictable traffic.

For example:

• background jobs
• batch processing
• third-party APIs
• write-heavy services

The easiest way I remember the difference is:

Token bucket allows controlled bursts. Leaky bucket smooths traffic.

So which algorithm should you use?

There isn’t one answer.

My rough mental model is:

Simple internal API ->Fixed window

Need precise moving limits -> Sliding window log

Large-scale approximate limits -> Sliding window counter

Bursts are acceptable -> Token bucket

Downstream needs smooth traffic -> Leaky bucket

The important thing is to start with the traffic behavior.

Not with Redis. Not with Lua.

Not with whatever architecture diagram looks the most impressive.

The problem starts when you add another server

Let’s say your limiter works perfectly locally.

Then you scale the API.

Each server stores the rate-limit counter in memory.

The limit is 100 requests/minute.

Node A thinks the user has sent 40 requests.

Node B thinks they’ve sent 35.

Node C thinks they’ve sent 30.

None of the servers sees the complete picture.

Even worse, if every server independently allows 100 requests:

Node A → 100
Node B → 100
Node C → 100

Your “100 requests per minute” limit may effectively become 300.

The algorithm didn’t fail. The state model failed.

This was probably the biggest shift in how I understood rate limiting.

Once you have multiple servers, the question becomes:

Who owns the rate-limit state?

Redis as shared rate-limit state

A common solution is to move the state somewhere every API node can access.

Multiple API nodes, one shared source of truth for rate-limit state.

Now all three servers look at the same rate-limit state.

A key might look like:

rl:user:42:POST:/api/messages

What you store depends on the algorithm.

For fixed window: count and expiry.

For token bucket: tokens and last_refill.

For a sliding-window counter: previous_count, current_count, and window_timestamp.

GitHub has publicly documented using a replicated Redis backend with client-side sharding for its API rate limiter.

The lesson isn’t “GitHub uses Redis, so everyone should use Redis.”

The interesting part is that the rate limiter itself eventually becomes infrastructure that needs scaling.

You can build a limiter to protect your API and then accidentally make Redis the bottleneck protecting the API.

Systems are fun like that.

The race condition hiding in simple code

Consider this:

count = GET key
if count < limit:
    INCR key
    allow

Looks fine.

But imagine limit = 100 and current count = 99.

Two requests arrive at almost the same time.

Request A reads 99.

Request B also reads 99.

Both check 99 < 100 and both allow the request.

The issue is that reading, checking, and updating are separate operations.

The decision needs to be atomic.

The atomic check and increment script

Redis can execute Lua scripts atomically relative to other commands.

A simplified fixed-window script could look like this:

local current = redis.call("GET", KEYS[1])
if current and tonumber(current) >= tonumber(ARGV[1]) then
    return {0, tonumber(current)}
end
local new_count = redis.call("INCR", KEYS[1])
if new_count == 1 then
    redis.call("EXPIRE", KEYS[1], ARGV[2])
end
return {1, new_count}

The application provides:

KEYS[1] = rate-limit key
ARGV[1] = request limit
ARGV[2] = window duration

The script checks the limit and updates the counter as one atomic operation.

It returns something like allowed? and current count.

Then the application decides whether to continue processing the request.

The main idea is simple:

The decision and the state update should happen atomically.

Otherwise, concurrency can quietly break your limit.

Where should the rate limiter live?

This is another question where the answer is “it depends.”

At the API gateway

Client
  ↓
Gateway + Rate Limiter
  ↓
Application

This is useful for broad platform level policies. You can reject traffic before it reaches your application servers.

Inside application middleware

Client
  ↓
Application
  ↓
Rate-limit middleware
  ↓
Route

The application understands more context.

It knows user ID, tenant ID, route, and operation.

So you can create policies like:

POST /login      → strict
GET /profile     → relaxed
POST /draw-event → high-frequency burst limit

Inside individual services

Maybe your email service can only send a certain number of emails.

Or an AI service needs to protect expensive GPU jobs.

Those services may need their own limits.

In reality, I think layered rate limiting makes the most sense for larger systems:

Edge        → basic abuse protection
Application → user and route policies
Service     → resource-specific protection

Each layer protects something different.

IP addresses are not user IDs

The easiest rate-limit key is:

rate_limit:<ip>

But IP-based rate limiting gets messy.

Think about college Wi-Fi — hundreds of students could appear behind the same public IP.

The same thing happens with office networks, mobile carrier NAT, public Wi-Fi, and VPNs.

Now one abusive user can get everyone throttled.

The opposite problem also exists: an attacker can rotate IP addresses.

So instead of blindly rate limiting by IP, the key should depend on what you’re protecting.

Possible dimensions include IP address, user ID, API key, tenant ID, organization ID, endpoint, HTTP method, and session.

You can combine them:

rl:user:42:POST:/api/messages
rl:tenant:acme:/generate
rl:ip:203.0.113.10:/login

The question I find more useful is:

Who is consuming the resource I’m trying to protect?

That’s probably what you should limit.

Not every request should cost one request

Imagine:

GET /health
GET /profile
POST /search
POST /generate

A flat policy says 100 requests/minute, But those endpoints may have completely different backend costs.

So maybe the policy should be:

GET /health      → cost 0
GET /profile     → cost 1
POST /search     → cost 3
POST /generate   → cost 20

Token bucket makes this pretty natural.

Instead of one request = one token, you have one operation = N tokens.

This becomes especially interesting for AI APIs where one request might generate a tiny text response while another processes a huge context or runs expensive media generation.

Counting both as exactly one request feels a little meaningless.

429 is not enough

Suppose your API responds with:

429 Too Many Requests

Okay. Now what? Should the client retry in one second? One minute? Tomorrow? A better response gives the client recovery information.

HTTP/1.1 429 Too Many Requests
Retry-After: 30
Content-Type: application/json
{
  "error": "rate_limit_exceeded",
  "message": "Too many requests",
  "retry_after": 30
}

Now the client knows what to do.

There is also ongoing IETF work around standardized HTTP rate-limit metadata.

As of the May 2026 Internet-Draft, the proposed fields are RateLimit and RateLimit-Policy.

It’s still an Internet-Draft, not a finalized HTTP standard, so the exact header names may evolve.

Many APIs still expose vendor-specific or older X-RateLimit-* headers, so clients may encounter both conventions in practice.

For a new API, I’d document one clear contract and follow the evolving HTTP specification rather than exposing duplicate metadata without a compatibility reason.

The broader point matters more:

Rate limiting is part of your API contract.

The client should understand how to recover.

The client can still ruin everything

Even with a good server-side limiter, a badly written client can create problems.

Imagine:

while (true) {
  await callAPI();
}

The API returns 429.

The client retries immediately.

Another 429.

Retry.

Another 429.

Congratulations, we built a tiny self-inflicted DDoS loop.

The usual solution is exponential backoff:

1 second
2 seconds
4 seconds
8 seconds
16 seconds

Then add jitter:

delay = exponential_backoff + random_jitter

Why jitter?

Suppose 10,000 clients get rejected at the same time.

Without jitter:

10,000 clients retry after exactly 8 seconds

You just moved the traffic spike eight seconds into the future.

With jitter, retries spread out:

8.1s
8.7s
9.3s
10.0s
...

A decent client should roughly:

1. Respect Retry-After
2. Use bounded exponential backoff
3. Add jitter
4. Stop after a maximum number of retries
5. Queue work if that makes sense
6. Cache responses where possible

Rate limiting isn’t only a server problem.

The client is part of the system too.

Should failed requests count?

Initially, I thought maybe only successful requests should consume quota.

Then consider a login endpoint:

POST /login → 401
POST /login → 401
POST /login → 401

If failed requests don’t count, someone can keep guessing passwords forever.

So “count only 2xx requests” is not a general solution.

The policy depends on what you’re protecting.

For /login, failed attempts probably matter a lot.

For an expensive generation API, maybe the quota should be consumed once expensive work is accepted.

For an internal API, maybe client errors and server errors need separate budgets.

The better question is:

What behavior consumes or threatens the resource this limit exists to protect?

Count that.

What happens when Redis goes down?

This question is uncomfortable because there isn’t a perfect answer.

If Redis is unavailable, the application can’t check the global rate limit.

You have two obvious options.

Fail open

limiter unavailable → allow request

Your API stays available, but your abuse and quota protection disappears.

Fail closed

limiter unavailable → reject request

Your limits remain strict, but now Redis can effectively take down your API.

The answer probably depends on the endpoint.

GET /public-content might fail open.

POST /send-otp probably fails closed.

POST /expensive-ai-job might fail closed or use a conservative fallback.

What matters is that this should be a deliberate decision not something you discover during an outage.

I almost misunderstood local rate limiting

At first, this architecture sounded smart to me:

Each API server keeps a local token bucket.
Local token exists?
→ allow immediately.
Local bucket empty?
→ check Redis.

Fewer Redis calls. Lower latency. Nice.

Except there’s a problem.

Suppose the global limit is 100 requests and we have three nodes.

Node A → 50 local tokens
Node B → 50 local tokens
Node C → 50 local tokens

The cluster can potentially allow:

50 + 50 + 50 = 150 requests

Our global limit is now fiction.

A local bucket can’t independently represent a strict global quota.

One way to approach this is quota leasing.

            Global Budget Redis
               /   |   \
              /    |    \
        lease 20  20   lease 20
            ↓       ↓       ↓
         Node A   Node B   Node C

Redis owns the global budget.

Each node atomically leases a small number of tokens.

The node can then spend those tokens locally.

local leased token available
        ↓
allow locally

When the local lease runs out, request another lease from Redis.

This reduces the need to coordinate with Redis for every single request.

But it introduces another problem.

What if Node A crashes while holding 15 unused tokens?

Those tokens may temporarily disappear from the available budget.

You now need lease expiration, reclamation, or some acceptable oversubscription strategy.

This is something I keep noticing while learning system design: optimizations usually don’t remove complexity. They move it somewhere else.

You need to observe the limiter too

Imagine users are suddenly complaining about 429 responses.

Is someone abusing the API? Did you deploy the wrong limit? Is Redis slow? Is one endpoint suddenly expensive?

Without metrics, you’re guessing.

At minimum, I’d want to track:

allowed requests
rejected requests
rejection rate by endpoint
rejection rate by tenant/user
Redis latency
rate-limit evaluation latency
script errors
fallback activations

The pattern of rejections matters too.

If three users are constantly getting throttled, maybe the limiter is doing its job.

If thousands of unrelated users suddenly start getting throttled, something is probably wrong.

Structured logs can help:

{
  "event": "rate_limit_decision",
  "key_type": "user",
  "policy": "message_send",
  "allowed": false,
  "limit": 100,
  "retry_after": 12
}

Obviously, be careful about logging raw API keys, IPs, or other sensitive identifiers.

The architecture I’d start with

If I had to design a public API rate limiter today, I’d probably start somewhere around here:

                    Client
                       │
                       ▼
               CDN / Edge Layer
                       │
             basic abuse protection
                       │
                       ▼
                 API Gateway
                       │
                  authentication
                       │
                       ▼
             Rate Limit Middleware
                  │          │
                  │          │
            policy lookup    │
                  │          │
                  ▼          ▼
             Config Store   Redis
                         atomic decision
                              │
                     Lua / server logic
                              │
                    ┌─────────┴─────────┐
                    │                   │
                  ALLOW                DENY
                    │                   │
                    ▼                   ▼
             Route Handler      429 + Retry-After

I’d use a token bucket when bursts are expected and sliding-window counters when the policy needs to represent traffic over a moving interval.

Redis when multiple application nodes need coordinated state.

Atomic server-side logic for the decision and state update.

Authenticated identities like user IDs, API keys, or tenant IDs whenever possible.

Different policies for different operations.

Explicit fail-open or fail-closed behavior.

And enough metrics to know when the limiter is doing something weird.

Would this architecture work for every system?

Definitely not.

But I think it’s a much better starting point than:

INCR user_ip

if count > 100:
    return 429

The part I actually found interesting

I started this topic thinking rate limiting was about counting requests. It isn’t. Or at least, the counter is the easy part.

You start with 100 requests per minute.

Then someone asks:

Across how many servers?

Okay.

Then:

100 requests from whom?

Then:

What if two requests arrive at the same time?

Then:

What if Redis is down?

Then:

What if one request costs 100 times more than another?

Then:

What happens when 10,000 clients retry together?

And suddenly your tiny counter has become a distributed systems problem.

Rate limiting was never just counting requests.

It was about deciding what to count, who to count for, and what guarantees the system should provide.

That’s probably my biggest takeaway from studying rate limiting:

System design gets interesting when you stop asking only “How do I build this?” and start asking “What guarantees is this system actually supposed to provide?”

References and things I read

• Cloudflare Engineering — How we built rate limiting capable of scaling to millions of domains
• Stripe Engineering — Scaling your API with rate limiters
• GitHub Engineering — How we scaled the GitHub API with a sharded, replicated rate limiter in Redis
• Postman — What is API Rate Limiting? Understanding Best Practices
• IETF HTTPAPI Working Group — RateLimit header fields for HTTP
• Anurag Kumbhare — API Rate Limiter — System Design Deep Dive
• Inni Chang — API Rate Limiting: Implementation Strategies and Best Practices
• Yapi Kredi Technology — A Fundamental Go-To Guide for API Rate Limiting

I’m still learning backend and system design, and writing these deep dives is basically how I force myself to understand a topic beyond just memorizing definitions.

If I misunderstood a trade-off or missed an interesting edge case, feel free to point it out to me in comments or on X I’d rather fix my mental model than pretend I know everything.