Beating .lower() and .upper() in Python: Performance Secrets

Generally, we rely on built-in lower and upper functions in most programming languages to convert strings into the desired case. I used to do the same—until I came across this code snippet in an open-source repository.

Nginx

I was exploring this open-source repository for a personal project because I wanted to understand how load balancing works and eventually build my own load balancer or proxy — similar to what Nginx provides — using Go.

While doing so, I came across a small code snippet that completely changed my perspective on how tiny optimizations can significantly impact system performance at scale.

#define ngx_tolower(c)      (u_char) ((c >= 'A' && c <= 'Z') ? (c | 0x20) : c)
#define ngx_toupper(c)      (u_char) ((c >= 'a' && c <= 'z') ? (c & ~0x20) : c)

Nginx is written in C, which is already a highly optimized language for low-level programming. Yet, the developers still went further to optimize even small parts of the code.

Why there is a need of writing their own lower and upper functions ?

c | 0x20

What Nginx does here is simple: instead of calling a lower() function, it uses a bitwise OR operation. Let’s break it down.

0x20 — the 0x prefix tells the compiler that the number is in hexadecimal. Converting it, 2 * 16 + 0 * 1 = 32, so the value is 32 in decimal.

Now look at ASCII values: 'A' = 65 and 'a' = 97 — the difference is exactly 32.

So OR-ing a capital letter with 0x20 flips the correct bit and converts it to lowercase. It makes perfect sense.

Evaluating performance

To compare their performance, I created two code snippets: one using Python’s built-in lower() function and the other using the Nginx-style bitwise approach.

python.py

x=""
for i in range(100000000):
    x+='A'
t=x.lower()

real    0m11.861s
user    0m11.962s
sys     0m0.290s

nginx.py

x=""
for i in range(100000000):
    x+='A'
t=""
for i in t:
    t+=i | 0X20

real    0m6.414s
user    0m6.478s
sys     0m0.058s

From the results, it’s obvious that the Nginx-style logic is faster. The reason Nginx contributors wrote it this way is because the server processes massive amounts of incoming request data, which must be parsed and evaluated quickly. While this optimization may seem insignificant for small strings, at large scale it makes a meaningful performance difference.

Conclusion

Exploring well-known open-source projects can feel overwhelming at first, but the way their code is written for massive scale teaches you how to write cleaner, more efficient code.