How I Reduced Database Queries by 93% Using a Bloom Filter and Spring Boot

Every time I create a new account on platforms like Google, GitHub, or X, I’m amazed at how quickly they tell me whether a username is available.

Type a username. A fraction of a second later:

✓ Available or ✗ Already taken!

That speed got me thinking. Even though modern databases are incredibly fast, performing a database lookup for every username check at scale still feels expensive. Millions of users checking usernames every day would generate a massive amount of database traffic.

Of course, companies like Google use sophisticated caching layers, distributed systems, and countless optimizations behind the scenes. But while researching how large-scale systems reduce unnecessary database work, I came across a fascinating probabilistic data structure: The Bloom Filter. I was immediately hooked. To understand it better, I decided to build my own username availability service using Spring Boot, MySQL, and Google’s Guava Bloom Filter implementation.

The result was a system that was able to answer 93.3% of username availability requests without ever touching the database.

The Problem

Imagine a traditional username availability check.

User enters username
        ↓
Query Database
        ↓
Username exists?
        ↓
Return response

Every request requires a database query. While a single query might only take a few milliseconds, the cost becomes significant when thousands or millions of users are checking usernames every day. What if we could eliminate most of those queries entirely? That’s exactly where Bloom Filters shine.

What Is a Bloom Filter?

A Bloom Filter is a space-efficient probabilistic data structure used to test whether an element is a member of a set.

The key properties are:

No False Negatives -> If the Bloom Filter says: “Username does not exist” then the username is definitely not present. This guarantee is extremely valuable.

Possible False Positives ->If the Bloom Filter says: “Username might exist” the username may or may not actually be present. In this case, we verify by querying the database.

Because false positives are possible but false negatives are not, Bloom Filters work perfectly as a first-pass filter before hitting a database.

System Architecture

I built the project using a standard Spring Boot MVC architecture.

React Frontend
       ↓
Spring Boot Controller
       ↓
Bloom Filter Service
       ↓
MySQL Database

The request flow looks like this:

User enters username
        ↓
Check Bloom Filter
        ↓
 ┌───────────────────────┐
 │ Definitely Not Present│
 └───────────────────────┘
        ↓
 Return Available
 (No DB Query)

OR

 ┌───────────────────────┐
 │ Might Be Present      │
 └───────────────────────┘
        ↓
 Query MySQL
        ↓
 Return Result

This simple optimization drastically reduces unnecessary database traffic.

Loading 100,000 Usernames

To make the simulation more realistic, I pre-seeded the database with approximately 100,000 usernames.

When the application starts:

1. Usernames are loaded from MySQL.
2. Each username is inserted into the Bloom Filter.
3. The filter becomes an in-memory representation of the username set.

After initialization, most availability checks can be answered instantly without requiring a database lookup.

Designing the Bloom Filter

Instead of implementing the data structure from scratch, I used Google’s Guava library.

BloomFilter<String> bloomFilter =
    BloomFilter.create(
        Funnels.stringFunnel(StandardCharsets.UTF_8),
        100_000,
        0.01
    );

The two parameters that matter are:

• Expected insertions = 100,000
• False positive probability = 1%

From these values, Guava automatically calculates:

• Bit array size
• Number of hash functions

The formulas are:

Where:

• m = number of bits
• n = expected insertions
• p = false positive probability

And:

Where:

• k = number of hash functions

Internally, Guava uses MurmurHash3 and a technique called double hashing to efficiently simulate multiple hash functions.

This provides excellent performance while keeping memory consumption extremely low.

Why Not Just Use a HashSet?

This was one of the first questions I asked myself. A HashSet provides O(1) lookups and is incredibly fast. However, it stores every username explicitly in memory. As the dataset grows, memory usage becomes substantial. A Bloom Filter stores only bits, not the actual strings. For large datasets, the memory savings become dramatic.

| Structure    | Stores Data? | Memory Usage |
| ------------ | ------------ | ------------ |
| HashSet      | Yes          | High         |
| Bloom Filter | No           | Very Low     |

The tradeoff is simple:

• HashSet → exact answers, more memory
• Bloom Filter → probabilistic answers, much less memory

For large-scale systems, that tradeoff is often worthwhile.

Engineering Challenges I Encountered

The most interesting part of the project wasn’t actually the Bloom Filter. It was all the engineering problems surrounding it.

Problem #1: Application Startup Ordering

Initially, I used a data.sql script to seed the database. The plan seemed straightforward:

1. Start Spring Boot
2. Load data.sql
3. Populate Bloom Filter

Unfortunately, that’s not what happened. The Bloom Filter initialized before the usernames were inserted into MySQL. As a result, the filter started empty. Every username appeared available.

Solution

I replaced the SQL-based initialization with a custom ApplicationRunner. This ensured:

Database Ready
      ↓
Usernames Loaded
      ↓
Bloom Filter Warmed
      ↓
Application Accepts Requests

After making this change, the filter accurately reflected the database contents from startup.

Problem #2: The Race Condition

This was my favorite bug. Imagine two users trying to register the same username simultaneously.

User A checks username
User B checks username

Both see AVAILABLE

Then:

User A registers
User B registers

The first request succeeds. The second request attempts to insert a duplicate username. Initially, this caused a server error because MySQL’s unique constraint was violated.

Solution

Instead of trusting the availability check, I treated the database as the final source of truth.

I wrapped the save operation in a try-catch block and handled:

DataIntegrityViolationException

Now the second user receives a clean: “Username already taken” response instead of a 500 Internal Server Error.

This pattern is common in production systems and demonstrates why database constraints should always be treated as the ultimate safeguard.

Problem #3: Thread-Safe Statistics

I wanted to track metrics such as:

• Total checks
• Database queries
• Bloom Filter short-circuits

My first implementation used plain long variables. That worked perfectly during local testing. Until I remembered that Spring Boot serves requests using multiple threads. Concurrent increments on a regular long are not thread-safe. Some updates would be lost.

Solution

I switched to:

AtomicLong

which uses Compare-And-Swap (CAS) operations internally. This allowed me to safely collect statistics under concurrent load without introducing locks.

Results

After running the application and collecting metrics, the numbers were encouraging.

| Metric                   | Result |
| ------------------------ | ------ |
| Username checks          | 100%   |
| Answered by Bloom Filter | 93.3%  |
| Required DB lookup       | 6.7%   |

That means nearly all availability requests were resolved without touching MySQL. From a scalability perspective, that’s a huge win.

Every avoided query means:

• Less database load
• Lower latency
• Better scalability
• Reduced infrastructure costs

What I Learned

Before building this project, I thought of databases as the default answer for lookup problems. This project changed that perspective. The biggest lesson wasn’t how Bloom Filters work. It was learning that system design often involves preventing work rather than making work faster. A database query that never happens is always faster than an optimized database query.

Along the way, I gained hands-on experience with:

• Probabilistic data structures
• Spring Boot application lifecycle
• Concurrency and thread safety
• Race conditions
• Database constraints
• Cache-friendly system design

Most importantly, I got a glimpse into the kind of engineering tradeoffs that appear in large-scale backend systems every day.

Final Thoughts

Bloom Filters won’t replace databases. They won’t magically solve every scalability problem. But when used correctly, they can dramatically reduce unnecessary work and help systems scale far beyond what would be possible with direct database lookups alone.

What started as curiosity about how large platforms check username availability eventually became one of the most interesting backend projects I’ve built.

And it reinforced a simple idea that appears repeatedly in software engineering: “The fastest database query is the one you never have to execute.”

If you’re interested in backend engineering, distributed systems, or scalable architectures, implementing a Bloom Filter is a fantastic weekend project. You’ll learn far more than just a new data structure — you’ll learn how real systems think about performance. Here’s the live link (I know the UI isn’t that great lol)— https://bloom-filter-ui.vercel.app/