Magic of Caching: From Slow to Lightning Fast

Caching

Caching is a performance optimization technique that stores copies of frequently accessed data in a temporary, high-speed storage layer called a Cache. Caches are much faster than traditional data sources like disks, databases, or remote servers, allowing applications to respond more quickly.

Caching strategies For Read Heavy Systems

Lazy Loading Caching

Application only loads the data into the cache when it is needed. In this strategy, application checks the cache first. If the data is found to the cache (cache hit), it returns the data to application. If the data is missing (cache miss), it fetches data from the database. The fetched data is saved back into the cache so that future read request get a faster response.

Lazy loading cache

Read-Through Caching

Cache sits between the application and the database. When the app requests data, cache checks itself. If missing, it automatically retrieves the data from the database. Data is stored in the cache and returned to application.

Read through cache

Pre-loading Caching

This strategy loads data into the cache before it is requested. When we know which data is frequently accessed, it loads the data before hand and eliminates the cold start latency.

Long TTL (Time-To-Live)

Read-heavy data that doesn’t change often can use long expiration times.

Caching Strategies For Write Heavy Systems

Write-Through Caching

Every write operation goes first to the cache and then to the database immediately. This strategy can be applied when consistency is important and write latency is tolerable.

Write through cache

Write-Behind Caching

Application writes data only to the cache. The cache asynchronously updates the database by batches in the background.

Write behind caching

Selective Caching

Not all data is worth caching, cache only the most frequently read items. Some use cases can be profile data of active users, recently viewed products etc.

Event-Driven Caching

Instead of updating cache on every write, use an event system such as message queue or pub/sub to notify the cache when a specific value should be refreshed.

Use Short TTL (Time-To-Live)

For data that changes frequently, we use very short expiration times so stale values don’t live long.

Cache Eviction Policies

1. LRU (Least Recently Used): Removes items not used for the longest time.
2. MRU (Most Recently Used): Removes the most recently used item.
3. LFU (Least Frequently Used): Removes items with the fewest uses.
4. FIFO (First In, First Out): Removes oldest inserted item.
5. Random (RR): Removes a random item. It is useful in high-memory, high-speed caches.

Caching Pros

1. Cache reduces latency, improving performance and efficiency.
2. It reduces workloads from data sources.
3. It enhances scalability by handling more traffic/active users.
4. Improves overall user experience.

Caching Cons

1. Cache will return stale data if cache invalidation is not implemented.
2. Search time may increase if the cache size becomes very large.
3. RAM is expensive and it’s volatile memory.
4. It adds more system complexity.

Caching real world applications

Web browsers: Store copies of website elements like images, scripts, and HTML files to load previously visited pages faster.

Content Delivery Networks (CDNs): Store website content on servers located around the world, allowing users to download content from a server geographically closer to them, which reduces latency.

Databases: Cache frequently queried data in memory to avoid repeatedly hitting the database, which speeds up application performance and reduces server load.

Domain Name System (DNS): Cache DNS records to speed up the process of translating domain names into IP addresses.

Implementation Examples

1. Lazy Loading Cache (Read-Heavy Systems)

import json

TTL = 300

def get_data(cache_key):
    # Check cache
    cached_data = cache.get(cache_key)
    if cached_data:
        # Cache hit and return data
        return json.loads(cached_data)

    # Cache miss: fetch from database
    db_data = get_data_from_database(item_id)

    if db_data:
        # Store in cache
        cache.setex(cache_key, TTL, json.dumps(db_data))
        return db_data
    else:
        return None

2. Write-Through Caching (Write-Heavy Systems)

import json

TTL = 300

def set_data(item_id, item_value):
    try:
        # 1. Write database first (source of truth)
        db_obj = write_data_to_database(item_id, item_value)
    except Exception as db_error:
        print(f"Database write failed: {db_error}")
        return False

    try:
        # 2. Write to cache
        cache.setex(item_id, TTL, json.dumps(item_value))
    except Exception as cache_error:
        print(f"Cache write failed: {cache_error}")

    return True

Conclusion

Caching is a simple yet powerful way to make applications faster and more scallable. By using cache, systems can respond quickly while reducing load on databases. Choosing the right caching strategy ensures consistency, improves performance, and delivers a smoother user experience.