How I Cut Our Database Load by Over Half — Without Adding Any New Infrastructure

Summary

A serverless architecture built on AWS Lambda scales compute automatically, up to an account-wide concurrency limit — 1,000 concurrent executions by default. When that limit is reached, new invocations are rejected across every service sharing the account, not just the one causing the load. I ran into this problem with a Lambda-based market data service that turned out to be the single largest contributor to repeated account-wide concurrency incidents. This is what I built to fix it, and what it actually took to get there: less “add a cache,” more figuring out how stale each piece of data was allowed to get before I could safely cache it.

The problem

The account had hit its default Lambda concurrency limit of 1,000 several times within a short period, each time producing 5xx errors across all services on the account — including services with no relation to the root cause. An audit across services was run to identify the largest contributors to concurrency usage.

The market data service I own — handling quotes, charts, sector classifications, leaderboards, and reference content — came back as the single largest individual consumer of account concurrency: a peak of 370 concurrent executions during traffic bursts, or 37% of the account limit, on approximately 12.6 million invocations per week. Its typical, non-burst peak was far lower — burst concurrency and steady-state concurrency are different numbers, and it’s the burst that exhausts the account limit. The service had no application-level caching at any layer. Its primary DynamoDB table, holding roughly 75,000 items, was consuming approximately 15.7 million read capacity units per day — around 209 reads per item per day. At a higher traffic multiple, this service alone was projected to consume 700 or more units of concurrency, putting the account limit within reach on a regular basis.

Why an in-memory cache instead of a managed cache

I considered two approaches: a managed, cross-instance cache (e.g., ElastiCache or DAX), and an in-process cache scoped to a single Lambda execution environment.

A managed cache gives consistency across all warm instances but brings VPC networking, another piece of infrastructure to operate, and ongoing cost. An in-process cache has none of that, but it’s scoped to a single execution environment — cache state isn’t shared across concurrent instances, and a cold start begins with an empty cache.

Lambda execution environments persist in memory between invocations while warm, and this service’s traffic was frequent enough that environments stayed warm most of the time. Given that reuse pattern, an in-process TTL cache captures most of the benefit of a distributed cache without the added infrastructure. I kept the managed-cache option as an upgrade path in case cross-instance consistency ever became a real requirement — it wasn’t needed here.

The actual work: figuring out how stale each data type was allowed to be

The easy part was writing a TTL cache. The part that took real thought was deciding, for each type of data this service serves, exactly how long a stale answer was acceptable — because getting that wrong in either direction is a real cost. Too short, and the cache barely helps. Too long, and users see wrong prices.

A single TTL applied uniformly would have done one of those two things. Instead, I went through each data type and matched the cache behavior to what actually determines its freshness in the real world — not an arbitrary number, but the thing that governs how often the underlying data can even change:

What actually decided each of these:

• Market quotes / prices — a quote can’t be fresher than the last upstream sync, so the TTL tracks the upstream data-refresh cron schedule instead of an arbitrary number.
• ETF provider metadata / Sector data / Reference lookups — change rarely and predictably, so a long TTL costs nothing in accuracy.
• Chart data — this is the clearest example of why per-type reasoning mattered. A fixed 30-minute TTL would have served stale intraday charts for hours after market close. A fixed 4-hour TTL would have missed real intraday movement during trading hours. The cache had to know what the market was doing, not just what time it was.
• Education content — static, but a “not found yet” result shouldn’t be cached as if it were permanent.
• Leaderboards— rankings shift continuously, so a short TTL was the acceptable tradeoff between freshness and load.

A few implementation details mattered as much as the TTL choices themselves:

• Partial-hit caching: For bulk requests (e.g., quotes for a list of tickers), a partial cache hit returns the cached subset and fetches only the missing entries from the database, rather than treating any miss within the batch as a full cache miss.
• Cache placement: The cache layer sits between the request handler and the database connector, rather than inside the connector. This keeps the cache a separate, swappable concern from data access logic.
• Invalidation: Cache invalidation is handled via a single environment variable. Incrementing the variable value causes all warm execution environments to treat existing cache entries as invalid on next access, without requiring a redeploy or a shared invalidation mechanism across instances.
• Caller audit: Before rollout, I traced every caller of the endpoints I was about to cache, including internal service-to-service calls that didn’t go through the primary request handlers. A few of those calls bypassed the planned cache layer entirely and needed separate handling.

Results

I captured metrics immediately before caching was introduced, one week after deployment, and five weeks after deployment, to check whether the reduction in database load would actually persist under continued traffic growth, rather than just looking good in a day-one graph.

Metrics captured immediately before deploy, one week after, and five weeks after. Invocation volume is unchanged — the read drop is from caching, not traffic.

The read-usage trend over the full window makes the effect visible without needing the table. Usage held in a consistently high, jagged band for months, then dropped to roughly half that band immediately at deployment — and stayed there for the five weeks that followed:

DynamoDB consumed read capacity, average units/second. The step-down starting in late May lines up exactly with the caching deploy and holds through the rest of the window.

The obvious follow-up question is whether that drop just reflects less traffic. It doesn’t — invocation volume over the same window shows no corresponding dip:

Lambda invocations over the same period. Volume and its weekly pattern are unchanged across the deploy — the read-usage drop is not explained by reduced traffic.

Database read load remained at roughly half of pre-caching levels five weeks after deployment, despite continued invocation volume. Latency did not regress at any measured percentile.

The peak concurrency figures in the table above are steady-state values, measured over full-week windows with no burst incident occurring during them — they aren’t directly comparable to the 370-concurrent burst figure cited earlier, which was captured during the specific traffic spikes that triggered the original audit. No burst of that magnitude recurred in the weeks following deployment, though burst events are infrequent enough that this isn’t a controlled before/after comparison on that specific metric.

The bug that showed up somewhere else entirely

A few days after deploying, I got pulled into a crash report in a completely different service — one that consumes this service’s data over an internal API, not one I’d touched. It took longer than it should have to connect the two: one of the new caches was returning its cached objects by reference. The downstream service was mutating a field on the object it received, in place, and because it held a reference rather than a copy, that mutation landed on the shared cache entry itself — corrupting it for every other caller until the entry expired.

The lesson wasn’t really about that one cache. It was that a caching decision made inside one service has a blast radius that extends past that service’s own code, into whatever else consumes its output — and none of my own tests would have caught it, because from this service’s point of view, nothing was broken.

The fix: a cache accessor returns a copy of the cached value, at minimum one level into any nested structure a caller might mutate. Not necessarily a full deep copy — the right copy depth depends on which fields callers actually touch, which isn’t obvious without tracing every caller, including the ones outside your own service boundary.

Conclusion

An in-process TTL cache cut database read load by roughly half without adding any new infrastructure, and it’s held for over a month under real traffic. But the caching mechanism was the easy 20%. The work that made it safe to ship was classifying how stale each data type was allowed to get, and the work that made it hold up in production was tracing every consumer of that data — including the one outside this service that I only found out about after it broke.

This doesn’t generalise to every service. It depends on invocation frequency, execution environment reuse, tolerance for per-instance cache state, and how well-understood your downstream consumers actually are. A service with lower invocation frequency, a stricter cross-instance consistency requirement, or callers you can’t fully enumerate may not get the same result — and would need to weigh a managed caching layer instead.