Skip to content
System Design Lab
Design to Scale

Caching Strategies

Put a cache in front of a struggling database: how cache-aside works, what drives the hit ratio, how to keep data fresh, and how to survive the moment the cache goes cold.

Intermediate 5 min read· Updated 2026-11-18

What you will learn

  • Implement cache-aside and reason about hit ratio, TTL and key popularity.
  • Compare cache-aside, read-through, write-through and write-behind.
  • Choose an invalidation strategy and understand the staleness it allows.
  • Prevent a cache stampede with request coalescing and TTL jitter.

Before you start

Requirements

Product pages need an expensive query (~80 ms). The database has 8 connections, so it can serve about 100 of these per second. Traffic is 120 reads per second and growing, and a small number of products receive most of the views.

A cache is a notepad next to the phone. The first time someone asks for a number you look it up in the directory (slow) and write it on the notepad. Next time you read the notepad (fast). The notepad can be out of date, and it is useless the moment you throw it away.

Evolution

Stage 1 of 3

Every read hits the database

Current architecture
The Product API runs the query for every request.
Limitation
Read capacity equals database capacity: ~100 expensive queries/s.
Bottleneck
The 8-connection pool. Queries queue, latency climbs to seconds and requests fail.
Improvement
Most requests ask for the same few products. Keep recent answers in memory.
Performance impact
~74% success at 120 reads/s, p95 near 600 ms.
Complexity & operational cost
Nothing extra to run.
Trade-off
Always fresh data, but capacity is capped by the most expensive component.
Loading simulation…

Caching patterns

PatternRead pathWrite pathGood for
Cache-asideApp checks cache, loads from DB on miss, fills cache.App writes DB, then deletes the cache key.General purpose; the default.
Read-throughCache library loads from DB on miss.As cache-aside.Same as cache-aside with less app code.
Write-throughAlways hits cache.Write cache and DB synchronously.Read-after-write consistency; slower writes.
Write-behindAlways hits cache.Write cache; flush to DB asynchronously.Write-heavy counters; risk of losing writes.

Keeping the cache fresh

  • TTL — every entry expires. Simple and self-healing; staleness is bounded by the TTL.
  • Delete on write — after updating the database, delete the key so the next read reloads it. Delete rather than overwrite to avoid racing writers putting an old value back.
  • Event-driven invalidation — consume change events (e.g. from an outbox or CDC) to delete keys in other services.
  • Combine them: delete on write for correctness, a TTL as a safety net for missed invalidations.

Implementation with Spring Cache and Redis

ProductService.java
@Service
public class ProductService {

    private final ProductRepository products;

    ProductService(ProductRepository products) {
        this.products = products;
    }

    // sync = true: concurrent misses for the same key in this JVM wait for one load (coalescing).
    @Cacheable(cacheNames = "product-page", key = "#id", sync = true)
    public ProductPage page(String id) {
        return products.loadPage(id);           // the expensive query
    }

    @CacheEvict(cacheNames = "product-page", key = "#command.id()")
    @Transactional
    public void update(UpdateProduct command) {
        products.update(command);               // write DB, then evict the key
    }
}
application.yml
spring:
  cache:
    type: redis
    redis:
      time-to-live: 10s
      cache-null-values: false
  data:
    redis:
      host: redis
      port: 6379

Advanced sync = true coalesces misses inside one instance only; with 20 instances you can still get 20 concurrent loads per key. For very expensive keys add a short distributed lock, serve the stale value while one caller refreshes (stale-while-revalidate), or refresh hot keys proactively before they expire. Randomise TTLs (e.g. 10 s ± 20%) so keys written together do not expire together.

Trade-offs

BenefitsCosts
CapacityReads scale with the cache, not the database.The database must still survive a cold cache.
FreshnessBounded by TTL and invalidation.Readers can see stale data; some data must never be cached.
ComplexityCache-aside is a few lines of code.Invalidation across services is genuinely hard.

Production considerations

  • Monitor hit ratio, evictions and memory. A falling hit ratio is often the first sign of a key-design bug.
  • Never let the cache become the only copy of data unless you accept losing it.
  • Treat a cache outage as a database load test — size the database, or shed load, accordingly.

Further reading

Related topics