Design to Scale
Caching Strategies
Put a cache in front of a struggling database: how cache-aside works, what drives the hit ratio, how to keep data fresh, and how to survive the moment the cache goes cold.
What you will learn
- Implement cache-aside and reason about hit ratio, TTL and key popularity.
- Compare cache-aside, read-through, write-through and write-behind.
- Choose an invalidation strategy and understand the staleness it allows.
- Prevent a cache stampede with request coalescing and TTL jitter.
Before you start
Requirements
Product pages need an expensive query (~80 ms). The database has 8 connections, so it can serve about 100 of these per second. Traffic is 120 reads per second and growing, and a small number of products receive most of the views.
A cache is a notepad next to the phone. The first time someone asks for a number you look it up in the directory (slow) and write it on the notepad. Next time you read the notepad (fast). The notepad can be out of date, and it is useless the moment you throw it away.
Evolution
Every read hits the database
- Current architecture
- The Product API runs the query for every request.
- Limitation
- Read capacity equals database capacity: ~100 expensive queries/s.
- Bottleneck
- The 8-connection pool. Queries queue, latency climbs to seconds and requests fail.
- Improvement
- Most requests ask for the same few products. Keep recent answers in memory.
- Performance impact
- ~74% success at 120 reads/s, p95 near 600 ms.
- Complexity & operational cost
- Nothing extra to run.
- Trade-off
- Always fresh data, but capacity is capped by the most expensive component.
Caching patterns
| Pattern | Read path | Write path | Good for |
|---|---|---|---|
| Cache-aside | App checks cache, loads from DB on miss, fills cache. | App writes DB, then deletes the cache key. | General purpose; the default. |
| Read-through | Cache library loads from DB on miss. | As cache-aside. | Same as cache-aside with less app code. |
| Write-through | Always hits cache. | Write cache and DB synchronously. | Read-after-write consistency; slower writes. |
| Write-behind | Always hits cache. | Write cache; flush to DB asynchronously. | Write-heavy counters; risk of losing writes. |
Keeping the cache fresh
- TTL — every entry expires. Simple and self-healing; staleness is bounded by the TTL.
- Delete on write — after updating the database, delete the key so the next read reloads it. Delete rather than overwrite to avoid racing writers putting an old value back.
- Event-driven invalidation — consume change events (e.g. from an outbox or CDC) to delete keys in other services.
- Combine them: delete on write for correctness, a TTL as a safety net for missed invalidations.
Implementation with Spring Cache and Redis
@Service
public class ProductService {
private final ProductRepository products;
ProductService(ProductRepository products) {
this.products = products;
}
// sync = true: concurrent misses for the same key in this JVM wait for one load (coalescing).
@Cacheable(cacheNames = "product-page", key = "#id", sync = true)
public ProductPage page(String id) {
return products.loadPage(id); // the expensive query
}
@CacheEvict(cacheNames = "product-page", key = "#command.id()")
@Transactional
public void update(UpdateProduct command) {
products.update(command); // write DB, then evict the key
}
}spring:
cache:
type: redis
redis:
time-to-live: 10s
cache-null-values: false
data:
redis:
host: redis
port: 6379Advanced sync = true coalesces misses inside one instance only; with 20 instances you can still get 20 concurrent loads per key. For very expensive keys add a short distributed lock, serve the stale value while one caller refreshes (stale-while-revalidate), or refresh hot keys proactively before they expire. Randomise TTLs (e.g. 10 s ± 20%) so keys written together do not expire together.
Trade-offs
| Benefits | Costs | |
|---|---|---|
| Capacity | Reads scale with the cache, not the database. | The database must still survive a cold cache. |
| Freshness | Bounded by TTL and invalidation. | Readers can see stale data; some data must never be cached. |
| Complexity | Cache-aside is a few lines of code. | Invalidation across services is genuinely hard. |
Production considerations
- Monitor hit ratio, evictions and memory. A falling hit ratio is often the first sign of a key-design bug.
- Never let the cache become the only copy of data unless you accept losing it.
- Treat a cache outage as a database load test — size the database, or shed load, accordingly.