Microservice Patterns
Backpressure
When a consumer cannot keep up, push the problem back to the producer — bounded queues and 429 Retry-After — instead of buffering in memory until the process dies and takes accepted work with it.
What you will learn
- Explain why unbounded buffers turn overload into data loss.
- Bound queues and signal overload with 429 / Retry-After.
- Make producers slow down, spool or drop deliberately.
- Recognise backpressure in streams, HTTP and thread pools.
Before you start
Overview
A restaurant kitchen that accepts every order during a rush ends up with a pile of tickets it cannot cook, cold food and angry guests. A good host says "there's a 20-minute wait" at the door. Guests wait outside or come back later, and the kitchen keeps working at full speed.
Backpressure is a signal from a slower component to a faster one: *slow down*. Without it, the gap between arrival rate and processing rate has to go somewhere — usually an in-memory queue that grows until latency explodes or the process runs out of memory. With it, queues are bounded, and when they fill the system says so (a full buffer blocks, a stream requests fewer items, an API returns 429 Too Many Requests with Retry-After).
Run it
Devices spike from 150 to 400 uploads/s between t = 8 s and 18 s; the API processes ~200/s. Watch Where the backlog lives: in A it lives in server memory until the out-of-memory crash, in B at the producers. In C producers ignore Retry-After and hammer the API with rejected requests.
Forms of backpressure
| Where | Mechanism |
|---|---|
| HTTP APIs | Bounded work queue; 429 or 503 with Retry-After; clients back off. |
| Thread pools | Bounded queue + rejection policy (CallerRunsPolicy slows the submitter). |
| Reactive streams | Subscriber requests N items (request(n)); publisher never sends more. |
| Kafka / queues | Consumers pull at their own pace; lag grows on the broker's disk, not in memory. |
| TCP | Receive window shrinks; the sender blocks. |
Implementation
private final BlockingQueue<Reading> queue = new ArrayBlockingQueue<>(200);
@PostMapping("/readings")
ResponseEntity<Void> ingest(@RequestBody Reading reading) {
if (!queue.offer(reading)) { // full: refuse instead of growing memory
return ResponseEntity.status(HttpStatus.TOO_MANY_REQUESTS)
.header(HttpHeaders.RETRY_AFTER, "1")
.build();
}
return ResponseEntity.accepted().build();
}HttpResponse<Void> response = http.send(request, BodyHandlers.discarding());
if (response.statusCode() == 429) {
long waitSeconds = response.headers().firstValueAsLong("Retry-After").orElse(1);
spool.keep(reading); // keep it locally (disk), do not drop silently
scheduler.schedule(this::flushSpool, waitSeconds + jitter(), TimeUnit.SECONDS);
}Trade-offs
| Benefits | Costs | |
|---|---|---|
| Stability | The server stays healthy and fast under any load. | Some requests are refused during overload. |
| Data | Nothing accepted is lost; the backlog sits where it is safe. | Producers need somewhere to keep it — or a policy for what to drop. |
| Cooperation | Well-behaved clients smooth the load automatically. | Clients that ignore Retry-After waste capacity; enforce with rate limits. |
Production considerations
- Every queue needs a bound — including thread-pool, HTTP client and executor queues hidden in frameworks.
- Size bounds by acceptable waiting time (bound = throughput × max wait), not by available memory.
- Decide per data type what happens when producers' own buffers fill: block, spool to disk, sample, or drop with a metric.
- Combine with rate limiting per client so one noisy producer cannot use all the capacity.