Microservice Patterns
Timeout
Put an upper bound on how long you wait for any remote call, so one hanging dependency cannot quietly hold every thread, connection and user request hostage.
What you will learn
- Explain why a call without a timeout is a resource leak waiting to happen.
- See how a hung dependency exhausts a worker pool and breaks unrelated features.
- Choose connect, read and overall timeouts from latency percentiles.
- Configure HTTP client timeouts in Spring and propagate a deadline across services.
Overview
Imagine phoning a shop and nobody picks up — but the line never disconnects either. If you refuse to hang up, you can never make another call. A timeout is simply deciding in advance: "if nobody answers in 2 seconds, I hang up and do something else."
A timeout limits how long a caller waits for a response. When the limit passes, the caller abandons the call, frees whatever it was holding, and treats the call as failed. It is the most basic resilience mechanism — and the one most often missing, because many HTTP clients and drivers default to waiting forever.
The problem: a hang is worse than a crash
A crashed service refuses connections, so callers fail immediately. A hung service — stuck in a deadlock, a long GC pause or an exhausted connection pool — accepts the connection and then says nothing. Each caller thread waits. In a thread-per-request server, those threads are a fixed pool: once they are all waiting, the server cannot serve anything else, including requests that have nothing to do with the hung dependency.
Run it: one hang, two outcomes
Half of the traffic is browsing, which never calls Inventory. Watch the Shop API workers busy tile and the browse requests in scenario A: they fail even though their code path is perfectly healthy.
Choosing the value
- Start from the dependency's measured latency: a common rule is a little above its p99 under normal load.
- Connect timeout short (100–500 ms inside a data centre): failing to even connect is a strong signal.
- Read timeout per call based on what the operation does; a report export and a price lookup deserve different values.
- The caller's own timeout must be longer than the sum of what it waits on downstream, or work keeps running after nobody wants the answer.
Advanced Better than fixed per-hop timeouts is a deadline that travels with the request: the edge sets "answer by 10:00:02.500", and each service passes the remaining budget downstream (gRPC does this natively; over HTTP use a header). A service that receives a request with 20 ms left and needs 100 ms can refuse immediately instead of doing useless work.
Implementation in Spring
@Configuration
class InventoryClientConfig {
@Bean
RestClient inventoryClient(RestClient.Builder builder) {
HttpClient httpClient = HttpClient.newBuilder()
.connectTimeout(Duration.ofMillis(200))
.build();
JdkClientHttpRequestFactory factory = new JdkClientHttpRequestFactory(httpClient);
factory.setReadTimeout(Duration.ofMillis(300));
return builder.baseUrl("http://inventory-service")
.requestFactory(factory)
.build();
}
}@TimeLimiter(name = "inventory")
@CircuitBreaker(name = "inventory", fallbackMethod = "stockUnknown")
public CompletableFuture<Stock> stock(String sku) {
return CompletableFuture.supplyAsync(() -> inventoryApi.stock(sku), inventoryExecutor);
}
private CompletableFuture<Stock> stockUnknown(String sku, Throwable ex) {
return CompletableFuture.completedFuture(Stock.unknown(sku));
}
// application.yml
// resilience4j.timelimiter.instances.inventory.timeout-duration: 300ms
// resilience4j.timelimiter.instances.inventory.cancel-running-future: trueTrade-offs
| Benefits | Costs | |
|---|---|---|
| Resource safety | Threads, connections and memory are always released in bounded time. | None — every remote call should have one. |
| Too short | Fails fast when something is wrong. | Slow-but-successful calls are cut off, causing false failures and retries. |
| Too long | Rarely cuts off legitimate work. | Barely better than no timeout: pools still fill up during an incident. |
| Semantics | Simple to reason about for idempotent reads. | For writes, a timeout means "unknown outcome", which the caller must handle. |
Production considerations
- Audit every client: HTTP, database driver, message broker, cache, SMTP. Defaults are often infinite or minutes long.
- Record timeouts as their own metric, separate from other errors — a jump in timeouts is the signature of a hung dependency.
- Revisit timeout values when latency SLOs change; a stale timeout set years ago is a common root cause.