Skip to content
System Design Lab
Microservice Patterns

Timeout

Put an upper bound on how long you wait for any remote call, so one hanging dependency cannot quietly hold every thread, connection and user request hostage.

Beginner 5 min read· Updated 2026-11-18

What you will learn

  • Explain why a call without a timeout is a resource leak waiting to happen.
  • See how a hung dependency exhausts a worker pool and breaks unrelated features.
  • Choose connect, read and overall timeouts from latency percentiles.
  • Configure HTTP client timeouts in Spring and propagate a deadline across services.

Overview

Imagine phoning a shop and nobody picks up — but the line never disconnects either. If you refuse to hang up, you can never make another call. A timeout is simply deciding in advance: "if nobody answers in 2 seconds, I hang up and do something else."

A timeout limits how long a caller waits for a response. When the limit passes, the caller abandons the call, frees whatever it was holding, and treats the call as failed. It is the most basic resilience mechanism — and the one most often missing, because many HTTP clients and drivers default to waiting forever.

The problem: a hang is worse than a crash

A crashed service refuses connections, so callers fail immediately. A hung service — stuck in a deadlock, a long GC pause or an exhausted connection pool — accepts the connection and then says nothing. Each caller thread waits. In a thread-per-request server, those threads are a fixed pool: once they are all waiting, the server cannot serve anything else, including requests that have nothing to do with the hung dependency.

BeforeNo timeout
Shoppersbrowse + checkoutShoppersShop API6 worker threadsShop APIInventorycheckout onlyInventory
Every checkout that reaches the hung Inventory pins a Shop API worker indefinitely.
AfterTimeout per call
Shoppersbrowse + checkoutShoppersShop API6 worker threadsShop APITimeoutper callTimeoutInventorycheckout onlyInventory
Workers are released after 300 ms, whatever Inventory does.

Run it: one hang, two outcomes

Half of the traffic is browsing, which never calls Inventory. Watch the Shop API workers busy tile and the browse requests in scenario A: they fail even though their code path is perfectly healthy.

Loading simulation…

Choosing the value

  • Start from the dependency's measured latency: a common rule is a little above its p99 under normal load.
  • Connect timeout short (100–500 ms inside a data centre): failing to even connect is a strong signal.
  • Read timeout per call based on what the operation does; a report export and a price lookup deserve different values.
  • The caller's own timeout must be longer than the sum of what it waits on downstream, or work keeps running after nobody wants the answer.

Advanced Better than fixed per-hop timeouts is a deadline that travels with the request: the edge sets "answer by 10:00:02.500", and each service passes the remaining budget downstream (gRPC does this natively; over HTTP use a header). A service that receives a request with 20 ms left and needs 100 ms can refuse immediately instead of doing useless work.

Implementation in Spring

InventoryClientConfig.java
@Configuration
class InventoryClientConfig {

    @Bean
    RestClient inventoryClient(RestClient.Builder builder) {
        HttpClient httpClient = HttpClient.newBuilder()
                .connectTimeout(Duration.ofMillis(200))
                .build();
        JdkClientHttpRequestFactory factory = new JdkClientHttpRequestFactory(httpClient);
        factory.setReadTimeout(Duration.ofMillis(300));

        return builder.baseUrl("http://inventory-service")
                .requestFactory(factory)
                .build();
    }
}
Advanced
Async calls: Resilience4j TimeLimiter
@TimeLimiter(name = "inventory")
@CircuitBreaker(name = "inventory", fallbackMethod = "stockUnknown")
public CompletableFuture<Stock> stock(String sku) {
    return CompletableFuture.supplyAsync(() -> inventoryApi.stock(sku), inventoryExecutor);
}

private CompletableFuture<Stock> stockUnknown(String sku, Throwable ex) {
    return CompletableFuture.completedFuture(Stock.unknown(sku));
}

// application.yml
// resilience4j.timelimiter.instances.inventory.timeout-duration: 300ms
// resilience4j.timelimiter.instances.inventory.cancel-running-future: true

Trade-offs

BenefitsCosts
Resource safetyThreads, connections and memory are always released in bounded time.None — every remote call should have one.
Too shortFails fast when something is wrong.Slow-but-successful calls are cut off, causing false failures and retries.
Too longRarely cuts off legitimate work.Barely better than no timeout: pools still fill up during an incident.
SemanticsSimple to reason about for idempotent reads.For writes, a timeout means "unknown outcome", which the caller must handle.

Production considerations

  • Audit every client: HTTP, database driver, message broker, cache, SMTP. Defaults are often infinite or minutes long.
  • Record timeouts as their own metric, separate from other errors — a jump in timeouts is the signature of a hung dependency.
  • Revisit timeout values when latency SLOs change; a stale timeout set years ago is a common root cause.

Further reading

Related topics