Design to Scale
Stateless Application Design
Keep no user state in the memory of an application instance, so any instance can serve any request — the precondition for load balancing, autoscaling and painless deploys.
What you will learn
- Explain what makes an instance stateful and why it hurts scaling.
- Compare in-memory sessions, sticky sessions and a shared session store.
- Move sessions, uploads and caches out of the instance.
- Recognise hidden state that breaks horizontal scaling.
Before you start
Overview
If every bank clerk kept your file in their own desk drawer, you would have to queue for the same clerk every time — and if they were off sick, nobody could help you. Keep the files in a shared cabinet and any free clerk can serve you.
An instance is stateful when something a user needs later lives only in its memory or on its local disk: login sessions, shopping carts, uploaded files, in-progress wizards. A stateless instance keeps all of that in shared services (a session store, a database, object storage) or in the request itself (a signed token), so instances become interchangeable.
Run it
Three instances, 300 logged-in users, and app-2 crashes at t = 10 s. Compare Users logged out unexpectedly: round robin over in-memory sessions (A), sticky sessions (B) and a shared session store (C).
Where session state can live
| Option | Survives instance loss | Load balancing | Notes |
|---|---|---|---|
| Instance memory | No | Needs sticky sessions | Simplest; breaks on every deploy, crash and scale-in. |
| Sticky sessions | No | Uneven, pinned users | A workaround, not a fix: users of a lost instance are logged out. |
| Shared store (Redis) | Yes | Any algorithm | One extra fast lookup per request; the store must be highly available. |
| Signed token (JWT, cookie) | Yes | Any algorithm | No lookup; revocation and size need care — keep tokens small and short-lived. |
Spring Session with Redis
spring:
session:
store-type: redis # spring-session-data-redis on the classpath
timeout: 30m
data:
redis:
host: sessions.internal
port: 6379
# Controllers keep using HttpSession — the data now lives in Redis, not in this instance.Trade-offs
| Benefits | Costs | |
|---|---|---|
| Scaling | Add, remove or replace instances freely; autoscaling and rolling deploys just work. | Shared stores become critical dependencies that must scale too. |
| Resilience | A crash loses only in-flight requests. | If the session store is down, everyone is affected. |
| Latency | No need to route users to a particular instance. | An extra network call per request (usually ~1 ms). |
Production considerations
- Test statelessness: kill a random instance during a load test and check nobody is logged out.
- Run the session store replicated, with timeouts and a clear failure behaviour.
- Store uploads in object storage (S3, GCS), not on the instance disk.
- Run singleton jobs with a lock or a scheduler, not on every instance.