Skip to content
System Design Lab
Design to Scale

Scaling with Message Queues

Move slow work out of the request path: a queue absorbs bursts, competing consumers scale the processing, backpressure keeps the system honest and a dead-letter queue catches poison messages.

Intermediate 5 min read· Updated 2026-11-18

What you will learn

  • Explain queue-based load leveling and when asynchronous processing is acceptable.
  • Scale processing with competing consumers and predict drain time.
  • Apply backpressure with bounded queues instead of letting delays grow without limit.
  • Handle failing messages with retries and a dead-letter queue.

Before you start

Requirements

Users upload images that must be resized. Resizing takes ~200 ms of CPU. Normal traffic is 10 uploads per second, but a marketing email produces bursts of 50 per second for several seconds. Users should never see an error, and thumbnails may appear a few seconds later.

A restaurant does not cook your meal at the counter while the queue behind you waits. The cashier takes the order, puts the ticket on the rail, and the kitchen works through tickets at its own pace. More cooks clear the rail faster. That rail is a message queue.

Evolution

Stage 1 of 3

Do the work in the request

Current architecture
The upload API resizes the image before responding.
Limitation
Capacity is what the API workers can process: 4 workers × 5 images/s = 20/s.
Bottleneck
During the 50/s burst requests queue up, time out and are rejected — users see errors.
Improvement
Accept the upload quickly and do the resize later, in the background.
Performance impact
Fine at 10/s; during the burst p95 rises past 600 ms and ~23% of uploads fail.
Complexity & operational cost
Simplest code path.
Trade-off
Immediate results, but capacity must be sized for the peak, not the average.
Loading simulation…

Will the queue drain?

A queue only levels load if average processing capacity exceeds average arrival rate. During a burst of 50/s for 6 s against 20/s of capacity, the backlog grows by 30 × 6 = 180 messages. Afterwards, at 10/s arrivals, it drains at 20 − 10 = 10/s: about 18 seconds. If arrivals stay above capacity, the queue grows forever — that is not leveling, that is an outage with a delay (see Kafka Consumer Lag).

Poison messages and the dead-letter queue

A corrupt image fails every time. Without a limit it is retried forever, wasting a worker and blocking ordered queues. After a fixed number of deliveries, move it to a dead-letter queue (DLQ) where it can be inspected and replayed after a fix. In advanced mode, raise the failure rate in the queue scenarios and watch the Dead-lettered tile.

Implementation with Spring AMQP (RabbitMQ)

ResizeQueueConfig.java
@Configuration
class ResizeQueueConfig {

    @Bean
    Queue resizeJobs() {
        return QueueBuilder.durable("resize-jobs")
                .maxLength(10_000)                        // bounded: backpressure
                .overflow(QueueBuilder.Overflow.rejectPublish)
                .deadLetterExchange("")
                .deadLetterRoutingKey("resize-jobs.dlq")
                .build();
    }

    @Bean
    Queue resizeJobsDlq() {
        return QueueBuilder.durable("resize-jobs.dlq").build();
    }
}
ResizeWorker.java
@Component
class ResizeWorker {

    // 4–16 concurrent consumers per instance; prefetch 1 so a slow job does not hoard messages.
    @RabbitListener(queues = "resize-jobs", concurrency = "4-16")
    public void resize(ResizeJob job) {
        images.resize(job.imageId(), job.sizes());   // must be idempotent: redelivery happens
    }

    private final ImageService images;

    ResizeWorker(ImageService images) {
        this.images = images;
    }
}

// application.yml
// spring.rabbitmq.listener.simple.prefetch: 1
// spring.rabbitmq.listener.simple.retry.enabled: true
// spring.rabbitmq.listener.simple.retry.max-attempts: 3   # then rejected → DLQ
// spring.rabbitmq.listener.simple.default-requeue-rejected: false

Trade-offs

BenefitsCosts
CapacitySize for the average load; bursts are absorbed.Work is delayed during bursts.
ResilienceWorkers can restart or fail without losing accepted work.Messages can be redelivered: handlers must be idempotent.
User experienceFast, reliable acknowledgement.Asynchronous results need polling, notifications or websockets.
OrderingFIFO per queue with one consumer.Competing consumers process in parallel and can finish out of order.

Production considerations

  • Alert on queue depth and on the age of the oldest message — depth alone hides slow drains.
  • Autoscale workers on depth or age (KEDA does this on Kubernetes), with a maximum that the downstream systems can survive.
  • Monitor the DLQ: anything there is a bug or bad data that someone must look at.

Further reading

Related topics