Skip to content
System Design Lab
Case Studies

Design a Notification System

Send push, SMS and email at scale without letting a marketing blast delay a login code. From a synchronous call to a queue with workers and priority lanes, plus retries, preferences and deduplication.

Intermediate 5 min read· Updated 2026-11-18

What you will learn

  • Separate accepting a notification from delivering it.
  • Protect urgent notifications from bulk traffic with priority lanes.
  • Handle provider failures with retries, dead letters and idempotency.
  • Respect user preferences, quiet hours and rate limits per channel.

Before you start

Requirements

  • Functional: services request a notification for a user (login code, order shipped, marketing); the system renders a template, picks channels (push, SMS, email) from the user's preferences, and delivers.
  • Non-functional: urgent messages (login codes, security alerts) arrive within seconds; no notification is lost; users are never spammed with duplicates; bulk campaigns must not hurt urgent traffic.
  • Constraints: third-party providers (APNs/FCM, an SMS gateway, an email service) are slow, rate-limited and sometimes down.

Back-of-the-envelope estimates

QuantityAssumptionResult
Transactional10 M users, ~5 per user per day~600/s average, bursts ×5
CampaignsA blast to 10 M users10 M messages that should drain within ~1 hour (~2,800/s)
Provider latency100 ms (push) to 1 s (SMS)In-flight sends = rate × latency (thousands at once)
Storage~1 KB per notification record, 90 days~5 TB, mostly cold

High-level design

  1. Notification API validates the request, checks preferences and quiet hours, stores a notification record and enqueues one message per channel — then returns 202 Accepted.
  2. Queues per priority and channel buffer work (queue-based load leveling).
  3. Sender workers (competing consumers) render and call providers, with per-provider rate limits and retries.
  4. Delivery status (sent, delivered, failed, opened) flows back via provider callbacks into the record.

Evolve the design

At t = 10 s marketing launches a 1,600-message campaign while users keep requesting login codes. Watch what each version does to the codes.

Stage 1 of 3

Send inside the API request

Current architecture
The API calls the provider and answers when the message is sent.
Limitation
API threads are held for the provider's full latency; throughput = threads ÷ latency.
Bottleneck
The API thread pool: the campaign fills it, login codes slow past a second and some are rejected.
Improvement
Accept quickly and deliver in the background.
Performance impact
Most campaign messages fail; login-code p95 > 1 s with failures.
Complexity & operational cost
No extra infrastructure.
Trade-off
Simple, but every slow provider becomes your outage.
Loading simulation…

Deep dives

  • Exactly-once feel: queues deliver at least once, so give each notification an id and make workers idempotent (skip ids already sent); pass the id to providers that support idempotency keys.
  • Failures: retry transient provider errors with backoff, move permanent failures to a dead-letter queue, and fail over to a second provider for critical channels.
  • Provider limits: apply rate limiting per provider account so workers do not get throttled or blocked.
  • User experience: respect opt-outs and quiet hours, collapse duplicates ("5 new messages" instead of 5 pushes), and expire messages that are useless when late (a login code after 5 minutes).

Trade-offs

BenefitsCosts
AsynchronyAPI stays fast and providers can fail without losing requests.Callers learn delivery status later, via callbacks or polling.
PrioritiesUrgent messages arrive in seconds during any campaign.More queues and capacity planning per lane.
At-least-onceNothing is silently dropped.Deduplication is required to avoid double sends.

Interview checklist

  • Split accept from deliver, and say why (slow, flaky providers).
  • Prioritise transactional over bulk; mention per-provider rate limits.
  • Cover retries, DLQ, idempotency and provider failover.
  • Mention preferences, quiet hours, templates and delivery tracking.

Further reading

Related topics