Design WhatsApp / a messaging service

The setup

"Design the messaging backbone of WhatsApp. Focus on 1-on-1 chats. Include sending and delivering messages, sent/delivered/read receipts, and online presence. Assume 2 billion users globally; peak ~100M concurrent connections."

The interviewer is probing for: connection management at scale, message delivery semantics, ordering guarantees, and the operational realities of mobile networks. They're not interested in encryption details ("assume it's a black box that turns plaintext into bytes"). They want to know if you understand the distributed-systems realities of "deliver a message reliably, eventually, in order."

Clarifying questions

Capacity estimation

Users: 2B total, ~500M active in a day, ~100M concurrent peak
Messages: ~100B / day → ~1.2M msg/sec average, 5M/sec peak
Avg message size: ~100 bytes text + ~200 bytes metadata = 300 bytes
Storage: 100B × 300 bytes = 30 TB / day raw
30-day retention before drop: ~900 TB working storage
Connections: 100M concurrent WebSockets
Connection bandwidth: ~10 KB / sec sustained per connection (heartbeats, presence) = 1 TB/sec total (handled by edge tier)

The three fundamental challenges

WhatsApp-style messaging breaks down into three subproblems, each its own engineering effort:

1. Connection management: holding 100M concurrent WebSocket connections, knowing where each user is currently connected, and routing messages to the right server.
2. Message delivery: reliably moving a message from sender to recipient with ordering and at-least-once guarantees, including offline buffering.
3. State synchronization: delivery receipts, read receipts, presence, typing indicators. All of these are real-time state pushed back to interested parties.

The architecture has separate concerns for each, glued together by a central message service.

Design Decision 1: Connection architecture

100M concurrent WebSockets is a hard problem. Per-server connection limits:

For 100M concurrent at ~30K stable per node: ~3,300 connection-handling nodes. That's a "WebSocket gateway tier" — thin, stateful (holds the connection), specialized for connection management. Other services don't know about connections directly.

Connection registry: a key-value store mapping user_id → gateway_node_id. Updated when user connects/disconnects. Redis is fast enough; sharded by user_id. Lookup latency <1ms.

Design Decision 2: Message delivery path

sequenceDiagram
    participant S as Sender
    participant SG as Sender Gateway
(WebSocket)
    participant Q as Recipient queue
    participant RG as Recipient Gateway
    participant R as Recipient (online)

    S->>SG: send "hello", msg_id=42
    SG->>Q: enqueue for recipient
    SG-->>S: server-ack (✓)

    alt recipient online
        Q->>RG: deliver
        RG->>R: push msg_id=42
        R-->>RG: client-ack
        RG-->>S: delivered-ack (✓✓)
    else recipient offline
        Q->>Q: hold until reconnect
        Note over R: reconnects later
        R->>RG: HELLO last_seen=41
        Q->>RG: deliver msg_id=42
    end

When Alice sends "hi" to Bob, what's the flow?

1. Alice's WSS gateway receives the encrypted payload (Alice's WSS connection)
2. Gateway forwards to Message Service with (from=Alice, to=Bob, msg_id, payload, timestamp)
3. Message Service:
   a. Persists to hot store (Redis): bob's_undelivered_queue.push(msg_id)
   b. Persists to cold store (Cassandra) asynchronously for history
   c. Looks up Bob's gateway in connection registry
   d. If Bob is online:
      → Send via internal pub/sub to Bob's gateway
      → Gateway forwards over WSS to Bob's device
      → Bob's device acks delivery
      → Ack flows back: Bob's gateway → Message Service → mark delivered → notify Alice's gateway → Alice sees "delivered"
   e. If Bob is offline:
      → Leave in undelivered queue
      → Will flush on Bob's next connect

Two storage tiers: hot store (Redis) for undelivered queue and "recent" messages; cold store (Cassandra) for historical messages users can scroll back through.

Conversation_id is the canonical pair: min(user_a, user_b) || "_" || max(user_a, user_b). Within a partition (one conversation, one time bucket), messages clustered by timestamp — efficient "scroll back" reads.

Design Decision 3: Ordering

Per-conversation strict ordering is required. Two challenges:

1. Alice sends 3 rapid messages. Bob must see them in order.
2. Alice on phone, Alice on laptop both send messages. Bob must see a consistent order.

Approach: every message gets a monotonically increasing message_id per conversation, assigned server-side at the Message Service. The client provides a clock hint (its local timestamp) but the server is authoritative. This avoids client clock skew issues.

Within a conversation, messages are dispatched in message_id order. The Message Service writes to Kafka with partition_key = conversation_id, so all messages for a conversation go to the same Kafka partition — consumed in order by Bob's gateway.

Cross-examination round 1: bad network

Cross-examination round 2: presence

Cross-examination round 3: storage and retention

messages table:
  PARTITION KEY: (conversation_id, year_month)
  CLUSTERING KEY: message_timestamp DESC, message_id

For "show me the last 50 messages in conversation X":
  SELECT * FROM messages
  WHERE conversation_id = $1 AND year_month = current_month
  ORDER BY message_timestamp DESC
  LIMIT 50

Why year_month bucketing: prevents unbounded partition growth.
A single conversation could accumulate millions of messages over years.
Without bucketing, the partition becomes a hot, oversized partition.

Failure mode analysis

Scaling phases

Takeaway

Messaging looks simple ("just send a message from A to B") but at scale every primitive becomes a distributed-systems problem. Connection management, delivery ordering, presence fanout, offline buffering, multi-region routing — each is its own subsystem. The architectural discipline is to keep each subsystem's responsibility tight (WSS gateway only handles connections; message service handles persistence and routing; presence is its own thing) and to design every flow with reconnect/retry/duplication in mind because mobile networks are hostile by default.

The deeper lesson: state on the edge (mobile clients) is fundamentally messy and must be treated as such. Server-side correctness is necessary but not sufficient — the protocols between server and client must be reconnect-safe, message-id-based, idempotent. Every protocol decision must answer: what happens if this fails partway through?