Design a Chat Application

What is it? #

A chat system delivers messages between users in near real time and stores them so they can be read later.

The hard parts are not the messages themselves. They are persistent connections spread across many servers, delivery when the recipient is offline, ordering, and the sheer volume of message history.

Two things must both work: live delivery to connected users, and durable storage so nothing is lost.

Presence, typing indicators and read receipts multiply the traffic considerably — often more than the messages do.

Think of it like this #

A postal service combined with a telephone exchange. If the recipient is on the line, the message goes straight through. If not, it is stored and delivered when they reconnect.

Both paths must exist, and the sender should not have to know which one applied.

Simple example #

Direct messages and group chats up to 200 members, 50,000 concurrent connections, 5,000 messages per second at peak, with full history retained.

Code #

1. Requirements

functional      1:1 and group chat, history, delivery and read receipts,
                presence, attachments, push when offline
non-functional  delivery under 500ms, messages never lost, ordered per conversation
scale           50K concurrent connections, 5K messages/s, 10B messages stored

2. Architecture

   clients ──WebSocket──▶ connection servers (stateful, many)
                                  │
                                  ├── publish ──▶ Redis pub/sub (fan-out)
                                  │
                                  └── write ────▶ message store (Cassandra / partitioned SQL)
                                                        │
                                  offline users ◀── push notification service

A user connects to ONE connection server. A message from a user on server A
to a user on server B travels through pub/sub. Without it, cross-server
delivery does not happen.

# 3. Sending a message: persist first, then deliver
def send_message(sender_id: int, conversation_id: str, text: str, client_msg_id: str):
    # client_msg_id makes retries safe: the same send is stored once
    message = {
        "id": snowflake_id(),              # time-ordered, globally unique
        "conversation_id": conversation_id,
        "sender_id": sender_id,
        "text": text,
        "created_at": now_ms(),
    }

    messages.insert_if_absent(client_msg_id, message)      # durable first
    bus.publish(f"conv:{conversation_id}", message)        # then deliver live

    for member_id in conversations.members(conversation_id):
        if member_id == sender_id:
            continue
        if not presence.is_online(member_id):
            push.enqueue(member_id, message)               # offline path
    return message

4. Storage and ordering

Partition key:  conversation_id        all messages of a chat stay together
Clustering:     message_id descending  recent messages read first
Message IDs:    Snowflake-style — timestamp + node + sequence

Why not rely on timestamps for ordering: clocks differ between servers,
and two messages can share a millisecond. A time-ordered ID gives a
total order within a conversation, which is what clients render.

Reading history:
  SELECT * FROM messages
  WHERE conversation_id = ? AND message_id < ?      -- cursor, not offset
  ORDER BY message_id DESC LIMIT 50

5. Receipts and presence — the hidden load

delivered  one event per recipient per message
read       one event per recipient per conversation, batched
typing     high frequency, never stored, throttled to one per 3 seconds
presence   heartbeat every 30s, stored in Redis with a TTL

A 200-member group means one message generates up to 400 receipt events.
Batch and throttle these, or they become the dominant traffic.

How it works #

Persisting before delivering is the ordering that matters. If you deliver first and the write fails, some users saw a message that does not exist. Writing first means the worst case is a message stored but not delivered live, which the client fixes by fetching history on reconnect.

client_msg_id makes sending idempotent. Mobile clients retry constantly on flaky networks, and without it every retry creates a duplicate message.

Connection servers are stateful, which makes them the awkward tier. Redis pub/sub connects them: each server subscribes to the conversations its connected users belong to, and a message published anywhere reaches the right servers.

Partitioning by conversation keeps every read local to one partition. The most common query — the last fifty messages of one conversation — touches one partition and uses the clustering order directly.

Snowflake-style IDs give a total order without relying on synchronised clocks. Clients sort by ID, so everyone sees the same sequence.

Cursor pagination is essential here, because new messages arrive constantly; offset-based paging would skip and duplicate.

Receipts are the load nobody predicts. Batching read receipts per conversation rather than per message, and throttling typing indicators, is what keeps the system affordable.

Real-world use #

Real systems add end-to-end encryption, which changes the design considerably: the server stores ciphertext and cannot search it, so search moves to the client.

Attachments follow the object storage lesson: upload directly with a presigned URL and send only the key in the message.

Mobile push notifications are a separate reliability problem, with platform-specific services, token management and the question of how much content to include in the notification.

Group size drives the architecture. Small groups can fan out to every member; a broadcast channel with 100,000 members needs a different model, usually a shared feed clients pull from rather than per-member fan-out.

Message history dominates storage cost over time. Retention policies, cold storage tiers and per-conversation archiving are standard once the system is a few years old.

Common mistakes #

• Delivering before persisting, so a failed write leaves phantom messages.
• No client message ID, so network retries create duplicates.
• Keeping connection state on one server with no pub/sub between servers.
• Ordering by server timestamp rather than a time-ordered ID.
• Treating typing and read receipts as cheap — they often exceed message traffic.

Practice #

Extend this design with message editing and deletion. Specify how history stays consistent for users who already received the message, how offline users learn about the change, and what the storage model looks like. Then estimate the receipt traffic for a 500-member group.