What is it? #
A social feed shows each user a list of posts from the accounts they follow, newest or most relevant first.
The whole design hinges on one decision: whether you build each user's feed when a post is created, or when they open the app.
Fan-out on write pushes each new post into every follower's precomputed feed. Reads become trivial; writes become expensive for popular accounts.
Fan-out on read queries the posts of everyone you follow at request time. Writes are cheap; reads become expensive for users who follow many accounts.
Real systems use both, chosen per account.
Think of it like this #
A newspaper delivery round versus a library.
Delivery means every subscriber wakes up to the paper already on the doorstep — instant to read, expensive to distribute. The library means nothing is delivered, but each visitor has to search the shelves themselves.
A publisher with ten million subscribers would find delivery impossible, so those readers collect from the library instead.
Simple example #
10 million users, average 200 follows, 1,000 posts per second, 50,000 feed reads per second. A small number of accounts have more than a million followers.
Code #
1. The two strategies
FAN-OUT ON WRITE FAN-OUT ON READ
post created user opens the app
→ append post ID to every → fetch the list of accounts followed
follower's feed list → query recent posts from each
→ read is one lookup → merge, rank and return
fast reads, expensive writes cheap writes, slow reads
breaks for accounts with breaks for users following
millions of followers thousands of accounts# 2. Hybrid: push for normal accounts, pull for very large ones
CELEBRITY_THRESHOLD = 100_000
def publish_post(author_id: int, content: str) -> str:
post_id = snowflake_id()
posts.insert(id=post_id, author_id=author_id, content=content, created_at=now())
follower_count = social.follower_count(author_id)
if follower_count < CELEBRITY_THRESHOLD:
# push: append to each follower's precomputed feed
for batch in social.followers_in_batches(author_id, size=1000):
fanout_queue.enqueue("append_to_feeds", {"post_id": post_id, "user_ids": batch})
# else: do nothing. Followers will pull this account's posts at read time.
return post_id
def get_feed(user_id: int, cursor: str | None = None, limit: int = 30) -> list:
pushed = feed_store.range(f"feed:{user_id}", cursor, limit) # precomputed
# pull posts from the large accounts this user follows
celebrities = social.followed_celebrities(user_id) # usually few
pulled = posts.recent_by_authors(celebrities, after=cursor, limit=limit)
merged = merge_by_id_desc(pushed, pulled)[:limit]
return rank(merged, viewer_id=user_id)3. Storage
posts id (snowflake), author_id, content, media_keys, created_at
follows follower_id, followee_id — indexed both directions
feeds Redis list or sorted set per user, capped at ~1000 entries
stores post IDs only, not content — hydrate on read
counters likes and comments counted separately, updated asynchronously
Capping the stored feed matters: nobody scrolls past a thousand posts,
and unbounded lists would dominate memory.4. Why the feed stores IDs, not posts
A post copied into 500,000 feeds is 500,000 copies to update when it is
edited or deleted. Storing IDs means one source of truth, hydrated at read
time from a cache. The hydration is a single multi-get, which is fast.How it works #
The threshold is the whole design. Below it, pushing to followers is affordable and reads are a single lookup. Above it, pushing would mean millions of writes per post, so those posts are pulled instead.
The pull side stays cheap because a user follows very few large accounts — typically a handful — so the read-time query covers a small set of authors.
Fan-out happens asynchronously through a queue. A post should be created in milliseconds; distributing it to 50,000 feeds happens in the background within a second or two.
Storing only post IDs in feeds is what makes edits and deletions manageable. Hydrating thirty IDs from a cache is one batch operation.
Capping each feed at around a thousand entries bounds memory. Users who scroll further fall back to a query, which is rare enough not to matter.
Ranking runs at read time on the merged list. Chronological order is simplest; engagement-based ranking needs signals, and doing it at read time means the model can change without rebuilding stored feeds.
Counters for likes and comments are updated asynchronously and often approximated at high values, because exact counts at that volume are expensive and nobody needs them.
Real-world use #
Every large social platform has publicly described some version of this hybrid approach, because the celebrity problem forces it.
Ranking is where most product work goes: recency, affinity, engagement prediction and diversity all combine, and the ranking layer usually changes far more often than the storage layer.
Feed regeneration is an operational reality. When a user follows a new account, or when a bug corrupts a feed, you need a way to rebuild one user's feed from scratch.
Deletion and privacy changes have to propagate. A post made private must disappear from feeds it was already pushed into, which is another reason to store IDs and check visibility at hydration time.
Media follows the object storage pattern, with a CDN in front, and thumbnails generated asynchronously after upload.
Common mistakes #
- Choosing one strategy for all accounts and breaking on either celebrities or heavy followers.
- Storing full post content in every follower feed, multiplying update cost.
- Fanning out synchronously during the post request.
- Unbounded feed lists that consume memory indefinitely.
- Checking visibility only at write time, so privacy changes do not take effect.
Practice #
Extend this design to support a "following" feed and a "for you" ranked feed. Specify what is stored for each, where ranking happens, and how a newly followed account appears in the feed. Then estimate the fan-out cost of a post by an account with 2 million followers under each strategy.