TweakCode
Worked Systems lessons

News Feed

Pull the celebrity

Before you start

  • A celebrity is not a hash problem

    The celebrity path is the hot-write fix from that lesson.

    Open lesson

Why it matters

The background work that is harmless for an author with 200 followers becomes an outage for an account with 20 million. One post can occupy workers long enough to delay ordinary posts, while some readers follow dozens of such accounts and still expect correctly ordered home pages. One celebrity post must not create 20 million timeline rows.

Example

Design the celebrity path

One account has 20 million followers and posts. Pushing that post into every timeline is the outage.

Input
user = "bo", timelines = {"bo": [[5, "ana: lunch"], [2, "cy: hi"]]}, celebrity_posts = {"star": [[6, "star: tour!"], [1, "star: hello"]]}, follows_celebs = {"bo": ["star"]}, limit = 3
Output
["star: tour!", "ana: lunch", "cy: hi"]

Mental model

The idea

Keep one example in view

Design the celebrity path

user = "bo", timelines = {"bo": [[5, "ana: lunch"], [2, "cy: hi"]]}, celebrity_posts = {"star": [[6, "star: tour!"], [1, "star: hello"]]}, follows_celebs = {"bo": ["star"]}, limit = 3→["star: tour!", "ana: lunch", "cy: hi"]

Keep this exact scenario and its constraint in view while you build the model.

Set a threshold that keeps Ana’s 200-follower post on normal fanout, but store the 20-million-follower star’s post once; when Bo opens home, merge that star list with Bo’s pushed timeline by timestamp.

Choose a follower threshold from worker capacity and freshness targets. Below it, posts follow the normal timeline fanout. Above it, “star: tour!” is written once to the star’s cached recent-post list rather than copied to 20 million followers.

When Bo opens home, load Bo’s pushed [ana: lunch, cy: hi] timeline and the recent lists for celebrities Bo follows. Merge all candidates by timestamp so star’s time-6 post appears above Ana’s time-5 post. Following 50 celebrities adds 50 cached reads, so cache and cap those lists.

postwrite per follower1 writeNormal authorFanout queueTimelinespushed postsCelebrityCelebrity postsstored onceHome readmerge both
Normal posts are pushed into timelines; celebrity posts are stored once. Home reads the timeline and the few celebrity lists, then merges them.

Pro

The rest of this is on Pro

The idea above is free. The walkthrough, the cost, and the practice are on Pro.