News Feed
Pull the celebrity
Before you start
- Open lesson
A celebrity is not a hash problem
The celebrity path is the hot-write fix from that lesson.
Why it matters
The background work that is harmless for an author with 200 followers becomes an outage for an account with 20 million. One post can occupy workers long enough to delay ordinary posts, while some readers follow dozens of such accounts and still expect correctly ordered home pages. One celebrity post must not create 20 million timeline rows.
Example
Design the celebrity path
One account has 20 million followers and posts. Pushing that post into every timeline is the outage.
- Input
- user = "bo", timelines = {"bo": [[5, "ana: lunch"], [2, "cy: hi"]]}, celebrity_posts = {"star": [[6, "star: tour!"], [1, "star: hello"]]}, follows_celebs = {"bo": ["star"]}, limit = 3
- Output
- ["star: tour!", "ana: lunch", "cy: hi"]
Mental model
The idea
Keep one example in view
Design the celebrity path
user = "bo", timelines = {"bo": [[5, "ana: lunch"], [2, "cy: hi"]]}, celebrity_posts = {"star": [[6, "star: tour!"], [1, "star: hello"]]}, follows_celebs = {"bo": ["star"]}, limit = 3→["star: tour!", "ana: lunch", "cy: hi"]
Keep this exact scenario and its constraint in view while you build the model.
Set a threshold that keeps Ana’s 200-follower post on normal fanout, but store the 20-million-follower star’s post once; when Bo opens home, merge that star list with Bo’s pushed timeline by timestamp.
Choose a follower threshold from worker capacity and freshness targets. Below it, posts follow the normal timeline fanout. Above it, “star: tour!” is written once to the star’s cached recent-post list rather than copied to 20 million followers.
When Bo opens home, load Bo’s pushed [ana: lunch, cy: hi] timeline and the recent lists for celebrities Bo follows. Merge all candidates by timestamp so star’s time-6 post appears above Ana’s time-5 post. Following 50 celebrities adds 50 cached reads, so cache and cap those lists.
Pro
The rest of this is on Pro
The idea above is free. The walkthrough, the cost, and the practice are on Pro.