Where we are
Last week's Webhook Delivery System sent events to servers you don't control. Today the other end is a person. A person can't return a 500, but they can switch off your notifications, mark your email as spam, or delete your app. Those are worse than any error code.
So a notification system has two jobs. Deliver the messages that matter, reliably, on the right channel. And refuse to send the ones that don't. By the end you'll have a working system that passes eight checks, from "the same event three times is one notification" to "an email outage can't slow down a single push".
The problem in one picture
A food app sends three kinds of messages. "Your code is 482913" must arrive in seconds or the login fails. "Your order is out for delivery" is useful for about half an hour. "Priya liked your review" is pleasant once and irritating the twentieth time.
Most designs treat all three the same: put a message on a queue, a worker sends it. That works on the whiteboard and fails in three ways in production.
Too much. LinkedIn cut its email volume in half after members complained, and complaints fell by more than 65% (LinkedIn blog, 2015). Sending less was the fix, not sending faster.
Too late. Netflix's RENO notification service drops events that are older than a set age, because many updates lose their value if they arrive late (InfoQ on Netflix RENO).
Stuck behind someone else. RENO also splits its queues and workers by priority, so a backlog of low-value events can't delay urgent ones.
The interviewer wants to hear you design for all three. Throughput is the easy part.
Core idea: four questions, then a lane
Every notification answers four questions, in this order, before it's allowed near a provider.
Have we seen it already? Producers retry. The order service may call you three times for one shipment. A dedupe key turns three calls into one notification.
Is this channel allowed? The user may have turned off email for likes but kept push. Preferences are checked per category and per channel, never as one on/off switch.
Is it the middle of their night? Quiet hours are computed in the user's own time zone. Non-urgent messages wait until morning. A login alert does not.
Have we sent too many? Past a cap, new messages of the same kind fold into one digest instead of each buzzing the phone.
Only then does a message enter its channel's lane: push, email or SMS. Each lane has its own workers, retry schedule and rate limit. One slow channel can't block another.
Two words you'll use all lesson:
Critical messages (OTPs, security alerts) skip quiet hours and caps. Keep this list short, or everything becomes critical.
Every "no" is written down. A suppressed or failed send is a row with a reason. When a user asks "why didn't I get my code?", support can answer from a query, not a guess.
