System Design Interview Questions, Answered

Start from users and actions, not from servers. The whole estimate is a few multiplications with aggressively rounded numbers.

1. DAU times actions per user per day10M DAU doing 5 writes each is 50M writes/day
2. ↓
   divide by ~100,000 seconds in a day500 QPS avgrounding 86,400 up keeps the math clean
3. ↓
   apply a peak factor of 2-5x2,500 QPS targettraffic is not flat; scale by how spiky the product is
4. ↓
   storage: rows x row size x retention x replication1 KB rows at 50M/day is 50 GB/day, ~18 TB/year before 3x replication

The skill interviewers grade is rounding aggressively and stating assumptions out loud, because the answer only needs to be right within an order of magnitude.

Know the ones that set floors on your design:

• memory read: ~100ns
• SSD read: ~100µs
• round trip inside one datacenter: ~0.5ms
• cross-region round trip: 50-150ms, depending on geography

Why they matter: every architecture question is secretly asking "how many of which hop can you afford?" A page with a 200ms budget can make hundreds of in-memory cache hits and a handful of intra-DC database calls, but only one cross-region call. Synchronous cross-region replication is off the table entirely.

The other number worth memorizing: a spinning disk seek costs ~10ms, which is why anything latency-sensitive lives on SSD or in RAM. Knowing these cold lets you reject a bad design in seconds instead of debating it.

Four algorithms cover it, and health checking matters more than any of them.

• Round robin: rotate through backends. The default; fine when requests cost roughly the same. Weighted variant sends more to bigger boxes.
• Least connections: pick the backend with the fewest in-flight requests. Wins when request cost varies wildly, because one slow endpoint would otherwise pile requests onto an already-busy backend.
• Hashing: map a key (client IP, user ID) to a consistent backend. Buys affinity, which keeps local caches warm and sticky sessions working.

The part that matters in production is health checking. The algorithm only chooses among backends the balancer believes are alive.

A check too slow to fail keeps routing traffic to a dead node; one too eager can eject the whole pool at once.

L4 balances connections; L7 balances requests. L4 forwards TCP on IP and port without reading the bytes; L7 terminates TLS and parses HTTP.

The difference bites on long-lived connections. gRPC and HTTP/2 multiplex many streams over one connection, so an L4 balancer pins all of a client's requests to a single backend and load skews badly; you need L7 to balance per stream. WebSockets live for hours, so least-connections at L4 slowly drifts unbalanced as backends restart.

Most real stacks run both: L4 at the edge, L7 in front of services.

Token bucket in Redis, keyed per user or per API key. Each key holds a token count that refills at the allowed rate; a request spends a token or gets a 429 with Retry-After.

The refill-and-spend must be atomic, so it runs as a single Lua script:

// one atomic Redis call per request
const allowed = await redis.eval(TOKEN_BUCKET_LUA, {
  keys: [`rl:${userId}`],
  arguments: [rate, burst, now],
});

Why token bucket over fixed windows: fixed windows allow a 2x burst at the boundary (full quota at 11:59, full quota again at 12:00). The bucket's capacity is your explicit burst policy instead of an accident.

At scale, the trap is Redis becoming a hot dependency on every request. The usual fix is a small local allowance per node that syncs to the shared store asynchronously, trading exactness for availability.

And fail open: a broken rate limiter should never take down the API it protects.

Match the tool to the traffic direction, not to whatever sounds most modern.

What breaks in production is the long-lived connection itself. Idle timeouts at every hop (ALB, nginx, corporate proxies) kill quiet connections around 60s, so both SSE and WebSockets need heartbeats.

And a box holding 50k open sockets makes deploys interesting. Every restart triggers a reconnect stampede, so you drain gradually and add jitter to client reconnect logic.

Default to Postgres and make NoSQL earn its place with a specific access pattern you can name. Relational gives you transactions, joins, constraints, and the freedom to ask questions you did not anticipate at design time; you give all of that up the moment you leave.

A single Postgres box handles tens of thousands of QPS, and modern Postgres covers JSON, full-text, and even vector search without new infrastructure.

The senior answer: this is a data-model decision, not a scale decision, until you have numbers proving otherwise.

It breaks read-your-writes: a user saves their profile, the redirect reads from a replica that has not applied the write yet, and the app shows their old data. It looks like data loss to the user and like a heisenbug to you.

The heisenbug part is because lag is single-digit milliseconds normally, then spikes to seconds or minutes during bulk writes, schema migrations, or vacuum pressure.

You live with it by deciding, per read, whether staleness is acceptable:

• Feeds and dashboards: replicas are fine.
• Anything a user just wrote: pin that session to the primary for 5-10 seconds after a write.
• Or track the replication position (Postgres LSN) in the session and only serve from replicas that have caught up.

The mistake to call out is treating replicas as free capacity for all reads. They scale the reads that tolerate staleness; the rest were always going to hit the primary.

Pick a shard key that matches how you read, usually tenant_id or user_id, so nearly every query lands on exactly one shard. Route in the application or a proxy layer, and make the key mandatory:

-- every hot-path query must carry the shard key
SELECT * FROM orders
WHERE user_id = $1 AND created_at > $2;

1. everything cross-shard breaks firstjoins across users, transactions touching two shards, global unique constraints
2. ↓
   analytics scatter-gatherscount(*) style queries fan out to every shard, or move to a warehouse
3. ↓
   auto-increment IDs stop being uniqueswitch to UUIDs or Snowflake-style IDs before sharding, not after
4. ↓
   your key choice breaks secondone whale tenant makes a shard hot

Resharding live data is the most dangerous migration most teams ever run. That is why you shard late and pick the key like it is permanent.

Only one thing: what happens during a network partition. Partition tolerance is not a choice; networks partition whether you like it or not, so "CA" is not a real option for a distributed system.

The interview trap is treating CAP as a permanent database personality. In practice the trade-off is per operation, not per system.

The extension worth naming is PACELC: even with no partition, you still trade latency against consistency, because a cross-region quorum on every write costs you 50-150ms. That trade is the one you pay every single day.

It solves the resize problem. With naive hash(key) % N, adding or removing one node changes N and remaps almost every key. For a cache cluster that means a near-total flush and a thundering herd on the database.

Consistent hashing places both nodes and keys on a ring (hash to a point on a circle); each key belongs to the first node clockwise from it. Now adding or removing a node only moves the keys in that node's slice, roughly 1/N of the data instead of nearly all of it.

The production refinement is virtual nodes: each physical node gets 100-200 points on the ring. Without them, a small cluster balances badly, and a node's failure dumps its entire load onto a single neighbor instead of spreading it.

This is the scheme under memcached clients, DynamoDB, and Cassandra, and why cache clusters scale without stampeding the tier below.

Four layers, outside in: browser cache, CDN at the edge, shared application cache (Redis or memcached), and the database's own buffer pool. Each layer absorbs traffic so the next one sees maybe a tenth of it, which is the only reason a 500-QPS database survives a 50k-QPS product.

They fail in reverse order of visibility:

1. browser cache fails quietlymisconfigured headers; you just pay for origin traffic you thought was cached
2. ↓
   CDN fails the same waysilent, and the bill is bigger
3. ↓
   shared cache fails loudlythe dangerous oneRedis dies or restarts cold
4. ↓
   database takes the full, unfiltered loadtraffic it has not seen in years, and it was sized assuming it never would

So the interview follow-through is: state your hit rate assumption (say 95%), then show the system survives the cache disappearing, via request coalescing, load shedding, or a database with actual headroom.

A TTL is a staleness budget, so start from the product question: how old can this data be before someone notices or money is wrong? Prices and permissions get seconds, profiles get minutes, a logo gets a day. Pick the number you could defend to a PM, not a number that makes the hit rate chart pretty.

Too long: you serve wrong data with no recall. That is why anything security-adjacent (sessions, permissions) needs explicit invalidation, not just expiry.

Too short: the cache stops protecting the origin, and you rediscover why it existed.

The failure people forget is synchronized expiry: cache 10,000 keys at deploy time with the same 3600s TTL and they all expire in the same second, stampeding the database. Add 10-20% random jitter to every TTL, and serve stale-while-revalidate for hot keys so one request refreshes while the rest keep getting the old value.

Feeds are read maybe 100x more than they are written, which is why fan-out on write is the default shape.

Pure write fan-out dies on celebrities: one post from an account with 50M followers is 50M queue jobs, minutes of delivery lag, and a storm of writes for a single action.

So production systems go hybrid, the approach Twitter made famous: fan out on write for normal accounts, and for the few thousand high-follower accounts, merge their posts in at read time instead.

The detail worth volunteering: fan-out is asynchronous through a queue, so feeds are eventually consistent, and the author sees their own post via read-your-writes special-casing.

Backpressure is the slow component telling its upstream to slow down, instead of silently absorbing the mismatch. Without it, the mismatch does not disappear; it accumulates in a queue somewhere until latency is unbounded or the process OOMs. An unbounded queue is just an outage with a delay.

Every hop applies it at its own boundary, because it only works when it propagates end to end:

1. consumer stops polling
2. ↓
   broker's bounded queue fills
3. ↓
   producer's send blocks or fails fast
4. ↓
   API returns 429Retry-After
5. ↓
   client backs off

TCP does exactly this with its receive window, which is why it is the canonical example.

The design decisions are where you bound each queue and what you do at the bound: block, shed the newest work, or drop the oldest. Choosing that explicitly is the difference between degrading and collapsing.

You gain decoupling in time and in knowledge; you lose the guarantees a synchronous call quietly gave you.

Debugging turns into archaeology: a request is now a story scattered across topics, so distributed tracing and a dead-letter queue stop being optional.

The sleeper cost is schema evolution. Events are a public API with unknown consumers, so changing a field becomes a versioned, negotiated migration.

A client-chosen token that makes a retried write safe. The server stores the first result under the key and replays it for duplicates instead of executing twice.

You need one anywhere a retry can double a side effect: payments, order creation, any POST that sits behind a queue or a flaky network. Reads and idempotent verbs (PUT, DELETE) do not need it; the method already guarantees safety.

POST /v1/charges
Idempotency-Key: 9f3c1e2a-order-8841

The implementation detail interviewers probe: the key plus the response must be stored atomically with the write itself (same transaction, or a unique constraint). Otherwise a crash between "do the work" and "record the key" reintroduces the duplicate.

Retries are a load multiplier that activates exactly when the system has the least capacity. A service running at 90% that starts failing 10% of requests, with clients retrying 3 times, suddenly serves 1.2-1.3x its normal traffic, tips over completely, and now every request retries.

That feedback loop is the retry storm, and stacked retries across layers (client, gateway, service mesh) multiply it further.

Safe retrying:

• only retry idempotent operations, or carry an idempotency key
• cap attempts at 2-3
• exponential backoff with full jitter, so retriers do not arrive in synchronized waves

const delay = Math.random() * base * 2 ** attempt;

The senior addition is a retry budget: allow retries to be at most, say, 10% of request volume, and stop retrying when the budget is spent. That keeps retries a repair mechanism instead of an amplifier.

A timeout protects one call; a circuit breaker remembers across calls and protects the fleet. With only a 2s timeout against a dead dependency, every request still burns a thread and a connection for the full 2 seconds, so your own pools drain and you fall over in sympathy. Timeouts cap the damage per request, not the aggregate.

1. closed: track the error ratea threshold like 50% failures over 30 calls trips it
2. ↓
   open: fail in microsecondsno traffic sentserve the fallback, a cached copy or a degraded response
3. ↓
   half-open: let a few probes throughafter a cool-off period
4. ↓
   close only on probe success

The open state is also what lets the sick dependency recover: it stops receiving your traffic while it is down.

The subtle part is half-open. Get it wrong and recovery itself re-triggers the outage.

Draw the request path end to end and ask of every box: what happens if this disappears right now? The obvious ones are a single load balancer, a database primary, or one region.

The ones that actually get people are boring: the NAT gateway all egress flows through, the DNS provider, the certificate that auto-renews on one cron box, the secrets service everything reads at boot, and that one Kafka topic every service consumes.

1. walk the request pathask of every box: what if it vanished right now?
2. ↓
   rank by blast radiuseach nine costs real money and complexity; do not remove alphabetically
3. ↓
   add redundancy plus failoverinstances behind health checks, replica with promotion, multi-AZ by default
4. ↓
   exercise the failoverkill a node in staging, run game days, actually pull the primary

Because redundancy you have never exercised is still a SPOF with extra steps.