How would you scale a backend from one server to millions of users?
System designMid-level
In shortMake the app servers stateless and run many behind a load balancer, then relieve the database step by step with indexes, caching, read replicas and queues, and partition data only when writes or size require it.
Model answer
Scale in response to bottlenecks, roughly in this order:
- Separate the database from the app server, and add the obvious indexes.
- Stateless app servers behind a load balancer: sessions in Redis or tokens, files in object storage, scheduled jobs run once. Health checks (readiness) and graceful shutdown make deploys and failures invisible.
- Caching: a CDN for static and public content, Redis for hot reads (cache-aside with TTLs), HTTP caching headers for clients.
- Read replicas for read-heavy load, with read-your-writes handling for users who just changed something.
- Queues for slow or unreliable work (emails, images, third-party calls), with idempotent consumers.
- Partitioning (sharding) when writes or data size outgrow one primary, with a shard key that spreads load and keeps hot queries on one shard.
At every step, watch the next bottleneck: database connections (pool size × servers), write throughput, hot keys. Many products never need step 6.
What they're looking for
A sensible order of steps tied to bottlenecks, stateless servers, health checks, caching and replicas before sharding, and awareness of the database as the usual limit.
Common mistakes
- Jumping straight to microservices or sharding.
- Forgetting where sessions and files go.
- Ignoring connection limits and replica lag.