Skip to main content

System Design

Overview​

System design covers the principles and patterns for building scalable, reliable, and maintainable distributed systems — a core topic in senior engineering interviews.

Key Concepts​

Scalability​

  • Vertical scaling (scale up) — bigger machine. Simple but has limits.
  • Horizontal scaling (scale out) — more machines. Requires stateless services.

CAP Theorem​

A distributed system can guarantee at most 2 of 3:

  • Consistency — every read gets the latest write.
  • Availability — every request gets a response (not necessarily latest).
  • Partition Tolerance — system works despite network partitions.

In practice: partition tolerance is required. Choose CP (e.g., HBase) or AP (e.g., Cassandra, DynamoDB).

Load Balancing​

Distributes traffic across multiple servers:

  • Round Robin — sequential distribution.
  • Least Connections — route to server with fewest active connections.
  • IP Hash — consistent routing based on client IP (session affinity).
  • Layer 4 (TCP) — faster, less context.
  • Layer 7 (HTTP) — routes based on URL, headers, cookies.

Caching​

Client → CDN → Load Balancer → App Server → Cache (Redis) → Database
  • Cache-aside (lazy loading) — app checks cache first; on miss, loads from DB and populates cache.
  • Write-through — write to cache and DB simultaneously.
  • Write-behind — write to cache, async flush to DB.
  • TTL — time-to-live; expiry for cached entries.
  • Eviction policies — LRU, LFU, FIFO.

Databases​

SQL vs NoSQL​

SQL (Relational)NoSQL
SchemaFixedFlexible
ScalingVertical (read replicas for reads)Horizontal
ACIDYesOften eventual consistency
Best forComplex queries, joinsHigh throughput, flexible data

Database Patterns​

  • Read replicas — offload reads from primary.
  • Sharding — partition data horizontally across multiple DBs.
  • Connection pooling — reuse DB connections (HikariCP).
  • Indexing — B-tree (range queries), Hash (equality), full-text.

Message Queues​

Decouple producers from consumers, enable async processing:

  • Kafka — high-throughput, durable, log-based, replay.
  • RabbitMQ — flexible routing, lower throughput, message acknowledgement.
  • SQS — managed, AWS-native, simple.

API Design​

  • REST — stateless, resource-based, HTTP verbs.
  • GraphQL — client specifies exactly what data it needs.
  • gRPC — binary protocol (Protobuf), high performance, streaming.

Rate limiting: Token bucket, leaky bucket, fixed/sliding window.

High Availability Patterns​

  • Circuit Breaker — stop calling a failing service; fallback response.
  • Retry with backoff — exponential backoff + jitter.
  • Bulkhead — isolate failures (thread pool per service).
  • Health checks — liveness + readiness probes.
  • Multi-region deployment — active-active or active-passive.

Interview Framework​

For any system design question, follow:

  1. Clarify requirements — functional and non-functional (scale, latency, consistency).
  2. Estimate scale — DAU, QPS, storage.
  3. High-level design — components and data flow diagram.
  4. Deep dive — focus on the hard parts (DB schema, API design, bottlenecks).
  5. Identify tradeoffs — explain your choices.

Cheat Sheet​

Scale: Vertical (bigger) | Horizontal (more)
CAP: Consistency | Availability | Partition Tolerance (pick 2)
Cache: Redis | CDN | Cache-aside | Write-through | TTL | LRU
Queue: Kafka | RabbitMQ | SQS
LB: Round Robin | Least Connections | Layer 4/7
HA: Circuit Breaker | Retry + Backoff | Health Checks | Multi-region
DB: Read replicas | Sharding | Connection pool | Indexing