38 Core System Design Concepts Every Senior Engineer Must Master
From Horizontal Autoscaling and Availability Nines to CAP Theorem, Consistent Hashing, and Raft Consensus.
Table of Contents (38 Concepts)
1. Scalability: Horizontal vs Vertical Dimensions
Scalability is a system’s ability to handle growing amounts of work or traffic by adding resources without compromising performance or stability. Vertical scaling (scaling up) upgrades single-server compute (CPU, RAM, NVMe). Horizontal scaling (scaling out) provisions stateless commodity server instances behind a load distributor.
"Vertical scaling is like upgrading from a small restaurant kitchen to a massive commercial industrial oven. Horizontal scaling is like opening 5 new restaurant branches in different neighborhoods to serve more customers concurrently."
[ Vertical Scaling ] [ Horizontal Scaling ]
┌──────────────────┐ ┌────────┐ ┌────────┐ ┌────────┐
│ Server (128GB) │ │ Server │ │ Server │ │ Server │
│ (64 Cores) │ │ (8GB) │ │ (8GB) │ │ (8GB) │
└──────────────────┘ └────────┘ └────────┘ └────────┘
(Hits Ceiling) ▲ ▲ ▲
└───── Load Balancer ─┘2. High Availability & The Rule of Nines
Availability measures the percentage of time a system remains operational and accessible to process requests over a given period. It is commonly expressed in "nines" (e.g. 99.9% allows 8.76 hours of downtime/year; 99.999% allows only 5.26 minutes of downtime/year).
"Availability is like a 24/7 convenience store. If the front automatic door breaks, a high-availability store has a second backup manual door so customers can still enter without interruption."
Three Nines (99.9%) -> ~8h 45m downtime / year Four Nines (99.99%) -> ~52m 35s downtime / year Five Nines (99.999%) -> ~5m 15s downtime / year (Financial target)
3. Reliability & Idempotent API Execution
Reliability is the probability that a system performs its required function correctly without error under specified operating conditions. While availability measures uptime, reliability ensures outputs are strictly accurate and consistent.
"A car might start 100% of the time (100% available), but if the brakes fail 5% of the time while driving, it is severely unreliable."
[Client] ──(Idempotency Key: req_981)──> [Payment Gateway]
│ (Network timeout?)
[Client] ──(Retry with SAME key)───────> [Payment Gateway]
│
[Check Cache / DB]
-> Already charged! Return cached receipt.4. Latency vs Throughput vs Bandwidth Optimization
Latency is the time delay for a packet to travel from source to destination. Throughput is the number of successful operations completed per unit of time (QPS/RPS). Bandwidth is the total data link capacity.
"Think of a highway: Latency is how fast a single sports car can drive from City A to City B. Throughput is how many total cars pass through the toll booth every hour. Bandwidth is the number of lanes on the highway."
Latency : Time for 1 byte to travel (e.g. 12ms) Throughput : Requests processed per second (e.g. 50,000 QPS) Bandwidth : Total pipeline capacity (e.g. 10 Gbps)
5. CAP Theorem & PACELC Trade-Offs
In any distributed data store experiencing a network partition (P), you must choose between Consistency (C) or Availability (A). PACELC extends this: if Partitioned (P), choose Availability (A) or Consistency (C); Else (E), choose Latency (L) or Consistency (C).
"When telephone lines are down between bank branches, a branch can either stop allowing withdrawals (Consistency over Availability) or allow withdrawals with the risk of an overdraft (Availability over Consistency)."
[ PACELC THEOREM ]
/ \
(Partition) (Normal / Else)
/ \ / \
[ A ] [ C ] [ L ] [ C ]
DynamoDB Spanner MongoDB Postgres6. Consistent Hashing & Virtual Ring Nodes
Consistent hashing maps both keys and storage nodes to a circular hash ring ($0$ to $2^{32}-1$). When a server is added or removed, only $K/N$ keys need to be remapped on average rather than re-hashing all keys.
"Imagine a circular train track with 4 stations. Passengers (keys) walk forward clockwise until they reach the nearest station (server). Adding a new station only affects passengers between the new station and the previous one."
[ Consistent Hash Ring (0 to 2^32) ]
Node A (Pos 120)
/ \
Key 1 (Pos 310) Key 2 (Pos 950)
/ \
Node C (Pos 2100) ───────── Node B (Pos 1450)7. Microservices vs Modular Monolith Boundaries
A modular monolith structures code into strictly separated domain modules within a single deployment unit. Microservices run these bounded contexts as independent services communicating over network RPC/REST/gRPC.
"A modular monolith is a multi-room house with sturdy soundproof walls and doors. Microservices are a village of separate independent cottages connected by roads."
[ Modular Monolith ] [ Microservices ]
┌───────────────────────┐ ┌─────────┐ ┌─────────┐
│ [Auth] [Billing] [AI] │ │ Auth │──>│ Billing │
│ (In-Memory Calls) │ └────┬────┘ └────┬────┘
└───────────────────────┘ ▼ ▼
(Single Binary) (Network gRPC / HTTP)Subscribe to continue reading the full masterclass
You’ve finished the first 7 concepts. Join 120,000+ senior engineers to unlock the remaining 31 concepts, deep-dive trade-off diagrams, and our 120+ edition archive.