🎁 Free System Design Interview Playbook included with newsletter membership.

Issue #12014 min read

#Architecture#Scalability#Distributed Systems#Cheat Sheet

System Design Concepts: Scalability, Availability, Reliability, and 35 Others

Part 1: The foundational and advanced building blocks every software engineer must master to build planet-scale systems.

Neo Kim✓

Staff Infrastructure Architect & Creator of SystemDesign.one

Published on Aug 14, 2026

Index of System Design Concepts in this Issue (13)

1. ScalabilityFREE 2. AvailabilityFREE 3. ReliabilityFREE 4. Latency vs Throughput vs BandwidthFREE 5. Client-Server Architecture🔒 LOCK 6. SQL vs NoSQL Databases🔒 LOCK 7. Load Balancing & Algorithms🔒 LOCK 8. Caching & Cache Invalidation Strategies🔒 LOCK 9. Content Delivery Network (CDN)🔒 LOCK 10. Rate Limiting Algorithms🔒 LOCK 11. CAP Theorem & PACELC🔒 LOCK 12. Database Sharding & Consistent Hashing🔒 LOCK 13. Message Queues & Event-Driven Architecture (Pub/Sub)🔒 LOCK

Concept #1 · Foundations

✓ Free Preview

1. Scalability

Scalability is a system’s ability to handle growing amounts of work or traffic by adding resources without compromising performance or stability. Vertical scaling (scaling up) means upgrading single-server hardware (CPU, RAM, NVMe). Horizontal scaling (scaling out) means adding more commodity server instances behind a load distributor.

Architecture Topology


  [ Vertical Scaling ]        [ Horizontal Scaling ]
  ┌──────────────────┐        ┌────────┐ ┌────────┐ ┌────────┐
  │  Server (128GB)  │        │ Server │ │ Server │ │ Server │
  │    (64 Cores)    │        │  (8GB) │ │  (8GB) │ │  (8GB) │
  └──────────────────┘        └────────┘ └────────┘ └────────┘
     (Hits Ceiling)             ▲          ▲          ▲
                                └───── Load Balancer ─┘

💡 Real-World Mental Model

"Vertical scaling is like upgrading from a small restaurant kitchen to a massive commercial kitchen with industrial ovens. Horizontal scaling is like opening 5 new restaurant branches in different neighborhoods to serve more customers concurrently."

⚖️ Trade-offs & Costs

Vertical scaling has physical hardware limits and introduces a single point of failure (downtime during upgrades). Horizontal scaling is virtually limitless and fault-tolerant, but introduces distributed consensus, network latency, and operational complexity.

🎯 Why It Matters at Scale

Traffic spikes are unpredictable. Systems that scale horizontally can dynamically autoscale up during Black Friday and scale down during off-peak hours to minimize cloud infrastructure costs.

Concept #2 · Foundations

✓ Free Preview

2. Availability

Availability measures the percentage of time a system remains operational and accessible to process requests over a given period. It is commonly expressed in "nines" (e.g. 99.9% = "three nines" allows 8.76 hours of downtime/year; 99.999% = "five nines" allows only 5.26 minutes of downtime/year).

Architecture Topology


  Three Nines (99.9%)   -> ~8h 45m downtime / year
  Four Nines (99.99%)   -> ~52m 35s downtime / year
  Five Nines (99.999%)  -> ~5m 15s downtime / year (Target for Financial Systems)

💡 Real-World Mental Model

"Availability is like a 24/7 convenience store. If the front automatic door breaks, a high-availability store has a second backup manual door so customers can still enter without interruption."

⚖️ Trade-offs & Costs

Achieving 99.999% availability requires multi-region redundancy, automated failovers, continuous health-checks, and active-active replicas, which exponentially increases infrastructure and engineering costs.

🎯 Why It Matters at Scale

Every second of unplanned downtime for mission-critical platforms (e.g. Stripe, AWS, NASDAQ) directly translates to millions of dollars in lost transactions and brand reputation erosion.

Concept #3 · Foundations

✓ Free Preview

3. Reliability

Reliability is the probability that a system performs its required function correctly without error under specified operating conditions for a given duration. While availability is about uptime, reliability is about correctness and consistency of outputs.

Architecture Topology


  [Client] ──(Idempotency Key: req_981)──> [Payment Gateway]
                                                    │ (Network timeout?)
  [Client] ──(Retry with SAME key)───────> [Payment Gateway]
                                                    │
                                         [Check Cache / DB]
                                         -> Already charged! Return cached receipt.

💡 Real-World Mental Model

"A car might turn on 100% of the time (100% available), but if the brakes fail 5% of the time while driving, it is severely unreliable."

⚖️ Trade-offs & Costs

High reliability requires idempotent APIs, defensive retry strategies with exponential backoff, circuit breakers, and end-to-end integration testing.

🎯 Why It Matters at Scale

A system that is available but constantly returns corrupted user data or HTTP 500 error responses is useless to end-users.

Concept #4 · Networking & Performance

✓ Free Preview

4. Latency vs Throughput vs Bandwidth

Latency is the time delay for a single packet/request to travel from source to destination (measured in ms). Throughput is the number of successful operations completed per unit of time (measured in RPS or QPS). Bandwidth is the maximum theoretical capacity of the communication link (measured in Gbps).

Architecture Topology


  Bandwidth : [============ Highway Width ============] (Capacity)
  Latency   : [Car ───> 120km/h ───> Toll Gate]         (Time elapsed)
  Throughput: [Car][Car][Car][Car] / minute             (Total volume)

💡 Real-World Mental Model

"Think of a highway: Bandwidth is the number of lanes. Latency is the speed limit (how fast one car reaches the toll booth). Throughput is the total count of cars passing the toll booth per minute."

⚖️ Trade-offs & Costs

Optimizing for ultra-low latency often requires memory-caching and expensive localized edge computing, whereas maximizing throughput involves batching and streaming queues that can slightly increase per-request latency.

🎯 Why It Matters at Scale

Video streaming demands high bandwidth and high throughput, while high-frequency trading and gaming demand ultra-low single-digit millisecond latency.

🔒

Free Preview Ended

Unlock the Remaining 9 System Design Concepts

You've explored the foundational concepts. Subscribe to access the rest of Issue #120, including CAP Theorem, Consistent Hashing, Sharding, Message Queues, Rate Limiting, and our entire archive of 120+ engineering case studies.

✓34 Advanced Concepts with Tradeoffs

✓Full System Architecture ASCII Diagrams

✓System Design Interview Playbook PDF

Looking for full annual benefits?

* Note: Full standalone purchase of individual articles is currently disabled. Access is provided exclusively via active newsletter subscription.

Written by Neo Kim

Staff Engineer sharing pragmatic software architecture patterns and visual blueprints for distributed scale.