System Design 101: What It Actually Means

If you have spent any time preparing for a backend engineering interview in the last decade, you have encountered system design. It usually arrives in one of two forms: a whiteboard prompt ("design Twitter"), or a thousand-page PDF someone circulates on Discord. Both give the impression that system design is a discrete body of knowledge you can memorise.

It is not. System design is the skill of making architectural decisions under real constraints with incomplete information, and the only way to get good at it is to build, read post-mortems, and watch systems fail in ways you did not predict. This article covers what "system design" actually refers to, the vocabulary you need to be fluent in, and how to practice it without a FAANG offer dangling in front of you.

Two different things called system design

The traditional meaning

In systems engineering and in university curricula before 2015, "system design" meant the discipline of designing any complex engineered system — hardware, software, or both — with formal requirements, component decomposition, interface specifications, and verification plans. This is still what the term means in aerospace, automotive, and defence. If you read a textbook called Systems Engineering Principles and Practice, this is the subject.

The modern interview-prep meaning

Since roughly 2015, "system design" in a software context has come to mean something narrower: how to architect a backend system that serves a lot of users. How to shard a database. When to add a cache. How to choose between SQL and NoSQL. How to design an API. How to scale horizontally.

This second meaning is what most people mean when they say system design today, and it is what the rest of this article covers. Both meanings are useful. The first is deeper and more durable. The second is what gets you a job.

The core questions

Every system design problem reduces to a small set of questions. If you can answer these crisply for a given system, you have designed it.

1. What is the data?

What entities does the system store? What are their fields, their relationships, their typical sizes? A blog post is a few KB; a video is a few GB. That 10,000x difference drives every other decision.

2. How much of it?

How many users, how many records per user, how many new records per second? Back-of-the-envelope numbers here matter more than precision. A system with 1,000 users is a completely different shape from one with 100 million. Get an order of magnitude early.

3. How do you read it?

Is the workload read-heavy (news feed, product catalogue) or write-heavy (analytics, logging)? What is the read-to-write ratio? Is a stale read acceptable, or does every read need to reflect the most recent write? The answer determines your caching strategy, your consistency model, your database choice.

4. Where does it live?

Single database, sharded databases, document store, key-value store, object storage, search index? Most serious systems use several in combination, with each piece of data in the store best suited to how it is accessed.

5. Who is upstream and downstream?

What does the client look like? What other services call this one? What SLAs do they expect? A system called by millions of mobile phones over flaky 4G has different requirements from a system called by a handful of internal batch jobs.

6. What breaks, and what happens when it does?

Every component fails eventually. A disk fills up. A region goes dark. A deployment breaks things. The quality of a design is largely about how gracefully it fails. Every answer above should be re-examined with the question "what happens when this dies?"

A minimal vocabulary

You need to be fluent in these terms before system design conversations make sense. This is not the full list, but it is the subset you will hit in the first hour of any design discussion.

Vertical vs horizontal scaling

Vertical scaling means making one machine bigger (more CPU, more RAM). Horizontal scaling means adding more machines. Vertical scaling has a ceiling; horizontal scaling has coordination costs. Most modern systems do both.

Sharding

Partitioning data across multiple databases, typically by a shard key (user ID, tenant ID, geographic region). Each shard holds a slice of the data and serves a slice of the traffic. Sharding solves scale at the cost of making some operations (cross-shard joins, transactions) much harder.

Replication

Keeping multiple copies of the same data. Primary-replica replication gives you read scaling and a failover target. Multi-primary replication gives you write scaling but introduces conflict resolution.

Consistency models

Strong consistency: every read sees the most recent write. Eventual consistency: reads may be stale for a window, but converge. Causal consistency: reads see writes in the order they causally depend on each other. Different systems make different tradeoffs; see The CAP Theorem Explained.

Load balancing

Distributing incoming requests across multiple servers. Round-robin, least-connections, consistent hashing, and various weighted variants. Modern service meshes (Envoy, Istio) give you this out of the box.

Caching

Storing frequently accessed data closer to the consumer (in-memory, in a CDN, in the browser) to reduce latency and load on the underlying store. See Caching Strategies Every Backend Developer Should Know for patterns.

Queues and event streams

Kafka, RabbitMQ, SQS, Kinesis. Decoupling producers from consumers, buffering load spikes, and enabling asynchronous workflows. The backbone of almost every non-trivial modern architecture.

The default shape of a modern backend. Almost every design discussion starts here and argues about which pieces to add or remove.

Architecture vs system design

These terms overlap but are not identical. Software architecture tends to refer to the structure of a single codebase or deployable unit — the layers, the module boundaries, the patterns within. System design tends to refer to the shape of multiple services communicating over a network.

In practice most senior engineers do both, and the terms are used interchangeably. The distinction matters mainly when you are organising what to learn: "clean architecture" and "hexagonal architecture" are about the inside of one service; "microservices" and "event-driven architecture" are about the shape of a distributed system.

How to practice without a big-tech offer

The usual advice is "design Twitter on a whiteboard". This is not wrong but it is lonely. Better ways to build the skill:

Build something small but real

Build a URL shortener. Add analytics. Add a cache. Add a replica. Break it on purpose. Watch the logs. Go read why it broke. This teaches more in a weekend than a month of videos.

Read post-mortems

GitHub publishes their outage reports. Cloudflare publishes theirs. Stripe publishes theirs. Every good post-mortem is a free system-design lesson from engineers who have real scars. The Google SRE book distils decades of this into 500 pages.

Work on systems that are already scaled

Joining a company with real traffic teaches you things that side projects cannot. A production database with a billion rows behaves differently from one with ten thousand.

Read the classic papers

The Dynamo paper. The MapReduce paper. The Bigtable paper. The Spanner paper. These are the foundations most modern systems are built on, and they are more readable than their reputation suggests.

Frequently Asked Questions

Do I need a computer science degree to learn system design?

No. Most of the skills are practical and build on software engineering experience rather than theory. Knowing data structures and big-O notation helps, but the core of system design is reasoning about tradeoffs, and that is learned by doing.

What is the best book on system design?

Designing Data-Intensive Applications by Martin Kleppmann is the consensus pick. It is dense but honest, and it covers the fundamentals (storage engines, replication, consistency, batch and stream processing) without shortcuts. After that, Site Reliability Engineering from Google and The Site Reliability Workbook fill in operational depth.

How do I prepare for a system design interview?

Learn the vocabulary above. Pick five common prompts (design Twitter, Uber, WhatsApp, a rate limiter, a URL shortener) and work through each one honestly — not just memorising the answer, but understanding why each component is there. Practise whiteboarding with a friend or mirror. The interview is less about knowing the right answer and more about demonstrating the thought process.

Is system design harder than coding interviews?

It is more open-ended and less about pattern recognition, which some engineers find harder and some find easier. The people who struggle with system design interviews usually struggle because they have never worked on systems at the scale the interview presumes. Build something, break it, and the confidence follows.