Load Balancer

Consistent Hashing is a cornerstone technology for distributed systems. Many of software developers don’t realize it, but Consistent Hashing is needed in many places: load balancers, caches, CDNs, id generators, databases, chats / social networks, and many other systems.

This topic consists of:

• Problem with rehashing and why we need hashing to be CONSISTENT
• Hash space and hash ring
• BASIC approach (introduced by Karger et al. at MIT)
• Advanced approach with VIRTUAL NODES

These items are disclosed in a very interesting Chapter 5 of the book:

Four standard steps for system design interview. However, I would think about them wider: as about four initial steps to design the software.

• Step 1. Understand the problem and establish design scope
• Step 2. Propose high-level design and get buy-in
• Step 3. Design deep dive
• Step 4. Wrap up

The chapter 3 of the book discovers details about each step, good questions to ask (to think about), DO’s and DONT’s. It also shows good example of the process of designing a news feed system.

A great generic plan for scaling any app from zero to millions of users.

• Single server setup
• Selection and usage of database
• Vertical scaling vs horizontal scaling approaches. And why you should prefer horizontal
• Adding load balancer for horizontal scaling
• Adding database replication for horizontal scaling
• Adding cache
• Adding CDN
• Stateless vs Stateful architecture and using external state storage
• Adding extra Data Centers
• Adding Message queue
• Adding Logging, Metrics, and Automation
• Scaling database (sharding)
• and futher steps…

All of these is carefully but briefly disclosed in the Chapter 1 of the book:

How to integrate event-driven microservices with request-response APIs?

There are two types of external events:

• Autonomously Generated Events (analytical events)
• Reactively Generated Events (events from request-reply)

There are two approaches of processing and serving requests using stateful services:

• using internal state stores (with silly or with smart routing)
• using external state stores (with regular or with composite microservice)

Ways of handling requests within an event-driven workflow:

• Handling requests directly
• Turning requests into events (big latency)
• Processing events for user interfaces (a mix of two ways)

Approaches for request-response applications, their pros and cons:

Earlier the book club of our company has studied excellent book:

Martin Kleppmann - Designing Data-Intensive Applications

This is the best book I have read about building complex scalable software systems. 💪

As usually I prepared an overview and mind-map.

Chapter 6 contains everything the DEV team should consider when designing storage for big data:

• Partition aka Shard aka Region aka Tablet aka vNode aka vBucket. It is another approach for storing the data in addition to Replication (reviewed in the previous chapter)
• How to partition key-value data (primary index). Problems with partitioning - skew and hotspot. Approaches: key range and hash of key.
• Partitioning for secondary indexes: Local index and Global index
• Rebalancing partitions as you grow. Bad and good aproaches, problems and how to deal with them. Manual vs automated rebalancing.
• Request routing. Different aproaches, issues and solutions.

Download full mind map (PDF)