PDF

Consistent Hashing is a cornerstone technology for distributed systems. Many of software developers don’t realize it, but Consistent Hashing is needed in many places: load balancers, caches, CDNs, id generators, databases, chats / social networks, and many other systems.

This topic consists of:

• Problem with rehashing and why we need hashing to be CONSISTENT
• Hash space and hash ring
• BASIC approach (introduced by Karger et al. at MIT)
• Advanced approach with VIRTUAL NODES

These items are disclosed in a very interesting Chapter 5 of the book:

Every popular software should have a Rate Limiter. It prevents DDOS attack, reduces cost and prevents servers from being overloaded.

There are some tricky questions to be considered during implementation of Rate Limiter:

• Where to put Rate Limiter: client-side, server-side, gateway?
• Algorithms for rate limiting. There are many algorithms with pros and cons: Token bucket, Leaking bucket, Fixed window counter, Sliding window log, Sliding window counter. Your business needs will define the right algorithm.
• How are rate limiting rules created?
• Where are the rules stored?
• How to handle requests that are rate limited?

These questions are disclosed in a very interesting Chapter 4 of the book:

Four standard steps for system design interview. However, I would think about them wider: as about four initial steps to design the software.

• Step 1. Understand the problem and establish design scope
• Step 2. Propose high-level design and get buy-in
• Step 3. Design deep dive
• Step 4. Wrap up

The chapter 3 of the book discovers details about each step, good questions to ask (to think about), DO’s and DONT’s. It also shows good example of the process of designing a news feed system.

A very short Chapter 2 is about how to make rough estimates to start from the most important parts when designing the software.

Some concepts that EVERY software developer should know:

• “Power of two”
• Standard latency numbers! How fast is memory, how slow is disk, etc…
• Availability numbers
• What are key metrics you should think about

These concepts and numbers are disclosed in chapter two of the book:

“System Design Interview – An insider’s guide” by Alex Xu

A great generic plan for scaling any app from zero to millions of users.

• Single server setup
• Selection and usage of database
• Vertical scaling vs horizontal scaling approaches. And why you should prefer horizontal
• Adding load balancer for horizontal scaling
• Adding database replication for horizontal scaling
• Adding cache
• Adding CDN
• Stateless vs Stateful architecture and using external state storage
• Adding extra Data Centers
• Adding Message queue
• Adding Logging, Metrics, and Automation
• Scaling database (sharding)
• and futher steps…

All of these is carefully but briefly disclosed in the Chapter 1 of the book:

Principles of Microservice Deployment Differentes between Continuous integration (CI), Continuous delivery and Continuous deployment. Deployment patterns:

• The Basic Full-Stop Deployment Pattern
• The Rolling Update Pattern
• The Breaking Schema Change Pattern: two options here - Eventual Migration and Synchronized Migration
• The Blue-Green Deployment Pattern

All of these is disclosed in the Chapter 16 of the book:

“Building Event-Driven Microservices: Leveraging Organizational Data at Scale” by Adam Bellemare

Sharing my mind map with all the details as usual:

Testing event-driven solutions is a challenging task. It includes:

• Common principles
• Unit testing: stateless and stateful
• Testing the Topology (I bet you didn’t do it! ;)
• Testing Schema Evolution and Compatibility
• Integration testing: Local, Remote and Hybrid - a set of common sense considerations not to forgot about
• Choosing the testing strategy

All of these is disclosed in the Chapter 15 of the book:

“Building Event-Driven Microservices: Leveraging Organizational Data at Scale” by Adam Bellemare

Quite often the tooling is a forgotten area during the beginning of development. This chapter overlooks the tooling that can help build and maintain event-driven microservices. Useful checklist! This topic includes:

• Org rules
• Documentation, tagging
• Quotas
• Schema registry
• Offset management
• ACL
• State management and app reset
• Consumer offset lag monitoring
• Container Management Controls
• Cluster Creation and Management
• Dependency Tracking

All of these is disclosed in the Chapter 14 of the book:

How to integrate event-driven microservices with request-response APIs?

There are two types of external events:

• Autonomously Generated Events (analytical events)
• Reactively Generated Events (events from request-reply)

There are two approaches of processing and serving requests using stateful services:

• using internal state stores (with silly or with smart routing)
• using external state stores (with regular or with composite microservice)

Ways of handling requests within an event-driven workflow:

• Handling requests directly
• Turning requests into events (big latency)
• Processing events for user interfaces (a mix of two ways)

Approaches for request-response applications, their pros and cons:

The 4th pattern to build microservices is to use Lightweight Frameworks.

Lightweight Frameworks provide similar functionality to Heavyweight Frameworks, but they heavily rely on:

• the event broker
• the container management system (CMS)

In many cases they exceed Heavyweight Frameworks.

Different apps can use any/different resources from the cluster which are better fit their needs. While still provide Scaling and Recovering from Failures (again by heavily relying on event broker and CMS).