Availability

The previous chapters have described the underlying technologies such as consistent hashing, ID generator, and now using these techniques we can develop a URL shortener that can generate 100 million URLs per day.

Design consists of the following elements:

• API endpoints
• URL redirecting
• URL shortening
• Data model
• Hash functions: Hash + collision resolution vs Base-62 conversion
• Additional topics to consider:
  • Rate limiter
  • Web server scaling
  • Database scaling
  • Analytics
  • Availability, consistency, and reliability

These items are disclosed in a very interesting Chapter 8 of the book:

A very short Chapter 2 is about how to make rough estimates to start from the most important parts when designing the software.

Some concepts that EVERY software developer should know:

• “Power of two”
• Standard latency numbers! How fast is memory, how slow is disk, etc…
• Availability numbers
• What are key metrics you should think about

These concepts and numbers are disclosed in chapter two of the book:

“System Design Interview – An insider’s guide” by Alex Xu

Earlier the book club of our company has studied excellent book:

Martin Kleppmann - Designing Data-Intensive Applications

This is the best book I have read about building complex scalable software systems. 💪

As usually I prepared an overview and mind-map.

Chapter 9 tells about Consistency and Consensus in distributed systems. It covers the following topics:

• What is consistency and eventual consistency
• Linearizability. Why it is needed. Difference from Serializability. How to implement Linearizability. The cost of Linearizability. CAP theorem.
• Ordering Guarantees. What is ordering and causality. It’s relation to Linearizability. Sequence Number Ordering and how to implement it. Total Order Broadcast and how to implement it.
• Distributed Transactions and Consensus. Why we need Consensus and Distributed Transactions. How to implement them, related problems and software that helps.

Summary:

Earlier the book club of our company has studied excellent book:

Martin Kleppmann - Designing Data-Intensive Applications

This is the best book I have read about building complex scalable software systems. 💪

As usually I prepared an overview and mind-map.

Chapter 5:

• Intro. How to scale apps. Replicating and partitioning.
• Three algos of replicating
• Single-leader Replication
  • Leaders and Followers
  • Sync and async replication
  • Adding new Followers
  • Handling node outages
  • Technical implementations and all potential problems
• Multi-Leader Replication
  • Use-cases when it is good
  • Handling write conflicts
  • Three topologies and potential problems
• Leaderless Replication
  • Writing to the database when a node is down
  • Quorums and problems with them
  • Detecting concurrent writes and how to resolve them

Download full mind map (PDF)