Consistency

The previous chapters have described the underlying technologies such as consistent hashing, ID generator, and now using these techniques we can develop a URL shortener that can generate 100 million URLs per day.

Design consists of the following elements:

• API endpoints
• URL redirecting
• URL shortening
• Data model
• Hash functions: Hash + collision resolution vs Base-62 conversion
• Additional topics to consider:
  • Rate limiter
  • Web server scaling
  • Database scaling
  • Analytics
  • Availability, consistency, and reliability

These items are disclosed in a very interesting Chapter 8 of the book:

Key-Value stores are the most basic but widely used data storages.

Design of key-value store consists of understanding the following topics:

• What do we want from key-value store?
• Single server key-value store
• DISTRIBUTED key-value store:
  • CAP theorem
  • Real-world trade-offs for distributed systems
• System components:
  • Data partition
  • Data replication
  • Consistency
  • Inconsistency resolution: Versioning
  • Handling all types of failures: Failure detection, Handling TEMPORARY failures, Handling PERMANENT failures, Handling data center outage
• System architecture diagram
• Write path
• Read path

These items are disclosed in a very interesting Chapter 6 of the book:

Earlier the book club of our company has studied excellent book:

Martin Kleppmann - Designing Data-Intensive Applications

This is the best book I have read about building complex scalable software systems. 💪

As usually I prepared an overview and mind-map.

Chapter 9 tells about Consistency and Consensus in distributed systems. It covers the following topics:

• What is consistency and eventual consistency
• Linearizability. Why it is needed. Difference from Serializability. How to implement Linearizability. The cost of Linearizability. CAP theorem.
• Ordering Guarantees. What is ordering and causality. It’s relation to Linearizability. Sequence Number Ordering and how to implement it. Total Order Broadcast and how to implement it.
• Distributed Transactions and Consensus. Why we need Consensus and Distributed Transactions. How to implement them, related problems and software that helps.

Summary:

Earlier the book club of our company has studied excellent book:

Martin Kleppmann - Designing Data-Intensive Applications

This is the best book I have read about building complex scalable software systems. 💪

As usually I prepared an overview and mind-map.

Chapter 7 is all your DEV team should know about Transactions:

• The purpose of transactions
• The concept of transaction: ACID, BASE, single-object and multi-object transactions
• Weak Isolation Levels: Read Committed, Snapshot Isolation and Repeatable Read. Problems and preventing them: Lost Updates, Write Skew and Phantoms
• Serializability and three approaches: Actual Serial Execution, Two-Phase Locking (2PL), Serializable Snapshot Isolation (SSI). Potential problems and performance.

Download full mind map (PDF)