The previous chapters have described the underlying technologies such as consistent hashing, ID generator, and now using these techniques we can develop a URL shortener that can generate 100 million URLs per day.
Design consists of the following elements:
API endpoints URL redirecting URL shortening Data model Hash functions: Hash + collision resolution vs Base-62 conversion Additional topics to consider: Rate limiter Web server scaling Database scaling Analytics Availability, consistency, and reliability These items are disclosed in a very interesting Chapter 8 of the book:
Key-Value stores are the most basic but widely used data storages.
Design of key-value store consists of understanding the following topics:
What do we want from key-value store? Single server key-value store DISTRIBUTED key-value store: CAP theorem Real-world trade-offs for distributed systems System components: Data partition Data replication Consistency Inconsistency resolution: Versioning Handling all types of failures: Failure detection, Handling TEMPORARY failures, Handling PERMANENT failures, Handling data center outage System architecture diagram Write path Read path These items are disclosed in a very interesting Chapter 6 of the book:
Consistent Hashing is a cornerstone technology for distributed systems. Many of software developers don’t realize it, but Consistent Hashing is needed in many places: load balancers, caches, CDNs, id generators, databases, chats / social networks, and many other systems.
This topic consists of:
Problem with rehashing and why we need hashing to be CONSISTENT Hash space and hash ring BASIC approach (introduced by Karger et al. at MIT) Advanced approach with VIRTUAL NODES These items are disclosed in a very interesting Chapter 5 of the book:
