Cassandra

Earlier the book club of our company has studied excellent book:

Martin Kleppmann - Designing Data-Intensive Applications

This is the best book I have read about building complex scalable software systems. 💪

As usually I prepared an overview and mind-map.

Chapter 8 discovers non-database related problems of distributed systems. DEV teams should consider them when designing distributed software.

• Faults and Partial Failures. The need to build a reliable system from unreliable components.
• Problem 1: Unreliable Networks. Why they happen. How to detect them. How to deal with them. Synchronous and Asynchronous networks. Hybrid networks and emulating hybrid networks.
• Problem 2: Unreliable Clocks. Monotonic Clocks vs Time-of-Day Clocks. What to use and what not to use in different cases. Good practices.
• Problem 3: Knowledge, Truth, and Lies. What is Truth in distributed systems. The Byzantine Generals Problem. System Model and Reality: what to use for what cases.

Download full mind map (PDF)

Earlier this year the book club of our company has studied excellent book:

Martin Kleppmann - Designing Data-Intensive Applications

This is the best book I have read about building complex scalable software systems. 💪

As usually I prepared an overview and mind-map.

Chapter 3:

• Data structures:
  • Log-structured. SSTables / LSM-trees (when we don’t update anything but write to the end). A very cool idea of how to store data.
    • Sorted files.
    • Indexes for each one.
    • Moreover, the indexes can be created not for all the records, because they are sorted, and if the sizes of the records are the same, then the search between two known indexed records is a binary O(log n) search.
    • We always write to the last file.
    • The process of merging files is a school algo: how to write O(n) merge of two sorted arrays into one sorted array.
  • Update-in-place. B trees (when we directly update records). Very smart too.
    • Trees - a simpler idea, but here it is very interesting sub-idea with segments that are tuned for the work of disk drives - and hence the possible problems with SSDs due to many segment rewrites
• Indexing: primary, secondary, multi-column, full-text
  • About indexes - it is obvious that miracles do not happen and additional structures are needed.
  • Interestingly, sometimes values can be stored inside the indexes.
  • Unfortunately, too few info about full-text search - it is interesting to learn more about it.
• OLTP vs OLAP. The clear separation of OLAP / OLTP is very interesting.
• Column-based storage.
  • There is a very interesting aspect about data compression: how many zeros, then how many ones - a kind of compressor. This is possible only in memory - just for one next SSTable

Much more details in the mind-map:

🚆✈️🚌🛳 + Math + Highload + Machine Learning…

Two and a half years ago we’ve got a request from the Travel-business IT company, whose B2C IT solution is the basis for one of the world’s largest travel companies that serves over 1 billion passengers every year.

What did they want to achieve?

They wanted to quickly design and implement an “internal startup” - an innovative B2B/B2C solution for the travel industry.