Architecture

Earlier the book club of our company has studied excellent book:

Martin Kleppmann - Designing Data-Intensive Applications

This is the best book I have read about building complex scalable software systems. 💪

As usually I prepared an overview and mind-map.

Chapter 7 is all your DEV team should know about Transactions:

• The purpose of transactions
• The concept of transaction: ACID, BASE, single-object and multi-object transactions
• Weak Isolation Levels: Read Committed, Snapshot Isolation and Repeatable Read. Problems and preventing them: Lost Updates, Write Skew and Phantoms
• Serializability and three approaches: Actual Serial Execution, Two-Phase Locking (2PL), Serializable Snapshot Isolation (SSI). Potential problems and performance.

Download full mind map (PDF)

Earlier the book club of our company has studied excellent book:

Martin Kleppmann - Designing Data-Intensive Applications

This is the best book I have read about building complex scalable software systems. 💪

As usually I prepared an overview and mind-map.

Chapter 6 contains everything the DEV team should consider when designing storage for big data:

• Partition aka Shard aka Region aka Tablet aka vNode aka vBucket. It is another approach for storing the data in addition to Replication (reviewed in the previous chapter)
• How to partition key-value data (primary index). Problems with partitioning - skew and hotspot. Approaches: key range and hash of key.
• Partitioning for secondary indexes: Local index and Global index
• Rebalancing partitions as you grow. Bad and good aproaches, problems and how to deal with them. Manual vs automated rebalancing.
• Request routing. Different aproaches, issues and solutions.

Download full mind map (PDF)

Earlier the book club of our company has studied excellent book:

Martin Kleppmann - Designing Data-Intensive Applications

This is the best book I have read about building complex scalable software systems. 💪

As usually I prepared an overview and mind-map.

Chapter 5:

• Intro. How to scale apps. Replicating and partitioning.
• Three algos of replicating
• Single-leader Replication
  • Leaders and Followers
  • Sync and async replication
  • Adding new Followers
  • Handling node outages
  • Technical implementations and all potential problems
• Multi-Leader Replication
  • Use-cases when it is good
  • Handling write conflicts
  • Three topologies and potential problems
• Leaderless Replication
  • Writing to the database when a node is down
  • Quorums and problems with them
  • Detecting concurrent writes and how to resolve them

Download full mind map (PDF)

Earlier this year the book club of our company has studied excellent book:

Martin Kleppmann - Designing Data-Intensive Applications

This is the best book I have read about building complex scalable software systems. 💪

As usually I prepared an overview and mind-map.

Chapter 4:

• What is evolvability. Backward and Forward compatibility
• Approaches to encode data:
  • JSON, XML, and their binary variants
  • Thrift and Protobuf
  • Apache Avro
• Models of data flow
  • Through databases
  • Through services: REST, SOAP, RPC and the future
  • Through message brokers - when they are better and when they are not

Much more details in the mind-map:

Earlier this year the book club of our company has studied excellent book:

Martin Kleppmann - Designing Data-Intensive Applications

This is the best book I have read about building complex scalable software systems. 💪

As usually I prepared an overview and mind-map.

Chapter 3:

• Data structures:
  • Log-structured. SSTables / LSM-trees (when we don’t update anything but write to the end). A very cool idea of how to store data.
    • Sorted files.
    • Indexes for each one.
    • Moreover, the indexes can be created not for all the records, because they are sorted, and if the sizes of the records are the same, then the search between two known indexed records is a binary O(log n) search.
    • We always write to the last file.
    • The process of merging files is a school algo: how to write O(n) merge of two sorted arrays into one sorted array.
  • Update-in-place. B trees (when we directly update records). Very smart too.
    • Trees - a simpler idea, but here it is very interesting sub-idea with segments that are tuned for the work of disk drives - and hence the possible problems with SSDs due to many segment rewrites
• Indexing: primary, secondary, multi-column, full-text
  • About indexes - it is obvious that miracles do not happen and additional structures are needed.
  • Interestingly, sometimes values can be stored inside the indexes.
  • Unfortunately, too few info about full-text search - it is interesting to learn more about it.
• OLTP vs OLAP. The clear separation of OLAP / OLTP is very interesting.
• Column-based storage.
  • There is a very interesting aspect about data compression: how many zeros, then how many ones - a kind of compressor. This is possible only in memory - just for one next SSTable

Much more details in the mind-map:

Earlier this year the book club of our company has studied excellent book:

Martin Kleppmann - Designing Data-Intensive Applications

This is the best book I have read about building complex scalable software systems. 💪

As usually I prepared an overview and mind-map.

Chapter 2:

• What is data model.
• Different relations between the data.
• Relational, Document, Graph data models. Which one is better and when.
• Schema-on-write, schema-on-read (schemaless). Data locality.
• Query languages: imperative, declarative, MapReduce. Why NoSQL is reinventing SQL 😀
• Storing graphs. Query languages for graphs: Cypher, SPARQL, Datalog.

Download full mind map (PDF)

Earlier this year the book club of our company has studied excellent book:

Martin Kleppmann - Designing Data-Intensive Applications

This is the best book I have read about building complex scalable software systems. 💪

As usually (to better learn) I prepared an overview and mind-map.

Chapter 1:

• Building blocks of the apps
• What is Reliability, Scalability and Maintainability. Examples and definitions.
  • Faults and Failures
  • Performance, Load, Latency and Response Time
  • Operability, Simplicity, Evolvability
• Why you should randomly kill your servers 😅
• How Twitter delivers 12,000 tweets per second to 300,000 readers per second. (VERY interesting!)
• How much money Amazon loses for each 100ms delay in their response time
• How to quickly calculate percentiles for monitoring response time in PROD

Download full mind map (PDF)

The book club of our company has chosen a new wonderful book for reading:

Robert Martin - Clean Architecture - a Craftsman’s Guide to Software Structure and Design

Fourth part of the book is about principles of combining components into software systems.

This part is more interesting. It contains:

• Overview of components history: Relocatability, Linkers
• Three principles of Component Cohesion
  • REP: The Reuse/Release Equivalence Principle
  • CCP: The Common Closure Principle
  • CRP: The Common Reuse Principle
• Three principles of Components Coupling
  • ADP: The Acyclic Dependencies Principle
  • SDP: The Stable Dependencies Principle
  • SAP: The Stable Abstractions Principle

I especially enjoyed this chapter because of presented metrics that could be used to measure(!) good software design (more precisely, how do you follow some design principles)

The book club of our company has chosen a new wonderful book for reading:

Robert Martin - Clean Architecture - a Craftsman’s Guide to Software Structure and Design

👍

Third part of the book is about SOLID principles

• Single Responsibility Principle: A module should be responsible to one, and only one, actor.
• Open-Closed Principle: A software artifact should be open for extension but closed for modification
• Liskov Substitution Principle: S is a subtype of T if instead of instance of T we can always use an instance of S
• Interface Segregation Principle: use interfaces to reduce dependency upon changes
• Dependency Inversion Principle: avoid dependencies on volatile concrete elements

I didn’t learn anything new from here (but I am in software engineering for 20+ years already ;). However, this is still a good summarization of key design principles. And it’s worth to remember about them.

When you are starting a new software solution need to select a technology for Frontend. There are currently three leading technologies: Angular, React, and Vue.

But how do you choose from them?

Our team has experience with all of them, but usually the choice is made on the basis “who is available from the team and what they prefer”.

I wanted a deeper Pros and Cons comparison, and I found it in great short Udemy course :