Archive for 2021

Earlier this year the book club of our company has studied excellent book:

Martin Kleppmann - Designing Data-Intensive Applications

This is the best book I have read about building complex scalable software systems. 💪

As usually I prepared an overview and mind-map.

Chapter 4:

• What is evolvability. Backward and Forward compatibility
• Approaches to encode data:
  • JSON, XML, and their binary variants
  • Thrift and Protobuf
  • Apache Avro
• Models of data flow
  • Through databases
  • Through services: REST, SOAP, RPC and the future
  • Through message brokers - when they are better and when they are not

Much more details in the mind-map:

Earlier this year the book club of our company has studied excellent book:

Martin Kleppmann - Designing Data-Intensive Applications

This is the best book I have read about building complex scalable software systems. 💪

As usually I prepared an overview and mind-map.

Chapter 3:

• Data structures:
  • Log-structured. SSTables / LSM-trees (when we don’t update anything but write to the end). A very cool idea of how to store data.
    • Sorted files.
    • Indexes for each one.
    • Moreover, the indexes can be created not for all the records, because they are sorted, and if the sizes of the records are the same, then the search between two known indexed records is a binary O(log n) search.
    • We always write to the last file.
    • The process of merging files is a school algo: how to write O(n) merge of two sorted arrays into one sorted array.
  • Update-in-place. B trees (when we directly update records). Very smart too.
    • Trees - a simpler idea, but here it is very interesting sub-idea with segments that are tuned for the work of disk drives - and hence the possible problems with SSDs due to many segment rewrites
• Indexing: primary, secondary, multi-column, full-text
  • About indexes - it is obvious that miracles do not happen and additional structures are needed.
  • Interestingly, sometimes values can be stored inside the indexes.
  • Unfortunately, too few info about full-text search - it is interesting to learn more about it.
• OLTP vs OLAP. The clear separation of OLAP / OLTP is very interesting.
• Column-based storage.
  • There is a very interesting aspect about data compression: how many zeros, then how many ones - a kind of compressor. This is possible only in memory - just for one next SSTable

Much more details in the mind-map:

Earlier this year the book club of our company has studied excellent book:

Martin Kleppmann - Designing Data-Intensive Applications

This is the best book I have read about building complex scalable software systems. 💪

As usually I prepared an overview and mind-map.

Chapter 2:

• What is data model.
• Different relations between the data.
• Relational, Document, Graph data models. Which one is better and when.
• Schema-on-write, schema-on-read (schemaless). Data locality.
• Query languages: imperative, declarative, MapReduce. Why NoSQL is reinventing SQL 😀
• Storing graphs. Query languages for graphs: Cypher, SPARQL, Datalog.

Download full mind map (PDF)

Earlier this year the book club of our company has studied excellent book:

Martin Kleppmann - Designing Data-Intensive Applications

This is the best book I have read about building complex scalable software systems. 💪

As usually (to better learn) I prepared an overview and mind-map.

Chapter 1:

• Building blocks of the apps
• What is Reliability, Scalability and Maintainability. Examples and definitions.
  • Faults and Failures
  • Performance, Load, Latency and Response Time
  • Operability, Simplicity, Evolvability
• Why you should randomly kill your servers 😅
• How Twitter delivers 12,000 tweets per second to 300,000 readers per second. (VERY interesting!)
• How much money Amazon loses for each 100ms delay in their response time
• How to quickly calculate percentiles for monitoring response time in PROD

Download full mind map (PDF)

Golang FINALLY introduces GENERICS (aka templates, aka type parameters) in release 1.18 (in Feb 2022)

I remember the early 2000s when generics where added to C#, and how they were awaited…

These days Go is my favourite language for writing highly-scalable solutions and generics are the key thing I’ve been waiting for. They should significantly simplify design of the apps in some cases.

My mind map with key things you should know:

CBINSIGHTS has published this year interesting research The Top 12 Reasons Startups Fail

They analyzed 110+ startup failure post-mortems and defined top 12 root causes.

I think it’s worth knowing reasons and their prior probabilities.

Do you use daily standup meetings?

How standard daily standups are organized

Everyone (one-by-one) is asked about: yesterday activities, today activities, blockers.

But this approach has common problem: people are not always listen to others! Because the next person thinks what to say when it’s their turn.

There is a better model - “Walking the Board”

Discuss every ticket on the board one-by-one. The person who worked on the ticket says a few words, can raise any problems that are immediately addressed.

The book club of our company has chosen a new wonderful book for reading:

Robert Martin - Clean Architecture - a Craftsman’s Guide to Software Structure and Design

Fourth part of the book is about principles of combining components into software systems.

This part is more interesting. It contains:

• Overview of components history: Relocatability, Linkers
• Three principles of Component Cohesion
  • REP: The Reuse/Release Equivalence Principle
  • CCP: The Common Closure Principle
  • CRP: The Common Reuse Principle
• Three principles of Components Coupling
  • ADP: The Acyclic Dependencies Principle
  • SDP: The Stable Dependencies Principle
  • SAP: The Stable Abstractions Principle

I especially enjoyed this chapter because of presented metrics that could be used to measure(!) good software design (more precisely, how do you follow some design principles)

The book club of our company has chosen a new wonderful book for reading:

Robert Martin - Clean Architecture - a Craftsman’s Guide to Software Structure and Design

👍

Third part of the book is about SOLID principles

• Single Responsibility Principle: A module should be responsible to one, and only one, actor.
• Open-Closed Principle: A software artifact should be open for extension but closed for modification
• Liskov Substitution Principle: S is a subtype of T if instead of instance of T we can always use an instance of S
• Interface Segregation Principle: use interfaces to reduce dependency upon changes
• Dependency Inversion Principle: avoid dependencies on volatile concrete elements

I didn’t learn anything new from here (but I am in software engineering for 20+ years already ;). However, this is still a good summarization of key design principles. And it’s worth to remember about them.

When you are starting a new software solution need to select a technology for Frontend. There are currently three leading technologies: Angular, React, and Vue.

But how do you choose from them?

Our team has experience with all of them, but usually the choice is made on the basis “who is available from the team and what they prefer”.

I wanted a deeper Pros and Cons comparison, and I found it in great short Udemy course :

Project Management Institute has published fresh salary survey - 12th edition (2021)

This survey contains salaries of professionals involved into project management activities: from Project Management specialists through three levels of Project Managers to the CEO of an organization.

Many countries were surveyed. And the data can be reviewed in many dimensions, such as years of work experience, educational level, PMP status, gender, industry and many more.

Direct link to the SUMMARY REPORT on PMI website

Today is International Volunteer Day!

Congratulations to all volunteers from all over the world!👋 We make this world a better place. 👍

I am volunteering for Project Management Institute and PMI Moscow since 2010. And I am sure that PMI is the best professional non-commercial organization in the world and it provides the best volunteering opportunities!

My main activities with PMI were:

• Work at PMI Moscow board - mostly IT-related stuff and helping colleagues
• Translation and translation validation of PMI standards, books and PMI exam questions
• PMI exam development (the most mind-blowing tasks) and validation.

During this time, I have met many wonderful people. Colleagues from my country and many colleagues from other countries. And, comparing with other professionals I met during my lifetime, I can say with confidence that the volunteers are the most professional and most active.

Recently Saxo Bank published their outrageous predictions for 2022 (and beyond). I also watched the clarifications of chief investment officer, Steen Jacobsen. Very brave and honest! 👍

Here’s my quick overview of the main points:

• Governments do not have a clear plan for the transition to a green future. So, oil and gas still reign. Meanwhile, many billions will be buried in ESG hype.
• Facebook is for oldies. TikTok (and/or other new social platforms) will kill it.
• US faces with crisis: election, integrity, markets volatility, and declining dollar influence
• 15% inflation in USD, and market expects higher rates soon
• New episode of “Star Wars” is beginning with the latest hypersonic missile tests. Military tech will get huge dollar
• Life expectancy extends by 25 years. This will pose new challenges for pension funds and tax systems, the environment, and many new ethical questions.

Download full slides pack

The book club of our company has chosen a new wonderful book for reading:

Robert Martin - Clean Architecture - a Craftsman’s Guide to Software Structure and Design

👍

The book is superficial so far. Here’s an overview of the second part:

There are three programming paradigms:

• Structured programming - is discipline imposed upon direct transfer of control.
• Object-oriented programming - is discipline imposed upon indirect transfer of control.
• Functional programming - is discipline imposed upon variable assignment

All three paradigms are about what NOT TO DO.

The book club of our company has chosen a new wonderful book for reading:

Robert Martin - Clean Architecture - a Craftsman’s Guide to Software Structure and Design

👍

Here’s an overview of the first part:

• The goal of software architecture
  • to minimize the human resources required to build and maintain the required system.
• Two values of software
  • Behaviour (function) - to satisfy stakeholders’ requirements
  • Structure (architecture) - difficulty of making change should be proportional to the scope, not to the “shape” of the change
• Ease of change is more important!
  • Software Developer should fight 🤺 for it with other stakeholders!

Download full mind map (PDF)

I just faced with the “State of AI ’2021” report and want to share it.

The authors are experts who analyze AI trends, make predictions, and invest(!) in AI. Last year, 5.5 out of 8 their predictions came true. The report has 188(!) pages.

What I can highlight briefly:

• for data scientists & IT managers:
  • Transformers/LLMs expanded far beyond NLP, they will replace recurrent networks
  • JAX framework worth trying
• for investors & founders:
  • AI-first is now everywhere: new researches in biology, skyrocketing new companies for IPOs and new markets
  • Semiconductor-related companies continue to accelerate and consolidate
  • AI influences politics: army, safety & regulations

Here is the link: https://www.stateof.ai/

Spring 2020. Lockdowns. Almost everything is remote. But some banking services 🏦still require physical contact between the bank and customer: opening an account for new customer, a new credit card, etc.

One of our clients - FinTech company that offers innovative banking solutions needed to QUICKLY build software for bank’s field managers - couriers that visit customers to provide services that can’t be provided remotely.

What did they want?

We were asked to develop a system that should optimally plan schedules and routes for hundreds of bank couriers, considering:

There is a known issue in Python - you have to choose between sync and async code models.

And if you are using async code you can call sync code but from that code you CAN’T call async code again.

Why does this problem occur? The event loop used by the async code is already stuck waiting for the result from the sync code. And if you want to call async code now, you cannot reuse the same event loop.

I’ve just read recently published Big Ideas 2021 by ARK invest (these guys are investing billions of USD into disruptive tech companies) and want to share it.

I really recommend to download and overview their slides for everybody who works in high-tech (yes, they are probably over-optimistic because of their role, but it’s really worth to look at): https://drive.google.com/file/d/1z3LpfrAQLRM07JDTi1ZnHEHPeszgIp7O/view?usp=sharing

I am especially excited by what they call “Software 2.0” (deep learning). Here are their key trends - just for your convenience:

Agile Manifesto changed the software development world in 2001.

However it is just a set of principles. This is why it has led to the evolve of many different Agile frameworks that have crystallized and become widespread.

However different frameworks has different focuses:

• Lean/Kanban - teaches how to eliminate waste in delivery pipeline
• Scrum - promotes leadership inside the teams and defines how to deal with constantly appearing changes
• Extreme Programming (XP) - shows how to build the highest quality software
• SAFe - enables large organizations to work as many agile teams connected together
• …

They also have different team structures/roles, different prescribed activities, different slang, etc…