Airflow

There are a number of businesses and business tasks when it is necessary to visually design, change and control the execution of business processes.

Large companies like SAP, IBM, Oracle play in this market and offer their heavy and expensive solutions.

But there is also a lighter and cheaper solution in this field.

Earlier the book club of our company has studied excellent book: Martin Kleppmann - Designing Data-Intensive Applications This is the best book I have read about building complex scalable software systems. 💪 As usually I prepared an overview and mind-map. Chapter 10 discovers all aspects about big data Batch Processing. If your system needs to process some data then your DEV team should learn this info. Unix tools for batch processing and brilliant concept of pipes.