Performance analysis of concurrent workflows

• Published in: Journal of big data Vol. 2; no. 1; pp. 1 - 14
• Main Author: Kempa-Liehr, Andreas
• Format: Journal Article
• Language: English
• Published: Cham Springer International Publishing 03.07.2015; Springer Nature B.V
• Subjects: Automation; Big Data; Communications Engineering; Computational Science and Engineering; Computer Science; Continuous improvement; Data management; Data Mining and Knowledge Discovery; Database Management; Graph theory; Graphical representations; Information Storage and Retrieval; Management systems; Mathematical Applications in Computer Science; Networks; Pipelines; Resource management; Visualization; Workflow; Communicating sequential processes; Big data pipeline; High performance computing; Complex event processing; Continuous improvement process
• ISSN: 2196-1115; 2196-1115
• DOI: 10.1186/s40537-015-0017-0

Automated workflows are the key concept of big data pipelines in science, engineering and enterprise applications. The performance analysis of automated workflows is an important topic of the continuous improvement process and the foundation of designing new workflows. This paper introduces the concept of process evolution functions and event reduction policies, which allow for the time resolved visualization of an unlimited number of concurrent workflows by means of aggregated task views. The visualization allows for an intuitive approach to the performance analysis of concurrent workflows. The theoretical foundation of this approach is applicable for workflows represented by directed acyclic graphs. It is explained on the basis of a simple IO-workflow model, which is typically found for distributed resource management systems utilized for many-task computing. AMS subject classification 68Mxx