Reproducible, scalable, and shareable analysis pipelines with bioinformatics workflow managers

• Published in: Nature methods Vol. 18; no. 10; pp. 1161 - 1168
• Main Authors: Wratten, Laura, Wilm, Andreas, Göke, Jonathan
• Format: Journal Article
• Language: English
• Published: New York Nature Publishing Group US 01.10.2021; Nature Publishing Group
• Subjects: 631/114/1751; 631/114/2398; 631/114/794; 631/1647/794; Bioinformatics; Biological Microscopy; Biological Techniques; Biomedical and Life Sciences; Biomedical Engineering/Biotechnology; Biomedical research; Biomedical Research - methods; Biomedical Research - standards; Complexity; Computational biology; Computational Biology - methods; Computer applications; Life Sciences; Medical research; Medicine, Experimental; Methods; Optimization; Perspective; Pipelines; Proteomics; Reproducibility; Reproducibility of Results; Technology application; Workflow; Singapore
• ISSN: 1548-7091; 1548-7105; 1548-7105
• DOI: 10.1038/s41592-021-01254-9

The rapid growth of high-throughput technologies has transformed biomedical research. With the increasing amount and complexity of data, scalability and reproducibility have become essential not just for experiments, but also for computational analysis. However, transforming data into information involves running a large number of tools, optimizing parameters, and integrating dynamically changing reference data. Workflow managers were developed in response to such challenges. They simplify pipeline development, optimize resource usage, handle software installation and versions, and run on different compute platforms, enabling workflow portability and sharing. In this Perspective, we highlight key features of workflow managers, compare commonly used approaches for bioinformatics workflows, and provide a guide for computational and noncomputational users. We outline community-curated pipeline initiatives that enable novice and experienced users to perform complex, best-practice analyses without having to manually assemble workflows. In sum, we illustrate how workflow managers contribute to making computational analysis in biomedical research shareable, scalable, and reproducible. This Perspective highlights workflow managers, which are useful for developing and managing complex bioinformatics pipelines.