Automating your routine data analysis with workflow management systems

Abstract

With the needs of pipeline automation for omics data analysis, many workflow management systems have been developed to handle such big data. These systems also help to build portable pipelines that can be executed in different computing environments. This portability feature allows us to share and reuse pipelines easily. Using a workflow management system will ease the pain to construct and execute your pipelines, since it provides simple workflow definitions, organizes your files, and records all the provenance of every task run. ? ??? In our research center, we have built our self-developed workflow system. This system currently supports our routine omics data processing, and it will eventually be published as an open-source project. You are more than welcome to try the alpha version. It can fit many computing environments including a high-performance computing cluster with the PBS job scheduler. Let’s accelerate our pipeline constructions and data analysis! ? ??? In this talk, I will first introduce the current workflow management systems. Second, I will also share our homemade workflow management system in Chang Gung Molecular Medicine Research Center. Last, I will present a miRNA-seq pipeline as a demonstration.