FireCloud, a scalable cloud-based platform for collaborative genome analysis: Strategies for reducing and controlling costs
Chet Birger, Megan Hanna, Edward A. Salinas et al. · bioRxiv (Cold Spring Harbor Laboratory) · 2017
Abstract FireCloud, one of three NCI Cloud Pilots, is a collaborative genome analysis platform built on a cloud computing infrastructure. FireCloud aims to solve the many challenges presented by the increasingly large data sets and computing requirements employed in cancer research. However, cost uncertainty associated with cloud computing’s pay-as-you-go model is proving to be a barrier to adoption of cloud computing. In this paper we present guidelines for optimizing workflows to minimize cost and reduce latency. Our guidelines include: (i) dynamic disk sizing to efficiently utilize virtual disks; (ii) tuned provisioning of virtual machines (VMs) using a performance monitoring tool; (iii) taking advantage of steep price discounts of preemptible VMs; and (iv) utilizing the optimal parallelization of a task’s workload.
Cited by 44 works. doi:10.1101/209494 Open-access PDF
Authors
- Chet Birger
- Megan Hanna
- Edward A. Salinas
- Gordon Saksena
- Dimitri G. Livitz
- Daniel Rosebrock
- Chip Stewart
- Ignaty Leshchiner
- Douglas Voet
- Kristian Cibulskis
- Eric Banks
- Anthony Philippakis
- Gad A. Getz
References
- Sensitive detection of somatic point mutations in impure and heterogeneous cancer samples
- International network of cancer genome projects
- The Sequence Read Archive
- The European Nucleotide Archive
- Toil enables reproducible, open source, big biomedical data analyses
- NCBI’s Database of Genotypes and Phenotypes: dbGaP
- Data analysis: Create a cloud commons
- A cloud-based workflow to quantify transcript-expression levels in public cancer compendia
- Bionimbus: a cloud for managing, analyzing and sharing large genomics datasets
- The dbGaP data browser: a new tool for browsing dbGaP controlled-access genomic data