Rethinking Data-Intensive Science Using Scalable Analytics Systems
Frank Austin Nothaft, Matt Massie, Timothy Danford et al. · Proceedings - ACM-SIGMOD International Conference on Management of Data · 2015
"Next generation" data acquisition technologies are allowing scientists to collect exponentially more data at a lower cost. These trends are broadly impacting many scientific fields, including genomics, astronomy, and neuroscience. We can attack the problem caused by exponential data growth by applying horizontally scalable techniques from current analytics systems to accelerate scientific processing pipelines.
Cited by 94 works. doi:10.1145/2723372.2742787 Open-access PDF
Authors
- Frank Austin Nothaft
- Timothy Danford
- Uri Laserson
- Carl Yeksigian
- Jey Kottalam
- Arun Ahuja
- Jeff Hammerbacher
- Michael D. Linderman
- Michael J. Franklin
- Anthony D. Joseph
- David A. Patterson
References
- The Sequence Alignment/Map format and SAMtools
- Scikit-learn: Machine Learning in Python
- Fast and accurate short read alignment with Burrows–Wheeler transform
- MapReduce
- Initial sequencing and analysis of the human genome
- The Genome Analysis Toolkit: A MapReduce framework for analyzing next-generation DNA sequencing data
- End-To-End Arguments In System Design
- Fast and accurate long-read alignment with Burrows–Wheeler transform
- A framework for variation discovery and genotyping using next-generation DNA sequencing data
- The Sloan Digital Sky Survey: Technical Summary