Twister
Jaliya Ekanayake, Hui Li, Bingjing Zhang et al. · 2010
MapReduce programming model has simplified the implementation of many data parallel applications. The simplicity of the programming model and the quality of services provided by many implementations of MapReduce attract a lot of enthusiasm among distributed computing communities. From the years of experience in applying MapReduce to various scientific applications we identified a set of extensions to the programming model and improvements to its architecture that will expand the applicability of MapReduce to more classes of applications. In this paper, we present the programming model and the architecture of Twister an enhanced MapReduce runtime that supports iterative MapReduce computations efficiently. We also show performance comparisons of Twister with other similar runtimes such as Hadoop and DryadLINQ for large scale data parallel applications.
Cited by 777 works. doi:10.1145/1851476.1851593
Authors
References
- MapReduce
- Some methods for classification and analysis of multivariate observations
- The anatomy of a large-scale hypertextual Web search engine
- Multidimensional Scaling by Optimizing Goodness of Fit to a Nonmetric Hypothesis
- The Google file system
- An improved algorithm for matching biological sequences
- Dryad
- Map-Reduce for Machine Learning on Multicore
- Nonmetric Individual Differences Multidimensional Scaling: An Alternating Least Squares Method with Optimal Scaling Features
- Evaluating MapReduce for Multi-core and Multiprocessor Systems