SystemML's Optimizer: Plan Generation for Large-Scale Machine Learning Programs.
Matthias Böhm, Douglas Burdick, Alexandre Evfimievski et al. · IEEE Data(base) Engineering Bulletin · 2014
SystemML enables declarative, large-scale machine learning (ML) via a high-level language with R-like syntax. Data scientists use this language to express their ML algorithms with full flexibility but without the need to hand-tune distributed runtime execution plans and system configurations. These ML programs are dynamically compiled and optimized based on data and cluster characteristics using ruleand cost-based optimization techniques. The compiler automatically generates hybrid runtime execution plans ranging from in-memory, single node execution to distributed MapReduce (MR) computation and data access. This paper describes the SystemML optimizer, its compilation chain, and selected optimization phases for generating efficient execution plans.
Cited by 35 works.
Authors
- Matthias Böhm
- Douglas Burdick
- Alexandre Evfimievski
- Berthold Reinwald
- Frederick R. Reiss
- Prithviraj Sen
- Shirish Tatikonda
- Yuanyuan Tian
References
- Compilers: Principles, Techniques, and Tools
- Trust Region Newton Method for Logistic Regression
- SystemML: Declarative machine learning on MapReduce
- Adaptive Query Processing
- The MADlib analytics library
- MLI: An API for Distributed Machine Learning
- Hybrid parallelization strategies for large-scale machine learning in SystemML
- Cumulon
- Scalable and Numerically Stable Descriptive Statistics in SystemML