SystemML's Optimizer: Plan Generation for Large-Scale Machine Learning Programs.

Matthias Böhm, Douglas Burdick, Alexandre Evfimievski et al. · IEEE Data(base) Engineering Bulletin · 2014

SystemML enables declarative, large-scale machine learning (ML) via a high-level language with R-like syntax. Data scientists use this language to express their ML algorithms with full flexibility but without the need to hand-tune distributed runtime execution plans and system configurations. These ML programs are dynamically compiled and optimized based on data and cluster characteristics using ruleand cost-based optimization techniques. The compiler automatically generates hybrid runtime execution plans ranging from in-memory, single node execution to distributed MapReduce (MR) computation and data access. This paper describes the SystemML optimizer, its compilation chain, and selected optimization phases for generating efficient execution plans.

Cited by 35 works.

Authors

References