Compiler Techniques for Massively Scalable Implicit Task Parallelism
Timothy G. Armstrong, Justin M. Wozniak, Michael Wilde et al. · 2012 International Conference for High Performance Computing, Networking, Storage and Analysis · 2014
Swift/T is a high-level language for writing concise, deterministic scripts that compose serial or parallel codes implemented in lower-level programming models into large-scale parallel applications. It executes using a data-driven task parallel execution model that is capable of orchestrating millions of concurrently executing asynchronous tasks on homogeneous or heterogeneous resources. Producing code that executes efficiently at this scale requires sophisticated compiler transformations: poorly optimized code inhibits scaling with excessive synchronization and communication. We present a comprehensive set of compiler techniques for data-driven task parallelism, including novel compiler optimizations and intermediate representations. We report application benchmark studies, including unbalanced tree search and simulated annealing, and demonstrate that our techniques greatly reduce communication overhead and enable extreme scalability, distributing up to 612 million dynamically load balanced tasks per second at scales of up to 262,144 cores without explicit parallelism, synchronization, or load balancing in application code.
Cited by 50 works. doi:10.1109/sc.2014.30
Authors
References
- Advanced Compiler Design and Implementation
- StarPU: a unified platform for task scheduling on heterogeneous multicore architectures
- Exploiting coarse-grained task, data, and pipeline parallelism in stream programs
- A COMPILER FOR THE MIT TAGGED-TOKEN DATAFLOW ARCHITECTURE
- Code generation in the polyhedral model is easier than you think
- Workflow Scheduling Algorithms for Grid Computing
- Dynamic task scheduling for linear algebra algorithms on distributed-memory multicore systems
- Swift: A language for distributed parallel scripting
- Hierarchical Task-Based Programming With StarSs
- DAGuE: A generic distributed DAG engine for High Performance Computing