Efficiently compiling efficient query plans for modern hardware
Thomas Neumann · Proceedings of the VLDB Endowment · 2011
As main memory grows, query performance is more and more determined by the raw CPU costs of query processing itself. The classical iterator style query processing technique is very simple and exible, but shows poor performance on modern CPUs due to lack of locality and frequent instruction mispredictions. Several techniques like batch oriented processing or vectorized tuple processing have been proposed in the past to improve this situation, but even these techniques are frequently out-performed by hand-written execution plans. In this work we present a novel compilation strategy that translates a query into compact and efficient machine code using the LLVM compiler framework. By aiming at good code and data locality and predictable branch layout the resulting code frequently rivals the performance of hand-written C++ code. We integrated these techniques into the HyPer main memory database system and show that this results in excellent query performance while requiring only modest compilation time.
Cited by 483 works. doi:10.14778/2002938.2002940
Authors
References
- MonetDB/X100: Hyper-Pipelining Query Execution
- The Volcano optimizer generator: extensibility and efficient search
- HyPer: A hybrid OLTP&OLAP main memory database system based on virtual memory snapshots
- MonetDB/X100 - A DBMS in the CPU cache
- SIMD-scan
- Block oriented processing of relational database operations in modern computer architectures
- Conjunctive selection conditions in main memory
- Constant-Time Query Processing
- Optimizing database architecture for the new bottleneck: memory access
- Database architecture evolution