Integrating compression and execution in column-oriented database systems
Daniel J. Abadi, Samuel R. Madden, Miguel Ferreira · Proceedings - ACM-SIGMOD International Conference on Management of Data · 2006
Column-oriented database system architectures invite a re-evaluation of how and when data in databases is compressed. Storing data in a column-oriented fashion greatly increases the similarity of adjacent records on disk and thus opportunities for compression. The ability to compress many adjacent tuples at once lowers the per-tuple cost of compression, both in terms of CPU and space overheads.In this paper, we discuss how we extended C-Store (a column-oriented DBMS) with a compression sub-system. We show how compression schemes not traditionally used in row-oriented DBMSs can be applied to column-oriented systems. We then evaluate a set of compression schemes and show that the best scheme depends not only on the properties of the data but also on the nature of the query workload.
Cited by 600 works. doi:10.1145/1142473.1142548
Authors
References
- A Method for the Construction of Minimum-Redundancy Codes
- A universal algorithm for sequential data compression
- Compression of individual sequences via variable-rate coding
- Improved query performance with variant indexes
- Weaving Relations for Cache Performance
- MonetDB/X100: Hyper-Pipelining Query Execution
- Database Architecture Optimized for the New Bottleneck: Memory Access
- Super-Scalar RAM-CPU Cache Compression
- MIL primitives for querying a fragmented world
- Compressing relations and indexes