Aggregation and ordering in factorised databases
Nurzhan Bakibayev, Tomáš Kočiský, Dan Olteanu et al. · Proceedings of the VLDB Endowment · 2013
A common approach to data analysis involves understanding and manipulating succinct representations of data. In earlier work, we put forward a succinct representation system for relational data called factorised databases and reported on the main-memory query engine FDB for select-project-join queries on such databases. In this paper, we extend FDB to support a larger class of practical queries with aggregates and ordering. This requires novel optimisation and evaluation techniques. We show how factorisation coupled with partial aggregation can effectively reduce the number of operations needed for query evaluation. We also show how factorisations of query results can support enumeration of tuples in desired orders as efficiently as listing them from the unfactorised, sorted results. We experimentally observe that FDB can outperform off-the-shelf relational engines by orders of magnitude.
Cited by 76 works. doi:10.14778/2556549.2556579
Authors
References
- Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference
- Foundations of databases
- Database Management Systems
- Foundations of databases
- Remarks on the algebra of non first normal form relations
- Megastore: Providing Scalable, Highly Available Storage for Interactive Services
- Eager Aggregation and Lazy Aggregation
- Database Architecture Optimized for the New Bottleneck: Memory Access
- Database Management Systems
- A consideration on normal form of not-necessarily-normalized relation in the relational data model