An adaptive query execution system for data integration
Zachary G. Ives, Daniela Florescu, Marc Friedman et al. · ACM SIGMOD Record · 1999
Query processing in data integration occurs over network-bound, autonomous data sources. This requires extensions to traditional optimization and execution techniques for three reasons: there is an absence of quality statistics about the data, data transfer rates are unpredictable and bursty, and slow or unavailable data sources can often be replaced by overlapping or mirrored sources. This paper presents the Tukwila data integration system, designed to support adaptivity at its core using a two-pronged approach. Interleaved planning and execution with partial optimization allows Tukwila to quickly recover from decisions based on inaccurate estimates. During execution, Tukwila uses adaptive query operators such as the double pipelined hash join, which produces answers quickly, and the dynamic collector, which robustly and efficiently computes unions across overlapping data sources. We demonstrate that the Tukwila architecture extends previous innovations in adaptive execution (such as query scrambling, mid-execution re-optimization, and choose nodes), and we present experimental evidence that our techniques result in behavior desirable for a data integration system.
Cited by 393 works. doi:10.1145/304181.304209
Authors
References
- Proceedings of the 19th International Joint Conference on Artificial Intelligence
- Proceedings of the International Joint Conference on Artificial Intelligence 2007
- Query evaluation techniques for large databases
- Decomposition—a strategy for query processing
- Principles of Distributed Database Systems
- Wrapper induction for information extraction
- The TSIMMIS project: Integration of heterogeneous information sources
- Querying Heterogeneous Information Sources Using Source Descriptions
- The TSIMMIS Approach to Mediation: Data Models and Languages
- Integration of heterogeneous databases without common domains using queries based on textual similarity