Data placement in Bubba
George P. Copeland, William Alexander, Ellen E. Boughter et al. · ACM SIGMOD Record · 1988
This paper examines the problem of data placement in Bubba, a highly-parallel system for data-intensive applications being developed at MCC. “Highly-parallel” implies that load balancing is a critical performance issue. “Data-intensive” means data is so large that operations should be executed where the data resides. As a result, data placement becomes a critical performance issue. In general, determining the optimal placement of data across processing nodes for performance is a difficult problem. We describe our heuristic approach to solving the data placement problem in Bubba. We then present experimental results using a specific workload to provide insight into the problem. Several researchers have argued the benefits of declustering (i e, spreading each base relation over many nodes). We show that as declustering is increased, load balancing continues to improve. However, for transactions involving complex joins, further declustering reduces throughput because of communications, startup and termination overhead. We argue that data placement, especially declustering, in a highly-parallel system must be considered early in the design, so that mechanisms can be included for supporting variable declustering, for minimizing the most significant overheads associated with large-scale declustering, and for gathering the required statistics.
Cited by 300 works. doi:10.1145/971701.50213 Open-access PDF
Authors
References
- Notes on data base operating systems
- GAMMA—a high performance dataflow database machine
- Quantitative system performance: computer system analysis using queueing network models
- The Case for Shared Nothing.
- The Operational Analysis of Queueing Network Models
- Multi-disk management algorithms
- The 5 minute rule for trading memory for disc accesses and the 10 byte rule for trading memory for CPU time
- Optimal allocation of resources in distributed information networks
- Optimum data base reorganization points
- Optimal reorganization of distributed space disk files