Quickly generating billion-record synthetic databases
Jim Gray, Prakash Sundaresan, Susanne Englert et al. · Proceedings - ACM-SIGMOD International Conference on Management of Data · 1994
Evaluating database system performance often requires generating synthetic databases—ones having certain statistical properties but filled with dummy information. When evaluating different database designs, it is often necessary to generate several databases and evaluate each design. As database sizes grow to terabytes, generation often takes longer than evaluation. This paper presents several database generation techniques. In particular it discusses: (1) Parallelism to get generation speedup and scaleup. (2) Congruential generators to get dense unique uniform distributions. (3) Special-case discrete logarithms to generate indices concurrent to the base table generation. (4) Modification of (2) to get exponential, normal, and self-similar distributions.
Cited by 353 works. doi:10.1145/191839.191886
Authors
References
- The Art of Computer Programming
- Stochastic Simulation
- The Art of Computer Systems Performance Analysis.
- The art of computer systems performance analysis
- Benchmarking Database Systems A Systematic Approach
- A More Portable Fortran Random Number Generator
- GAMMA—a high performance dataflow database machine
- The Gamma database machine project
- Discrete logarithms inGF(p)
- The Case for Shared Nothing.