Advection-Based Sparse Data Management for Visualizing Unsteady Flow
Hanqi Guo, Jiang Zhang, Richen Liu et al. · IEEE Transactions on Visualization and Computer Graphics · 2014
When computing integral curves and integral surfaces for large-scale unsteady flow fields, a major bottleneck is the widening gap between data access demands and the available bandwidth (both I/O and in-memory). In this work, we explore a novel advection-based scheme to manage flow field data for both efficiency and scalability. The key is to first partition flow field into blocklets (e.g. cells or very fine-grained blocks of cells), and then (pre)fetch and manage blocklets on-demand using a parallel key-value store. The benefits are (1) greatly increasing the scale of local-range analysis (e.g. source-destination queries, streak surface generation) that can fit within any given limit of hardware resources; (2) improving memory and I/O bandwidth-efficiencies as well as the scalability of naive task-parallel particle advection. We demonstrate our method using a prototype system that works on workstation and also in supercomputing environments. Results show significantly reduced I/O overhead compared to accessing raw flow data, and also high scalability on a supercomputer for a variety of applications.
Cited by 25 works. doi:10.1109/tvcg.2014.2346418
Authors
References
- Space/time trade-offs in hash coding with allowable errors
- Bigtable
- Consistent hashing and random trees
- The log-structured merge-tree (LSM-tree)
- Distinguished material surfaces and coherent structures in three-dimensional fluid flows
- The State of the Art in Flow Visualisation: Feature Extraction and Tracking
- The State of the Art in Flow Visualization: Dense and Texture‐Based Techniques
- Over Two Decades of Integration‐Based, Geometric Flow Visualization
- Efficient Computation and Visualization of Coherent Structures in Fluid Flow Applications
- ZHT: A Light-Weight Reliable Persistent Dynamic Scalable Zero-Hop Distributed Hash Table