PVFS : a parallel file system for linux clusters
Philip Carns, Walter B. Ligon III, Robert B. Ross et al. · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · 2000
As Linux clusters have matured as platforms for low-cost, high-performance parallel computing, software packages to provide many key services have emerged, especially in areas such as message passing and net-working. One area devoid of support, however, has been parallel file systems, which are critical for high-performance I/O on such clusters. We have developed a parallel file system for Linux clusters, called the Parallel Virtual File System (PVFS). PVFS is intended both as a high-performance parallel file system that anyone can download and use and as a tool for pursuing further re-search in parallel I/O and parallel file systems for Linux clusters. In this paper, we describe the design and implementa-tion of PVFS and present performance results on the Chiba City cluster at Argonne. We provide performance results for a workload of concurrent reads and writes for various numbers of compute nodes, I/O nodes, and I/O request sizes. We also present performance results for MPI-IO on PVFS, both for a concurrent read/write workload and for the BTIO benchmark. We compare the I/O performance when using a Myrinet network versus a fast-ethernet network for I/O-related communication in PVFS. We obtained read and write bandwidths as high as 700 Mbytes/sec with Myrinet and 225 Mbytes/sec with fast ethernet.
Cited by 835 works. Open-access PDF
Authors
References
- A high-performance, portable implementation of the MPI message passing interface standard
- Serverless network file systems
- Checkpoint and Migration of UNIX Processes in the Condor Distributed Processing System
- Data sieving and collective I/O in ROMIO
- Using MPI-2: Advanced features of the message-passing interface
- An abstract-device interface for implementing portable parallel-I/O interfaces
- On implementing MPI-IO portably and with high performance
- PPFS
- A 64-bit, shared disk file system for Linux
- Implementing the beam and warming method on the hypercube