Darkroom
James Hegarty, John S. Brunhaver, Zachary DeVito et al. · ACM Transactions on Graphics · 2014
Specialized image signal processors (ISPs) exploit the structure of image processing pipelines to minimize memory bandwidth using the architectural pattern of line-buffering , where all intermediate data between each stage is stored in small on-chip buffers. This provides high energy efficiency, allowing long pipelines with tera-op/sec. image processing in battery-powered devices, but traditionally requires painstaking manual design in hardware. Based on this pattern, we present Darkroom, a language and compiler for image processing. The semantics of the Darkroom language allow it to compile programs directly into line-buffered pipelines, with all intermediate values in local line-buffer storage, eliminating unnecessary communication with off-chip DRAM. We formulate the problem of optimally scheduling line-buffered pipelines to minimize buffering as an integer linear program. Finally, given an optimally scheduled pipeline, Darkroom synthesizes hardware descriptions for ASIC or FPGA, or fast CPU code. We evaluate Darkroom implementations of a range of applications, including a camera pipeline, low-level feature detection algorithms, and deblurring. For many applications, we demonstrate gigapixel/sec. performance in under 0.5mm 2 of ASIC silicon at 250 mW (simulated on a 45nm foundry process), real-time 1080p/60 video processing using a fraction of the resources of a modern FPGA, and tens of megapixels/sec. of throughput on a quad-core x86 processor.
Cited by 177 works. doi:10.1145/2601097.2601174
Authors
- John S. Brunhaver
- Zachary DeVito
- Jonathan Ragan‐Kelley
- Steven Bell
- Artem Vasilyev
- Mark Horowitz
- Pat Hanrahan
References
- A Computational Approach to Edge Detection
- A Combined Corner and Edge Detector
- An Iterative Image Registration Technique with an Application to Stereo Vision
- Bayesian-Based Iterative Method of Image Restoration*
- Static Scheduling of Synchronous Data Flow Programs for Digital Signal Processing
- Pyramidal implementation of the Lucas Kanade feature tracker description of the algorithm
- Retiming synchronous circuitry
- Halide
- Stencil computation optimization and auto-tuning on state-of-the-art multicore architectures
- Understanding sources of inefficiency in general-purpose chips