SCNN
Angshuman Parashar, Minsoo Rhu, Anurag Mukkara et al. · Proceedings - International Symposium on Computer Architecture/Proceedings · 2017
Convolutional Neural Networks (CNNs) have emerged as a fundamental technology for machine learning. High performance and extreme energy efficiency are critical for deployments of CNNs, especially in mobile platforms such as autonomous vehicles, cameras, and electronic personal assistants. This paper introduces the Sparse CNN (SCNN) accelerator architecture, which improves performance and energy efficiency by exploiting the zero-valued weights that stem from network pruning during training and zero-valued activations that arise from the common ReLU operator. Specifically, SCNN employs a novel dataflow that enables maintaining the sparse weights and activations in a compressed encoding, which eliminates unnecessary data transfers and reduces storage requirements. Furthermore, the SCNN dataflow facilitates efficient delivery of those weights and activations to a multiplier array, where they are extensively reused; product accumulation is performed in a novel accumulator array. On contemporary neural networks, SCNN can improve both performance and energy by a factor of 2.7x and 2.3x, respectively, over a comparably provisioned dense CNN accelerator.
Cited by 896 works. doi:10.1145/3079856.3080254
Authors
- Angshuman Parashar
- Minsoo Rhu
- Anurag Mukkara
- Antonio Puglielli
- Rangharajan Venkatesan
- Brucek Khailany
- Joel Emer
- Stephen W. Keckler
- W.J. Dally
References
- ImageNet classification with deep convolutional neural networks
- Going deeper with convolutions
- Deep learning
- Fully convolutional networks for semantic segmentation
- Framewise phoneme classification with bidirectional LSTM and other neural network architectures
- Deep Residual Learning for Image Recognition
- Natural Language Processing (almost) from Scratch
- Deep Networks with Stochastic Depth
- Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
- Deep Speech 2: End-to-End Speech Recognition in English and Mandarin