Machine Learning at Facebook: Understanding Inference at the Edge
Carole-Jean Wu, David M. Brooks, Kevin Yanjun Chen et al. · Proceedings - International Symposium on High-Performance Computer Architecture/Proceedings · 2019
At Facebook, machine learning provides a wide range of capabilities that drive many aspects of user experience including ranking posts, content understanding, object detection and tracking for augmented and virtual reality, speech and text translations. While machine learning models are currently trained on customized data-center infrastructure, Facebook is working to bring machine learning inference to the edge. By doing so, user experience is improved with reduced latency (inference time) and becomes less dependent on network connectivity. Furthermore, this also enables many more applications of deep learning with important features only made available at the edge. This paper takes a data-driven approach to present the opportunities and design challenges faced by Facebook in order to enable machine learning inference locally on smart phones and other edge platforms.
Cited by 439 works. doi:10.1109/hpca.2019.00048
Authors
- Carole-Jean Wu
- David M. Brooks
- Kevin Yanjun Chen
- Sy Choudhury
- Marat Dukhan
- Kim Hazelwood
- Yangqing Jia
- Tommer Leyvand
- Hao Lü
- Yang Lu
- Lin Qiao
- Brandon Reagen
- Joe Spisak
- Fei Sun
- Andrew Tulloch
- Péter Vajda
- Xiaodong Wang
References
- Going deeper with convolutions
- U-Net: Convolutional Networks for Biomedical Image Segmentation
- Mask R-CNN
- Practical Bayesian Optimization of Machine Learning Algorithms
- Optimal Brain Damage
- ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
- Neural Architecture Search with Reinforcement Learning
- In-Datacenter Performance Analysis of a Tensor Processing Unit
- Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference
- Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1