Static Analysis of Shape in TensorFlow Programs
The Theano Development Team, Rami Al‐Rfou, Guillaume Alain et al. · arXiv (Cornell University) · 2016
Machine learning has been widely adopted in diverse science and engineering domains, aided by reusable libraries and quick development patterns. The TensorFlow library is probably the best-known representative of this trend and most users employ the Python API to its powerful back-end. TensorFlow programs are susceptible to several systematic errors, especially in the dynamic typing setting of Python. We present Pythia, a static analysis that tracks the shapes of tensors across Python library calls and warns of several possible mismatches. The key technical aspects are a close modeling of library semantics with respect to tensor shape, and an identification of violations and error-prone patterns. Pythia is powerful enough to statically detect (with 84.62% precision) 11 of the 14 shape-related TensorFlow bugs in the recent Zhang et al. empirical study - an independent slice of real-world bugs.
Cited by 1,589 works. doi:10.4230/lipics.ecoop.2020.15 Open-access PDF
Authors
- The Theano Development Team
- Rami Al‐Rfou
- Guillaume Alain
- Amjad Almahairi
- Christof Angermueller
- Dzmitry Bahdanau
- Nicolas Ballas
- Frédéric Bastien
- Justin Bayer
- Anatoly Belikov
- Alexander Belopolsky
- Yoshua Bengio
- Arnaud Bergeron
- James Bergstra
- Valentin Bisson
- Josh Bleecher Snyder
- Nicolas Boulanger-Lewandowski
- Xavier Bouthillier
- Alexandre de Brébisson
References
- ImageNet classification with deep convolutional neural networks
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Going deeper with convolutions
- Caffe
- Large Scale Distributed Deep Networks
- Torch7: A Matlab-like Environment for Machine Learning
- The NumPy Array: A Structure for Efficient Numerical Computation
- Pylearn2: a machine learning research library
- Theano: new features and speed improvements
- One weird trick for parallelizing convolutional neural networks