Parallel Multiscale Autoregressive Density Estimation
Scott Reed, Aäron van den Oord, Nal Kalchbrenner et al. · arXiv (Cornell University) · 2017
PixelCNN achieves state-of-the-art results in density estimation for natural images. Although training is fast, inference is costly, requiring one network evaluation per pixel; O(N) for N pixels. This can be sped up by caching activations, but still involves generating each pixel sequentially. In this work, we propose a parallelized PixelCNN that allows more efficient inference by modeling certain pixel groups as conditionally independent. Our new PixelCNN model achieves competitive density estimation and orders of magnitude speedup - O(log N) sampling instead of O(N) - enabling the practical generation of 512x512 images. We evaluate the model on class-conditional image generation, text-to-image synthesis, and action-conditional video generation, showing that our model achieves the best results among non-pixel-autoregressive density models that allow efficient sampling.
Cited by 38 works. doi:10.48550/arxiv.1703.03664 Open-access PDF
Authors
- Scott Reed
- Aäron van den Oord
- Nal Kalchbrenner
- Sergio Gómez Colmenarejo
- Ziyu Wang
- Dan Belov
- Nando de Freitas
References
- ImageNet: A large-scale hierarchical image database
- Microsoft COCO: Common Objects in Context
- GAN(Generative Adversarial Nets)
- Identity Mappings in Deep Residual Networks
- Perceptual Losses for Real-Time Style Transfer and Super-Resolution
- Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
- WaveNet: A Generative Model for Raw Audio
- Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network
- 2D Human Pose Estimation: New Benchmark and State of the Art Analysis
- Generative Adversarial Text to Image Synthesis