Multi-Scale Frequency Reconstruction for Guided Depth Map Super-Resolution via Deep Residual Network
Yifan Zuo, Qiang Wu, Yuming Fang et al. · IEEE Transactions on Circuits and Systems for Video Technology · 2019
The depth maps obtained by the consumer-level sensors are always noisy in the low-resolution (LR) domain. Existing methods for the guided depth super-resolution, which are based on the pre-defined local and global models, perform well in general cases (e.g., joint bilateral filter and Markov random field). However, such model-based methods may fail to describe the potential relationship between RGB-D image pairs. To solve this problem, this paper proposes a data-driven approach based on the deep convolutional neural network with global and local residual learning. It progressively upsamples the LR depth map guided by the high-resolution intensity image in multiple scales. A global residual learning is adopted to learn the difference between the ground truth and the coarsely upsampled depth map, and the local residual learning is introduced in each scale-dependent reconstruction sub-network. This scheme can restore the depth structure from coarse to fine via multi-scale frequency synthesis. In addition, batch normalization layers are used to improve the performance of depth map denoising. Our method is evaluated in noise-free and noisy cases. A comprehensive comparison against 17 state-of-the-art methods is carried out. The experimental results show that the proposed method has faster convergence speed as well as improved performances based on the qualitative and quantitative evaluations.
Cited by 82 works. doi:10.1109/tcsvt.2018.2890271
Authors
References
- Deep Residual Learning for Image Recognition
- Adam: A Method for Stochastic Optimization
- Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Image Super-Resolution Using Deep Convolutional Networks
- Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
- Guided Image Filtering
- Guided Image Filtering
- Joint bilateral upsampling