Kernel Modeling Super-Resolution on Real Low-Resolution Images

Deep convolutional neural networks (CNNs), trained on corresponding pairs of high- and low-resolution images, achieve state-of-the-art performance in single-image super- resolution and surpass previous signal-processing based approaches. However, their performance is limited when applied to real photographs. The reason lies in their train- ing data: low-resolution (LR) images are obtained by bicu- bic interpolation of the corresponding high-resolution (HR) images. The applied convolution kernel significantly differs from real-world camera-blur. Consequently, while current CNNs well super-resolve bicubic-downsampled LR images, they often fail on camera-captured LR images. To improve generalization and robustness of deep super- resolution CNNs on real photographs, we present a ker- nel modeling super-resolution network (KMSR) that incor- porates blur-kernel modeling in the training. Our pro- posed KMSR consists of two stages: we first build a pool of realistic blur-kernels with a generative adversarial net- work (GAN) and then we train a super-resolution network with HR and corresponding LR images constructed with the generated kernels. Our extensive experimental vali- dations demonstrate the effectiveness of our single-image super-resolution approach on photographs with unknown blur-kernels.