Development of Learning-Based Techniques for Single-Image Super-Resolution

Abstract

Image acquisition remains a challenging task due to the substandard imaging environment, inaccurate camera settings, vibration in camera-mounting machinery, varying refractive index of atmosphere, and many more. Images thus acquired in such conditions are degraded and possess little scientific value. Furthermore, applications like medical imaging, surveillance, forensic, satellite imaging, etc., require zooming of images for better analysis. And a degraded image, when zoomed, cannot provide any additional information. Therefore, a classical ill-posed inverse problem called super-resolution (SR) has been widely used to reconstruct a high-resolution (HR) image from a given low-resolution (LR) image. Super-resolution techniques have been divided into Single-image SR and Multi-image SR, depending on how many LR images are used for generating the HR image. Both well-known methods to convert an ill-posed problem to a well-posed one and obtain a stable solution have been discussed in Chapter 1. A detailed literature survey has been done in Chapter 2 for both single-image and multi-image SR techniques. It has been inferred that the SR image reconstructed by traditional multi-image SR method produces ringing artifacts near strong edges, where single-image SR schemes have an advantage. However, single-image SR schemes have a lot of challenges and open problems to be resolved. In this thesis, learning-based single-image SR approach has been used for expected HR image generation. In Chapter 3, two frameworks based on locally linear embedding (LLE) have been developed. A new feature vector has been generated in the first framework by combining residual luminance inspired by the Gaussian pyramid and the first gradient. In the second framework, the feature vector is generated by the Zernike moment. Further, the global neighborhood selection approach has been used to find the appropriate neighborhood size k for embedding.
Robust locally linear embedding (RLLE) has been used in place of LLE to improve the embedding in Chapter 4. RLLE uses RPCA to remove the outlier to reduce the artifact generated during HR image reconstruction. This scheme utilizes a neighbor embedding approach and suitably named robust neighbor embedding based super-resolution (RNESR). RNESR is trained using known LR-HR image pairs to generate information with respect to local geometry and neighborhood. Further, it uses histogram matching to select the best LR-HR image pairs for training. Subsequently, the scheme is validated using LR images selected from training pairs as well as images not used during training to generate their corresponding HR image. Here, RLLE played an essential role in generating the best patch pair using RPCA. A global neighborhood selection by local processing has been used to find the best k value for expected HR image generation.
An improved learning-based SR algorithm for texture image to generate an HR image from a single LR image has been suggested in Chapter 5. The scheme utilizes a neighbor embedding technique (manifold learning) and is suitably named improved single-image super-resolution using manifold learning (ISSRM). In this approach, HR patches are reconstructed from the input test LR patches by the prior information fetched from the training LR-HR pairs. Hence, an optimal weight reconstruction has been generated by combining the least square error and non-negative factorization matrix. The pseudo-Zernike moment has been utilized for the feature selection technique. Due to the fixed k neighbors value during HR reconstruction, the manifold learning-based approach often generates artifacts generated due to over-fitting or under-fitting. The consistency constraint is usually ignored during overlap averaging. Prior knowledge of LR-HR pair relation is exploited in Chapter 6 wherein a convolutional sparse coding (CSC) based SR is introduced. In this scheme, slice based dictionary learning is used in the patch-based method to reconstruct the HR image. Five different SR frameworks are suggested in four chapters (3–6) of the thesis. While Chapter 3 presents two variations of a framework, the remaining three chapters present one each. The first framework exploits the first-order gradient and residual luminance generated by the image pyramid. The second framework employs the first-order gradient along with three Zernike moments to preserve the global structure. The third framework is built upon the previous frameworks, wherein a set of best training image pairs are searched, and the outliers issues are handled using a robust locally linear embedding. The fourth framework employs the phase and magnitude of pseudo-Zernike moment along with a collaborative optimal reconstruction weight. The fifth framework adopts dictionary learning where a convolutional sparse coding model is used for LR-HR mapping. All the five suggested frameworks are validated on standard images. Both qualitative and quantitative performance measures like PSNR, SSIM, and FSIM are used to compare with competent schemes. In general, it is observed that the suggested schemes outperform the existing schemes.