scholarly journals Deep Gradient Prior Regularized Robust Video Super-Resolution

Electronics ◽  
2021 ◽  
Vol 10 (14) ◽  
pp. 1641
Author(s):  
Qiang Song ◽  
Hangfan Liu
Keyword(s):  
High Resolution ◽  
Reference Frames ◽  
Low Frequency ◽  
Super Resolution ◽  
Mapping Function ◽  
Gradient Field ◽  
Motion Field ◽  
Low Resolution ◽  
Frequency Information ◽  
Gradient Mapping

This paper proposes a robust multi-frame video super-resolution (SR) scheme to obtain high SR performance under large upscaling factors. Although the reference low-resolution frames can provide complementary information for the high-resolution frame, an effective regularizer is required to rectify the unreliable information from the reference frames. As the high-frequency information is mostly contained in the image gradient field, we propose to learn the gradient-mapping function between the high-resolution (HR) and the low-resolution (LR) image to regularize the fusion of multiple frames. In contrast to the existing spatial-domain networks, we train a deep gradient-mapping network to learn the horizontal and vertical gradients. We found that adding the low-frequency information (mainly from the LR image) to the gradient-learning network can boost the performance of the network. A forward and backward motion field prior is used to regularize the estimation of the motion flow between frames. For robust SR reconstruction, a weighting scheme is proposed to exclude the outlier data. Visual and quantitative evaluations on benchmark datasets demonstrate that our method is superior to many state-of-the-art methods and can recover better details with less artifacts.

scholarly journals Super Resolution Digital Image Correlation (SR-DIC): an Alternative to Image Stitching at High Magnifications

2021 ◽  
Author(s):  
R. S. Hansen ◽  
D. W. Waldram ◽  
T. Q. Thai ◽  
R. B. Berke
Keyword(s):  
High Resolution ◽  
Digital Image Correlation ◽  
Spatial Resolution ◽  
Digital Image ◽  
Super Resolution ◽  
Image Correlation ◽  
Low Resolution ◽  
Strain Measurements ◽  
Resolution Image ◽  
High Resolution Image

Abstract Background High-resolution Digital Image Correlation (DIC) measurements have previously been produced by stitching of neighboring images, which often requires short working distances. Separately, the image processing community has developed super resolution (SR) imaging techniques, which improve resolution by combining multiple overlapping images. Objective This work investigates the novel pairing of super resolution with digital image correlation, as an alternative method to produce high-resolution full-field strain measurements. Methods First, an image reconstruction test is performed, comparing the ability of three previously published SR algorithms to replicate a high-resolution image. Second, an applied translation is compared against DIC measurement using both low- and super-resolution images. Third, a ring sample is mechanically deformed and DIC strain measurements from low- and super-resolution images are compared. Results SR measurements show improvements compared to low-resolution images, although they do not perfectly replicate the high-resolution image. SR-DIC demonstrates reduced error and improved confidence in measuring rigid body translation when compared to low resolution alternatives, and it also shows improvement in spatial resolution for strain measurements of ring deformation. Conclusions Super resolution imaging can be effectively paired with Digital Image Correlation, offering improved spatial resolution, reduced error, and increased measurement confidence.

The enhanced deep Plug-and-play super-resolution algorithm with residual channel attention networks

2021 ◽  
pp. 1-10
Author(s):  
Hongguang Pan ◽  
Fan Wen ◽  
Xiangdong Huang ◽  
Xinyu Lei ◽  
Xiaoling Yang
Keyword(s):  
Signal To Noise Ratio ◽  
Low Frequency ◽  
Super Resolution ◽  
Structural Similarity ◽  
Plug And Play ◽  
Attention Networks ◽  
Frequency Information ◽  
Blind Deblurring ◽  
Adaptive Adjustment ◽  
Deep Feature Extraction

In the field of super-resolution image reconstruction, as a learning-based method, deep plug-and-play super-resolution (DPSR) algorithm can be used to find the blur kernel by using the existing blind deblurring methods. However, DPSR is not flexible enough in processing images with high- and low-frequency information. Considering a channel attention mechanism can distinguish low-frequency information and features in low-resolution images, in this paper, we firstly introduce this mechanism and design a new residual channel attention networks (RCAN); then the RCAN is adopted to replace deep feature extraction part in DPSR to achieve the adaptive adjustment of channel characteristics. Through four test experiments based on Set5, Set14, Urban100 and BSD100 datasets, we find that, under different blur kernels and different scale factors, the average peak signal to noise ratio (PSNR) and structural similarity (SSIM) values of our proposed method increase by 0.31dB and 0.55%, respectively; under different noise levels, the average PSNR and SSIM values increase by 0.26dB and 0.51%, respectively.

scholarly journals Semi-Coupled Convolutional Sparse Learning for Image Super-Resolution

2019 ◽  
Vol 11 (21) ◽  
pp. 2593
Author(s):  
Li ◽  
Zhang ◽  
Jiao ◽  
Liu ◽  
Yang ◽  
...  
Keyword(s):  
High Resolution ◽  
Super Resolution ◽  
Mapping Function ◽  
Linear Mapping ◽  
Sparse Learning ◽  
Proposed Model ◽  
Filter Size ◽  
High Resolution Images ◽  
Resolution Problem ◽  
Image Super Resolution

In the convolutional sparse coding-based image super-resolution problem, the coefficients of low- and high-resolution images in the same position are assumed to be equivalent, which enforces an identical structure of low- and high-resolution images. However, in fact the structure of high-resolution images is much more complicated than that of low-resolution images. In order to reduce the coupling between low- and high-resolution representations, a semi-coupled convolutional sparse learning method (SCCSL) is proposed for image super-resolution. The proposed method uses nonlinear convolution operations as the mapping function between low- and high-resolution features, and conventional linear mapping can be seen as a special case of the proposed method. Secondly, the neighborhoods within the filter size are used to calculate the current pixel, improving the flexibility of our proposed model. In addition, the filter size is adjustable. In order to illustrate the effectiveness of SCCSL method, we compare it with four state-of-the-art methods of 15 commonly used images. Experimental results show that this work provides a more flexible and efficient approach for image super-resolution problem.

scholarly journals Crop Disease Classification on Inadequate Low-Resolution Target Images

Sensors ◽  
2020 ◽  
Vol 20 (16) ◽  
pp. 4601
Author(s):  
Juan Wen ◽  
Yangjing Shi ◽  
Xiaoshi Zhou ◽  
Yiming Xue
Keyword(s):  
High Resolution ◽  
Super Resolution ◽  
Classification Performance ◽  
Disease Classification ◽  
Generative Adversarial Networks ◽  
Low Resolution ◽  
Training Samples ◽  
Crop Disease ◽  
High Resolution Images ◽  
Model Training

Currently, various agricultural image classification tasks are carried out on high-resolution images. However, in some cases, we cannot get enough high-resolution images for classification, which significantly affects classification performance. In this paper, we design a crop disease classification network based on Enhanced Super-Resolution Generative adversarial networks (ESRGAN) when only an insufficient number of low-resolution target images are available. First, ESRGAN is used to recover super-resolution crop images from low-resolution images. Transfer learning is applied in model training to compensate for the lack of training samples. Then, we test the performance of the generated super-resolution images in crop disease classification task. Extensive experiments show that using the fine-tuned ESRGAN model can recover realistic crop information and improve the accuracy of crop disease classification, compared with the other four image super-resolution methods.

Distillation

2010 ◽  
pp. 244-264
Author(s):  
Dong Seon Cheng ◽  
Marco Cristani ◽  
Vittorio Murino
Keyword(s):  
High Resolution ◽  
Super Resolution ◽  
Video Data ◽  
Distillation Process ◽  
Low Resolution ◽  
Resolution Image ◽  
High Resolution Image ◽  
Image Super Resolution ◽  
Comparative Results ◽  
Low Resolution Images

Image super-resolution is one of the most appealing applications of image processing, capable of retrieving a high resolution image by fusing several registered low resolution images depicting an object of interest. However, employing super-resolution in video data is challenging: a video sequence generally contains a lot of scattered information regarding several objects of interest in cluttered scenes. Especially with hand-held cameras, the overall quality may be poor due to low resolution or unsteadiness. The objective of this chapter is to demonstrate why standard image super-resolution fails in video data, which are the problems that arise, and how we can overcome these problems. In our first contribution, we propose a novel Bayesian framework for super-resolution of persistent objects of interest in video sequences. We call this process Distillation. In the traditional formulation of the image super-resolution problem, the observed target is (1) always the same, (2) acquired using a camera making small movements, and (3) found in a number of low resolution images sufficient to recover high-frequency information. These assumptions are usually unsatisfied in real world video acquisitions and often beyond the control of the video operator. With Distillation, we aim to extend and to generalize the image super-resolution task, embedding it in a structured framework that accurately distills all the informative bits of an object of interest. In practice, the Distillation process: i) individuates, in a semi supervised way, a set of objects of interest, clustering the related video frames and registering them with respect to global rigid transformations; ii) for each one, produces a high resolution image, by weighting each pixel according to the information retrieved about the object of interest. As a second contribution, we extend the Distillation process to deal with objects of interest whose transformations in the appearance are not (only) rigid. Such process, built on top of the Distillation, is hierarchical, in the sense that a process of clustering is applied recursively, beginning with the analysis of whole frames, and selectively focusing on smaller sub-regions whose isolated motion can be reasonably assumed as rigid. The ultimate product of the overall process is a strip of images that describe at high resolution the dynamics of the video, switching between alternative local descriptions in response to visual changes. Our approach is first tested on synthetic data, obtaining encouraging comparative results with respect to known super-resolution techniques, and a good robustness against noise. Second, real data coming from different videos are considered, trying to solve the major details of the objects in motion.

scholarly journals Hyperspectral Super-Resolution Technique Using Histogram Matching and Endmember Optimization

2019 ◽  
Vol 9 (20) ◽  
pp. 4444
Author(s):  
Byunghyun Kim ◽  
Soojin Cho
Keyword(s):  
High Resolution ◽  
Spatial Information ◽  
Super Resolution ◽  
Spatial Mismatch ◽  
Low Resolution ◽  
Practical Applications ◽  
Histogram Matching ◽  
Shear Transformation ◽  
Entire Dataset ◽  
Rgb Image

In most hyperspectral super-resolution (HSR) methods, which are techniques used to improve the resolution of hyperspectral images (HSIs), the HSI and the target RGB image are assumed to have identical fields of view. However, because implementing these identical fields of view is difficult in practical applications, in this paper, we propose a HSR method that is applicable when an HSI and a target RGB image have different spatial information. The proposed HSR method first creates a low-resolution RGB image from a given HSI. Next, a histogram matching is performed on a high-resolution RGB image and a low-resolution RGB image obtained from an HSI. Finally, the proposed method optimizes endmember abundance of the high-resolution HSI towards the histogram-matched high-resolution RGB image. The entire procedure is evaluated using an open HSI dataset, the Harvard dataset, by adding spatial mismatch to the dataset. The spatial mismatch is implemented by shear transformation and cutting off the upper and left sides of the target RGB image. The proposed method achieved a lower error rate across the entire dataset, confirming its capability for super-resolution using images that have different fields of view.

scholarly journals Regularized Multiframe Super-Resolution Image Reconstruction Using Linear and Nonlinear Filters

2021 ◽  
Vol 2021 ◽  
pp. 1-16
Author(s):  
Mahmoud M. Khattab ◽  
Akram M. Zeki ◽  
Ali A. Alwan ◽  
Belgacem Bouallegue ◽  
Safaa S. Matter ◽  
...  
Keyword(s):  
High Resolution ◽  
Image Reconstruction ◽  
Synthetic Data ◽  
Stable State ◽  
Super Resolution ◽  
Low Resolution ◽  
Edge Preservation ◽  
Resolution Image ◽  
High Resolution Image ◽  
L2 Norm

The primary goal of the multiframe super-resolution image reconstruction is to produce an image with a higher resolution by integrating information extracted from a set of corresponding images with low resolution, which is used in various fields. However, super-resolution image reconstruction approaches are typically affected by annoying restorative artifacts, including blurring, noise, and staircasing effect. Accordingly, it is always difficult to balance between smoothness and edge preservation. In this paper, we intend to enhance the efficiency of multiframe super-resolution image reconstruction in order to optimize both analysis and human interpretation processes by improving the pictorial information and enhancing the automatic machine perception. As a result, we propose new approaches that firstly rely on estimating the initial high-resolution image through preprocessing of the reference low-resolution image based on median, mean, Lucy-Richardson, and Wiener filters. This preprocessing stage is used to overcome the degradation present in the reference low-resolution image, which is a suitable kernel for producing the initial high-resolution image to be used in the reconstruction phase of the final image. Then, L2 norm is employed for the data-fidelity term to minimize the residual among the predicted high-resolution image and the observed low-resolution images. Finally, bilateral total variation prior model is utilized to restrict the minimization function to a stable state of the generated HR image. The experimental results of the synthetic data indicate that the proposed approaches have enhanced efficiency visually and quantitatively compared to other existing approaches.

scholarly journals CONVOLUTIONAL NEURAL NETWORK BASED DEM SUPER RESOLUTION

2016 ◽  
Vol XLI-B3 ◽  
pp. 247-250 ◽  
Author(s):  
Zixuan Chen ◽  
Xuewen Wang ◽  
Zekai Xu ◽  
Wenguang Hou
Keyword(s):  
Neural Network ◽  
High Resolution ◽  
Convolutional Neural Network ◽  
Super Resolution ◽  
Low Resolution ◽  
Practical Applications ◽  
Interpolation Methods ◽  
New Strategy

DEM super resolution is proposed in our previous publication to improve the resolution for a DEM on basis of some learning examples. Meanwhile, the nonlocal algorithm is introduced to deal with it and lots of experiments show that the strategy is feasible. In our publication, the learning examples are defined as the partial original DEM and their related high measurements due to this way can avoid the incompatibility between the data to be processed and the learning examples. To further extent the applications of this new strategy, the learning examples should be diverse and easy to obtain. Yet, it may cause the problem of incompatibility and unrobustness. To overcome it, we intend to investigate a convolutional neural network based method. The input of the convolutional neural network is a low resolution DEM and the output is expected to be its high resolution one. A three layers model will be adopted. The first layer is used to detect some features from the input, the second integrates the detected features to some compressed ones and the final step transforms the compressed features as a new DEM. According to this designed structure, some learning DEMs will be taken to train it. Specifically, the designed network will be optimized by minimizing the error of the output and its expected high resolution DEM. In practical applications, a testing DEM will be input to the convolutional neural network and a super resolution will be obtained. Many experiments show that the CNN based method can obtain better reconstructions than many classic interpolation methods.

scholarly journals Low Resolution Information Also Matters: Learning Multi-Resolution Representations for Person Re-Identification

2021 ◽  
Author(s):  
Guoqing Zhang ◽  
Yuhao Chen ◽  
Weisi Lin ◽  
Arun Chandran ◽  
Xuan Jing
Keyword(s):  
Feature Extraction ◽  
High Resolution ◽  
Feature Fusion ◽  
State Of The Art ◽  
Super Resolution ◽  
Input Image ◽  
Low Resolution ◽  
Joint Learning ◽  
Novel Method ◽  
Valid Information

As a prevailing task in video surveillance and forensics field, person re-identification (re-ID) aims to match person images captured from non-overlapped cameras. In unconstrained scenarios, person images often suffer from the resolution mismatch problem, i.e., Cross-Resolution Person Re-ID. To overcome this problem, most existing methods restore low resolution (LR) images to high resolution (HR) by super-resolution (SR). However, they only focus on the HR feature extraction and ignore the valid information from original LR images. In this work, we explore the influence of resolutions on feature extraction and develop a novel method for cross-resolution person re-ID called Multi-Resolution Representations Joint Learning (MRJL). Our method consists of a Resolution Reconstruction Network (RRN) and a Dual Feature Fusion Network (DFFN). The RRN uses an input image to construct a HR version and a LR version with an encoder and two decoders, while the DFFN adopts a dual-branch structure to generate person representations from multi-resolution images. Comprehensive experiments on five benchmarks verify the superiority of the proposed MRJL over the relevent state-of-the-art methods.

scholarly journals Cascaded SR-GAN for Scale-Adaptive Low Resolution Person Re-identification

2018 ◽  
Author(s):  
Zheng Wang ◽  
Mang Ye ◽  
Fan Yang ◽  
Xiang Bai ◽  
Shin'ichi Satoh
Keyword(s):  
High Resolution ◽  
Super Resolution ◽  
Feature Representation ◽  
Image Feature ◽  
Low Resolution ◽  
Open World ◽  
In Series ◽  
High Resolution Images ◽  
Public Dataset ◽  
Image Super Resolution

Person re-identification (REID) is an important task in video surveillance and forensics applications. Most of previous approaches are based on a key assumption that all person images have uniform and sufficiently high resolutions. Actually, various low-resolutions and scale mismatching always exist in open world REID. We name this kind of problem as Scale-Adaptive Low Resolution Person Re-identification (SALR-REID). The most intuitive way to address this problem is to increase various low-resolutions (not only low, but also with different scales) to a uniform high-resolution. SR-GAN is one of the most competitive image super-resolution deep networks, designed with a fixed upscaling factor. However, it is still not suitable for SALR-REID task, which requires a network not only synthesizing high-resolution images with different upscaling factors, but also extracting discriminative image feature for judging person’s identity. (1) To promote the ability of scale-adaptive upscaling, we cascade multiple SRGANs in series. (2) To supplement the ability of image feature representation, we plug-in a reidentification network. With a unified formulation, a Cascaded Super-Resolution GAN (CSR-GAN) framework is proposed. Extensive evaluations on two simulated datasets and one public dataset demonstrate the advantages of our method over related state-of-the-art methods.

Sign in / Sign up

Export Citation Format

Share Document