Effects of the deep learning-based super-resolution method on thermal image classification applications

Conversion of one video bitstream to another video bitstream is a challenging task in the heterogeneous transcoder due to different video formats. In this paper, a region of interest (ROI) based super resolution technique is used to convert the lowresolution AVS (audio video standard) video to high definition HEVC (high efficiency video coding) video. Firstly, we classify a low-resolution video frame into small blocks by using visual characteristics, transform coefficients, and motion vector (MV) of a video. These blocks are further classified as blocks of most interest (BOMI), blocks of less interest (BOLI) and blocks of noninterest (BONI). The BONI blocks are considered as background blocks due to less interest in video and remains unchanged during SR process. Secondly, we apply deep learning based super resolution method on low resolution BOMI, and BOLI blocks to enhance the visual quality. The BOMI and BOLI blocks have high attention due to ROI that include some motion and contrast of the objects. The proposed method saves 20% to 30% computational time and obtained appreciable results as compared with full frame based super resolution method. We have tested our method on different official video sequences with resolution of 1K, 2K, and 4K. Our proposed method has an efficient visual performance in contrast to the full frame-based super resolution method.

Download Full-text

Block Based Enhancement using Deep Learning for Conversion of Low Resolution AVS Video to High Resolution HEVC Video

10.36227/techrxiv.14887917 ◽

2021 ◽

Author(s):

Rizwan Qureshi ◽

Mehmood Nawaz

Keyword(s):

Deep Learning ◽

High Efficiency ◽

Motion Vector ◽

Super Resolution ◽

Computational Time ◽

Video Frame ◽

Low Resolution ◽

High Efficiency Video Coding ◽

High Definition ◽

Resolution Method

Conversion of one video bitstream to another video bitstream is a challenging task in the heterogeneous transcoder due to different video formats. In this paper, a region of interest (ROI) based super resolution technique is used to convert the lowresolution AVS (audio video standard) video to high definition HEVC (high efficiency video coding) video. Firstly, we classify a low-resolution video frame into small blocks by using visual characteristics, transform coefficients, and motion vector (MV) of a video. These blocks are further classified as blocks of most interest (BOMI), blocks of less interest (BOLI) and blocks of noninterest (BONI). The BONI blocks are considered as background blocks due to less interest in video and remains unchanged during SR process. Secondly, we apply deep learning based super resolution method on low resolution BOMI, and BOLI blocks to enhance the visual quality. The BOMI and BOLI blocks have high attention due to ROI that include some motion and contrast of the objects. The proposed method saves 20% to 30% computational time and obtained appreciable results as compared with full frame based super resolution method. We have tested our method on different official video sequences with resolution of 1K, 2K, and 4K. Our proposed method has an efficient visual performance in contrast to the full frame-based super resolution method.

Download Full-text

Supplementary material to "Deep-learning based climate downscaling using the super-resolution method: a case study over the western US"

10.5194/gmd-2020-214-supplement ◽

2020 ◽

Author(s):

Xingying Huang

Keyword(s):

Deep Learning ◽

Super Resolution ◽

Resolution Method ◽

Western Us ◽

Supplementary Material ◽

Climate Downscaling

Download Full-text

DL-MRI: A Unified Framework of Deep Learning-Based MRI Super Resolution

Journal of Healthcare Engineering ◽

10.1155/2021/5594649 ◽

2021 ◽

Vol 2021 ◽

pp. 1-9

Author(s):

Huanyu Liu ◽

Jiaqi Liu ◽

Junbao Li ◽

Jeng-Shyang Pan ◽

Xiaqiong Yu

Keyword(s):

Deep Learning ◽

High Resolution ◽

Super Resolution ◽

Motion Artifacts ◽

Mr Images ◽

Unified Framework ◽

Resolution Method ◽

Learning Methods ◽

Scanning Time ◽

Detection And Diagnosis

Magnetic resonance imaging (MRI) is widely used in the detection and diagnosis of diseases. High-resolution MR images will help doctors to locate lesions and diagnose diseases. However, the acquisition of high-resolution MR images requires high magnetic field intensity and long scanning time, which will bring discomfort to patients and easily introduce motion artifacts, resulting in image quality degradation. Therefore, the resolution of hardware imaging has reached its limit. Based on this situation, a unified framework based on deep learning super resolution is proposed to transfer state-of-the-art deep learning methods of natural images to MRI super resolution. Compared with the traditional image super-resolution method, the deep learning super-resolution method has stronger feature extraction and characterization ability, can learn prior knowledge from a large number of sample data, and has a more stable and excellent image reconstruction effect. We propose a unified framework of deep learning -based MRI super resolution, which has five current deep learning methods with the best super-resolution effect. In addition, a high-low resolution MR image dataset with the scales of ×2, ×3, and ×4 was constructed, covering 4 parts of the skull, knee, breast, and head and neck. Experimental results show that the proposed unified framework of deep learning super resolution has a better reconstruction effect on the data than traditional methods and provides a standard dataset and experimental benchmark for the application of deep learning super resolution in MR images.

Download Full-text

Deep-learning based climate downscaling using the super-resolution method: a case study over the western US

10.5194/gmd-2020-214 ◽

2020 ◽

Author(s):

Xingying Huang

Keyword(s):

Deep Learning ◽

High Resolution ◽

Super Resolution ◽

Fine Scale ◽

Climate Information ◽

Climate Data ◽

Modeling Framework ◽

Resolution Method ◽

Coarse Resolution ◽

Climate Downscaling

Abstract. Demand for high-resolution climate information is growing rapidly to fulfill the needs of both scientists and stakeholders. However, deriving high-quality fine-resolution information is still challenging due to either the complexity of a dynamical climate model or the uncertainty of an empirical statistical model. In this work, a new downscaling framework is developed using the deep-learning based super-resolution method to generate very high-resolution output from coarse-resolution input. The modeling framework has been trained, tested, and validated for generating high-resolution (here, 4 km) climate data focusing on temperature and precipitation at daily scale from the year 1981 to 2010. This newly designed downscaling framework is composed of multiple convolutional layers involving batch normalization, rectification-linear unit, and skip connection strategies, with different loss functions explored. The overall logic for this modeling framework is to learn optimal parameters from the training data for later-on prediction applications. This new method and framework is found to largely reduce the time and computation cost (~ 23 milliseconds for one-day inference) for climate downscaling compared to current downscaling strategies. The strength and limitation of this deep-learning based downscaling have been investigated and evaluated using both fine-scale gridded observations and dynamical downscaling data from regional climate models. The performance of this deep-learning framework is found to be competitive in either generating the spatial details or maintaining the temporal evolutions at a very fine grid-scale. It is promising that this deep-learning based downscaling method can be a powerful and effective way to retrieve fine-scale climate information from other coarse-resolution climate data. When seeking an efficient and affordable way for intensive climate downscaling, an optimized convolution neural network framework like the one explored here could be an alternative option and applied to a broad relevant application.

Download Full-text