scholarly journals Visual Scan-Path based Data-Augmentation for CNN-based 360-degree Image Quality Assessment

2021 ◽  
Vol 2021 (1) ◽  
pp. 21-26
Author(s):  
Abderrezzaq Sendjasni ◽  
Mohamed-Chaker Larabi ◽  
Faouzi Alaya Cheikh
Keyword(s):  
Deep Learning ◽  
Image Quality ◽  
Quality Assessment ◽  
Data Augmentation ◽  
Image Quality Assessment ◽  
Ground Truth ◽  
Training Data ◽  
Training Scheme ◽  
Scan Path ◽  
Scan Paths

360-degree Image quality assessment (IQA) is facing the major challenge of lack of ground-truth databases. This problem is accentuated for deep learning based approaches where the performances are as good as the available data. In this context, only two databases are used to train and validate deep learning-based IQA models. To compensate this lack, a dataaugmentation technique is investigated in this paper. We use visual scan-path to increase the learning examples from existing training data. Multiple scan-paths are predicted to account for the diversity of human observers. These scan-paths are then used to select viewports from the spherical representation. The results of the data-augmentation training scheme showed an improvement over not using it. We also try to answer the question of using the MOS obtained for the 360-degree image as the quality anchor for the whole set of extracted viewports in comparison to 2D blind quality metrics. The comparison showed the superiority of using the MOS when adopting a patch-based learning.

scholarly journals Evaluation of focus and deep learning methods for automated image grading and factors influencing image quality in adaptive optics ophthalmoscopy

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Danuta M. Sampson ◽  
David Alonso-Caneiro ◽  
Avenell L. Chew ◽  
Jonathan La ◽  
Danial Roshandel ◽  
...  
Keyword(s):  
Image Quality ◽  
Quality Assessment ◽  
Adaptive Optics ◽  
Image Quality Assessment ◽  
Assessment Tool ◽  
Ground Truth ◽  
Disease Status ◽  
Retinal Location ◽  
Imaging Tool ◽  
Multiclass Classifier

AbstractAdaptive optics flood illumination ophthalmoscopy (AO-FIO) is an established imaging tool in the investigation of retinal diseases. However, the clinical interpretation of AO-FIO images can be challenging due to varied image quality. Therefore, image quality assessment is essential before interpretation. An image assessment tool will also assist further work on improving the image quality, either during acquisition or post processing. In this paper, we describe, validate and compare two automated image quality assessment methods; the energy of Laplacian focus operator (LAPE; not commonly used but easily implemented) and convolutional neural network (CNN; effective but more complex approach). We also evaluate the effects of subject age, axial length, refractive error, fixation stability, disease status and retinal location on AO-FIO image quality. Based on analysis of 10,250 images of 50 × 50 μm size, at 41 retinal locations, from 50 subjects we demonstrate that CNN slightly outperforms LAPE in image quality assessment. CNN achieves accuracy of 89%, whereas LAPE metric achieves 73% and 80% (for a linear regression and random forest multiclass classifier methods, respectively) compared to ground truth. Furthermore, the retinal location, age and disease are factors that can influence the likelihood of poor image quality.

scholarly journals Explicit-implicit dual stream network for image quality assessment

2020 ◽  
Vol 2020 (1) ◽  
Author(s):  
Guangyi Yang ◽  
Xingyu Ding ◽  
Tian Huang ◽  
Kun Cheng ◽  
Weizheng Jin
Keyword(s):  
Deep Learning ◽  
Image Quality ◽  
Quality Assessment ◽  
Frequency Domain ◽  
Image Quality Assessment ◽  
Feature Fusion ◽  
Learning Model ◽  
Stream Network ◽  
Perception System ◽  
Deep Learning Model

Abstract Communications industry has remarkably changed with the development of fifth-generation cellular networks. Image, as an indispensable component of communication, has attracted wide attention. Thus, finding a suitable approach to assess image quality is important. Therefore, we propose a deep learning model for image quality assessment (IQA) based on explicit-implicit dual stream network. We use frequency domain features of kurtosis based on wavelet transform to represent explicit features and spatial features extracted by convolutional neural network (CNN) to represent implicit features. Thus, we constructed an explicit-implicit (EI) parallel deep learning model, namely, EI-IQA model. The EI-IQA model is based on the VGGNet that extracts the spatial domain features. On this basis, the number of network layers of VGGNet is reduced by adding the parallel wavelet kurtosis value frequency domain features. Thus, the training parameters and the sample requirements decline. We verified, by cross-validation of different databases, that the wavelet kurtosis feature fusion method based on deep learning has a more complete feature extraction effect and a better generalisation ability. Thus, the method can simulate the human visual perception system better, and subjective feelings become closer to the human eye. The source code about the proposed EI-IQA model is available on github https://github.com/jacob6/EI-IQA.

Fusing deep learning and statistical visual features for no-reference image quality assessment

2020 ◽  
Vol 29 (04) ◽  
pp. 1
Author(s):  
Yin Zhang ◽  
Junhua Yan ◽  
Xuan Du ◽  
Xuehan Bai ◽  
Xiyang Zhi ◽  
...  
Keyword(s):  
Deep Learning ◽  
Image Quality ◽  
Quality Assessment ◽  
Image Quality Assessment ◽  
Visual Features ◽  
Reference Image

scholarly journals No-Reference Image Quality Assessment with Reinforcement Recursive List-Wise Ranking

2019 ◽  
Vol 33 ◽  
pp. 8336-8343
Author(s):  
Jie Gu ◽  
Gaofeng Meng ◽  
Cheng Da ◽  
Shiming Xiang ◽  
Chunhong Pan
Keyword(s):  
Image Quality ◽  
Quality Assessment ◽  
Image Quality Assessment ◽  
Ground Truth ◽  
Superior Performance ◽  
Reference Image ◽  
Model Parameters ◽  
Ranking Problem ◽  
Ranking List ◽  
Markov Decision

Opinion-unaware no-reference image quality assessment (NR-IQA) methods have received many interests recently because they do not require images with subjective scores for training. Unfortunately, it is a challenging task, and thus far no opinion-unaware methods have shown consistently better performance than the opinion-aware ones. In this paper, we propose an effective opinion-unaware NR-IQA method based on reinforcement recursive list-wise ranking. We formulate the NR-IQA as a recursive list-wise ranking problem which aims to optimize the whole quality ordering directly. During training, the recursive ranking process can be modeled as a Markov decision process (MDP). The ranking list of images can be constructed by taking a sequence of actions, and each of them refers to selecting an image for a specific position of the ranking list. Reinforcement learning is adopted to train the model parameters, in which no ground-truth quality scores or ranking lists are necessary for learning. Experimental results demonstrate the superior performance of our approach compared with existing opinion-unaware NR-IQA methods. Furthermore, our approach can compete with the most effective opinion-aware methods. It improves the state-of-the-art by over 2% on the CSIQ benchmark and outperforms most compared opinion-aware models on TID2013.

scholarly journals DLNR-SIQA: Deep Learning-Based No-Reference Stitched Image Quality Assessment

Sensors ◽  
2020 ◽  
Vol 20 (22) ◽  
pp. 6457
Author(s):  
Hayat Ullah ◽  
Muhammad Irfan ◽  
Kyungjin Han ◽  
Jong Weon Lee
Keyword(s):  
Deep Learning ◽  
Image Quality ◽  
Quality Assessment ◽  
Image Quality Assessment ◽  
State Of The Art ◽  
Reference Image ◽  
Primary Concern ◽  
Panoramic Images ◽  
Stitching Errors

Due to recent advancements in virtual reality (VR) and augmented reality (AR), the demand for high quality immersive contents is a primary concern for production companies and consumers. Similarly, the topical record-breaking performance of deep learning in various domains of artificial intelligence has extended the attention of researchers to contribute to different fields of computer vision. To ensure the quality of immersive media contents using these advanced deep learning technologies, several learning based Stitched Image Quality Assessment methods have been proposed with reasonable performances. However, these methods are unable to localize, segment, and extract the stitching errors in panoramic images. Further, these methods used computationally complex procedures for quality assessment of panoramic images. With these motivations, in this paper, we propose a novel three-fold Deep Learning based No-Reference Stitched Image Quality Assessment (DLNR-SIQA) approach to evaluate the quality of immersive contents. In the first fold, we fined-tuned the state-of-the-art Mask R-CNN (Regional Convolutional Neural Network) on manually annotated various stitching error-based cropped images from the two publicly available datasets. In the second fold, we segment and localize various stitching errors present in the immersive contents. Finally, based on the distorted regions present in the immersive contents, we measured the overall quality of the stitched images. Unlike existing methods that only measure the quality of the images using deep features, our proposed method can efficiently segment and localize stitching errors and estimate the image quality by investigating segmented regions. We also carried out extensive qualitative and quantitative comparison with full reference image quality assessment (FR-IQA) and no reference image quality assessment (NR-IQA) on two publicly available datasets, where the proposed system outperformed the existing state-of-the-art techniques.

Sign in / Sign up

Export Citation Format

Share Document