Subjective annotation for a frame interpolation benchmark using artefact amplification

Abstract Current benchmarks for optical flow algorithms evaluate the estimation either directly by comparing the predicted flow fields with the ground truth or indirectly by using the predicted flow fields for frame interpolation and then comparing the interpolated frames with the actual frames. In the latter case, objective quality measures such as the mean squared error are typically employed. However, it is well known that for image quality assessment, the actual quality experienced by the user cannot be fully deduced from such simple measures. Hence, we conducted a subjective quality assessment crowdscouring study for the interpolated frames provided by one of the optical flow benchmarks, the Middlebury benchmark. It contains interpolated frames from 155 methods applied to each of 8 contents. For this purpose, we collected forced-choice paired comparisons between interpolated images and corresponding ground truth. To increase the sensitivity of observers when judging minute difference in paired comparisons we introduced a new method to the field of full-reference quality assessment, called artefact amplification. From the crowdsourcing data (3720 comparisons of 20 votes each) we reconstructed absolute quality scale values according to Thurstone’s model. As a result, we obtained a re-ranking of the 155 participating algorithms w.r.t. the visual quality of the interpolated frames. This re-ranking not only shows the necessity of visual quality assessment as another evaluation metric for optical flow and frame interpolation benchmarks, the results also provide the ground truth for designing novel image quality assessment (IQA) methods dedicated to perceptual quality of interpolated images. As a first step, we proposed such a new full-reference method, called WAE-IQA, which weights the local differences between an interpolated image and its ground truth.

Download Full-text

A Combined Full-Reference Image Quality Assessment Method Based on Convolutional Activation Maps

Algorithms ◽

10.3390/a13120313 ◽

2020 ◽

Vol 13 (12) ◽

pp. 313

Author(s):

Domonkos Varga

Keyword(s):

Image Quality ◽

Quality Assessment ◽

Image Quality Assessment ◽

Feature Vector ◽

State Of The Art ◽

Prediction Performance ◽

Perceptual Quality ◽

Reference Image ◽

Full Reference

The goal of full-reference image quality assessment (FR-IQA) is to predict the perceptual quality of an image as perceived by human observers using its pristine (distortion free) reference counterpart. In this study, we explore a novel, combined approach which predicts the perceptual quality of a distorted image by compiling a feature vector from convolutional activation maps. More specifically, a reference-distorted image pair is run through a pretrained convolutional neural network and the activation maps are compared with a traditional image similarity metric. Subsequently, the resulting feature vector is mapped onto perceptual quality scores with the help of a trained support vector regressor. A detailed parameter study is also presented in which the design choices of the proposed method is explained. Furthermore, we study the relationship between the amount of training images and the prediction performance. Specifically, it is demonstrated that the proposed method can be trained with a small amount of data to reach high prediction performance. Our best proposal—called ActMapFeat—is compared to the state-of-the-art on six publicly available benchmark IQA databases, such as KADID-10k, TID2013, TID2008, MDID, CSIQ, and VCL-FER. Specifically, our method is able to significantly outperform the state-of-the-art on these benchmark databases.

Download Full-text

Evaluation of focus and deep learning methods for automated image grading and factors influencing image quality in adaptive optics ophthalmoscopy

Scientific Reports ◽

10.1038/s41598-021-96068-2 ◽

2021 ◽

Vol 11 (1) ◽

Author(s):

Danuta M. Sampson ◽

David Alonso-Caneiro ◽

Avenell L. Chew ◽

Jonathan La ◽

Danial Roshandel ◽

...

Keyword(s):

Image Quality ◽

Quality Assessment ◽

Adaptive Optics ◽

Image Quality Assessment ◽

Assessment Tool ◽

Ground Truth ◽

Disease Status ◽

Retinal Location ◽

Imaging Tool ◽

Multiclass Classifier

AbstractAdaptive optics flood illumination ophthalmoscopy (AO-FIO) is an established imaging tool in the investigation of retinal diseases. However, the clinical interpretation of AO-FIO images can be challenging due to varied image quality. Therefore, image quality assessment is essential before interpretation. An image assessment tool will also assist further work on improving the image quality, either during acquisition or post processing. In this paper, we describe, validate and compare two automated image quality assessment methods; the energy of Laplacian focus operator (LAPE; not commonly used but easily implemented) and convolutional neural network (CNN; effective but more complex approach). We also evaluate the effects of subject age, axial length, refractive error, fixation stability, disease status and retinal location on AO-FIO image quality. Based on analysis of 10,250 images of 50 × 50 μm size, at 41 retinal locations, from 50 subjects we demonstrate that CNN slightly outperforms LAPE in image quality assessment. CNN achieves accuracy of 89%, whereas LAPE metric achieves 73% and 80% (for a linear regression and random forest multiclass classifier methods, respectively) compared to ground truth. Furthermore, the retinal location, age and disease are factors that can influence the likelihood of poor image quality.

Download Full-text

A comprehensive assessment of content-based image retrieval using selected full reference image quality assessment algorithms

Multimedia Tools and Applications ◽

10.1007/s11042-021-10573-0 ◽

2021 ◽

Author(s):

Baisakhi Sur Phadikar ◽

Amit Phadikar ◽

Subro S. Thakur

Keyword(s):

Image Quality ◽

Image Retrieval ◽

Quality Assessment ◽

Image Quality Assessment ◽

Comprehensive Assessment ◽

Content Based Image Retrieval ◽

Reference Image ◽

Full Reference

Download Full-text

Interobserver variability in quality assessment of magnetic resonance images

BMC Medical Imaging ◽

10.1186/s12880-020-00505-z ◽

2020 ◽

Vol 20 (1) ◽

Author(s):

Rafal Obuchowicz ◽

Mariusz Oszust ◽

Adam Piorkowski

Keyword(s):

Magnetic Resonance ◽

Image Quality ◽

Quality Assessment ◽

Image Quality Assessment ◽

Interobserver Variability ◽

Poor Performance ◽

Magnetic Resonance Images ◽

Perceptual Quality ◽

Mr Images

Abstract Background The perceptual quality of magnetic resonance (MR) images influences diagnosis and may compromise the treatment. The purpose of this study was to evaluate how the image quality changes influence the interobserver variability of their assessment. Methods For the variability evaluation, a dataset containing distorted MRI images was prepared and then assessed by 31 experienced medical professionals (radiologists). Differences between observers were analyzed using the Fleiss’ kappa. However, since the kappa evaluates the agreement among radiologists taking into account aggregated decisions, a typically employed criterion of the image quality assessment (IQA) performance was used to provide a more thorough analysis. The IQA performance of radiologists was evaluated by comparing the Spearman correlation coefficients, ρ, between individual scores with the mean opinion scores (MOS) composed of the subjective opinions of the remaining professionals. Results The experiments show that there is a significant agreement among radiologists (κ=0.12; 95% confidence interval [CI]: 0.118, 0.121; P<0.001) on the quality of the assessed images. The resulted κ is strongly affected by the subjectivity of the assigned scores, separately presenting close scores. Therefore, the ρ was used to identify poor performance cases and to confirm the consistency of the majority of collected scores (ρmean = 0.5706). The results for interns (ρmean = 0.6868) supports the finding that the quality assessment of MR images can be successfully taught. Conclusions The agreement observed among radiologists from different imaging centers confirms the subjectivity of the perception of MR images. It was shown that the image content and severity of distortions affect the IQA. Furthermore, the study highlights the importance of the psychosomatic condition of the observers and their attitude.

Download Full-text

A Study of Stereoscopic Image Quality Assessment Model Corresponding to Disparate Quality of Left/Right Image for JPEG Coding

IEICE Transactions on Fundamentals of Electronics Communications and Computer Sciences ◽

10.1587/transfun.e95.a.1264 ◽

2012 ◽

Vol E95.A (8) ◽

pp. 1264-1269

Author(s):

Masaharu SATO ◽

Yuukou HORITA

Keyword(s):

Image Quality ◽

Quality Assessment ◽

Image Quality Assessment ◽

Assessment Model ◽

Stereoscopic Image

Download Full-text

Full-Reference Predictive Modeling of Subjective Image Quality Assessment with ANFIS

Lecture Notes in Computer Science - Agents and Artificial Intelligence ◽

10.1007/978-3-319-25210-0_18 ◽

2015 ◽

pp. 296-311

Author(s):

El-Sayed M. El-Alfy ◽

Mohammed Rehan Riaz

Keyword(s):

Image Quality ◽

Quality Assessment ◽

Predictive Modeling ◽

Image Quality Assessment ◽

Subjective Image Quality ◽

Full Reference

Download Full-text

No-Reference Image Quality Assessment with Reinforcement Recursive List-Wise Ranking

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v33i01.33018336 ◽

2019 ◽

Vol 33 ◽

pp. 8336-8343

Author(s):

Jie Gu ◽

Gaofeng Meng ◽

Cheng Da ◽

Shiming Xiang ◽

Chunhong Pan

Keyword(s):

Image Quality ◽

Quality Assessment ◽

Image Quality Assessment ◽

Ground Truth ◽

Superior Performance ◽

Reference Image ◽

Model Parameters ◽

Ranking Problem ◽

Ranking List ◽

Markov Decision

Opinion-unaware no-reference image quality assessment (NR-IQA) methods have received many interests recently because they do not require images with subjective scores for training. Unfortunately, it is a challenging task, and thus far no opinion-unaware methods have shown consistently better performance than the opinion-aware ones. In this paper, we propose an effective opinion-unaware NR-IQA method based on reinforcement recursive list-wise ranking. We formulate the NR-IQA as a recursive list-wise ranking problem which aims to optimize the whole quality ordering directly. During training, the recursive ranking process can be modeled as a Markov decision process (MDP). The ranking list of images can be constructed by taking a sequence of actions, and each of them refers to selecting an image for a specific position of the ranking list. Reinforcement learning is adopted to train the model parameters, in which no ground-truth quality scores or ranking lists are necessary for learning. Experimental results demonstrate the superior performance of our approach compared with existing opinion-unaware NR-IQA methods. Furthermore, our approach can compete with the most effective opinion-aware methods. It improves the state-of-the-art by over 2% on the CSIQ benchmark and outperforms most compared opinion-aware models on TID2013.

Download Full-text

DLNR-SIQA: Deep Learning-Based No-Reference Stitched Image Quality Assessment

Sensors ◽

10.3390/s20226457 ◽

2020 ◽

Vol 20 (22) ◽

pp. 6457

Author(s):

Hayat Ullah ◽

Muhammad Irfan ◽

Kyungjin Han ◽

Jong Weon Lee

Keyword(s):

Deep Learning ◽

Image Quality ◽

Quality Assessment ◽

Image Quality Assessment ◽

State Of The Art ◽

Reference Image ◽

Primary Concern ◽

Panoramic Images ◽

Stitching Errors

Due to recent advancements in virtual reality (VR) and augmented reality (AR), the demand for high quality immersive contents is a primary concern for production companies and consumers. Similarly, the topical record-breaking performance of deep learning in various domains of artificial intelligence has extended the attention of researchers to contribute to different fields of computer vision. To ensure the quality of immersive media contents using these advanced deep learning technologies, several learning based Stitched Image Quality Assessment methods have been proposed with reasonable performances. However, these methods are unable to localize, segment, and extract the stitching errors in panoramic images. Further, these methods used computationally complex procedures for quality assessment of panoramic images. With these motivations, in this paper, we propose a novel three-fold Deep Learning based No-Reference Stitched Image Quality Assessment (DLNR-SIQA) approach to evaluate the quality of immersive contents. In the first fold, we fined-tuned the state-of-the-art Mask R-CNN (Regional Convolutional Neural Network) on manually annotated various stitching error-based cropped images from the two publicly available datasets. In the second fold, we segment and localize various stitching errors present in the immersive contents. Finally, based on the distorted regions present in the immersive contents, we measured the overall quality of the stitched images. Unlike existing methods that only measure the quality of the images using deep features, our proposed method can efficiently segment and localize stitching errors and estimate the image quality by investigating segmented regions. We also carried out extensive qualitative and quantitative comparison with full reference image quality assessment (FR-IQA) and no reference image quality assessment (NR-IQA) on two publicly available datasets, where the proposed system outperformed the existing state-of-the-art techniques.

Download Full-text