Quality assessment of anatomical MRI images from Generative Adversarial Networks: human assessment and image quality metrics

Background: Generative Adversarial Networks (GANs) can synthesize brain images from image or noise input. So far, the gold standard for assessing the quality of the generated images has been human expert ratings. However, due to limitations of human assessment in terms of cost, scalability, and the limited sensitivity of the human eye to more subtle statistical relationships, a more automated approach towards evaluating GANs is required. New method: We investigated to what extent visual quality can be assessed using image quality metrics and we used group analysis and spatial independent components analysis to verify that the GAN reproduces multivariate statistical relationships found in real data. Reference human data was obtained by recruiting neuroimaging experts to assess real Magnetic Resonance (MR) images and images generated by a Wasserstein GAN. Image quality was manipulated by exporting images at different stages of GAN training. Results: Experts were sensitive to changes in image quality as evidenced by ratings and reaction times, and the generated images reproduced group effects (age, gender) and spatial correlations moderately well. We also surveyed a number of image quality metrics which consistently failed to fully reproduce human data. While the metrics Structural Similarity Index Measure (SSIM) and Naturalness Image Quality Evaluator (NIQE) showed good overall agreement with human assessment for lower-quality images (i.e. images from early stages of GAN training), only a Deep Quality Assessment (QA) model trained on human ratings was sensitive to the subtle differences between higher-quality images. Conclusions: We recommend a combination of group analyses, spatial correlation analyses, and both distortion metrics (SSIM, NIQE) and perceptual models (Deep QA) for a comprehensive evaluation and comparison of brain images produced by GANs.

A novel multiple subdivision-based algorithm for quantitative assessment of retinal vascular tortuosity

Vascular tortuosity as an indicator of retinal vascular morphological changes can be quantitatively analyzed and used as a biomarker for the early diagnosis of relevant disease such as diabetes. While various methods have been proposed to evaluate retinal vascular tortuosity, the main obstacle limiting their clinical application is the poor consistency compared with the experts’ evaluation. In this research, we proposed to apply a multiple subdivision-based algorithm for the vessel segment vascular tortuosity analysis combining with a learning curve function of vessel curvature inflection point number, emphasizing the human assessment nature focusing not only global but also on local vascular features. Our algorithm achieved high correlation coefficients of 0.931 for arteries and 0.925 for veins compared with clinical grading of extracted retinal vessels. For the prognostic performance against experts’ prediction in retinal fundus images from diabetic patients, the area under the receiver operating characteristic curve reached 0.968, indicating a good consistency with experts’ predication in full retinal vascular network evaluation.

Image-based assessment of plant disease progression identifies new genetic loci for resistance

A major challenge in global crop production is mitigating yield loss due to plant diseases. One of the best means of disease control is plant resistance, but the identification of genes that promote resistance has been limited by the subjective quantification of disease, which is typically scored by the human eye. We hypothesized that image-based quantification of disease phenotypes would enable the identification of new disease resistance loci. We tested this using the interaction between tomato and Ralstonia solanacearum , a soilborne pathogen that causes bacterial wilt disease. We acquired over 40,000 time-series images of disease progression in a tomato recombinant inbred line population, and developed an image analysis pipeline providing a suite of ten traits to quantify wilt disease based on plant shape and size. Quantitative trait loci (QTL) analyses using image-based phenotyping identified QTL that were both unique and shared compared with those identified by human assessment of wilting. When shared loci were identified, image-based phenotyping could detect some QTL several days earlier than human assessment. Thus, expanding the phenotypic space of disease with image-based phenotyping allowed both earlier detection and identified new genetic components of resistance.

The problem of loss of solutions in the task of searching similar documents: Applying terminology in the construction of a corpus vector model

This article considers the problem of finding text documents similar in meaning in the corpus. We investigate a problem arising when developing applied intelligent information systems that is non-detection of a part of solutions by the TF-IDF algorithm: one can lose some document pairs that are similar according to human assessment, but receive a low similarity assessment from the program. A modification of the algorithm, with the replacement of the complete vocabulary with a vocabulary of specific terms is proposed. The addition of thesauri when building a corpus vector model based on a ranking function has not been previously investigated; the use of thesauri has so far been studied only to improve topic models. The purpose of this work is to improve the quality of the solution by minimizing the loss of its significant part and not adding “false similar” pairs of documents. The improvement is provided by the use of a vocabulary of specific terms extracted from the text of the analyzed documents when calculating the TF-IDF values for corpus vector representation. The experiment was carried out on two corpora of structured normative and technical documents united by a subject: state standards related to information technology and to the field of railways. The glossary of specific terms was compiled by automatic analysis of the text of the documents under consideration, and rule-based NER methods were used. It was demonstrated that the calculation of TF-IDF based on the terminology vocabulary gives more relevant results for the problem under study, which confirmed the hypothesis put forward. The proposed method is less dependent on the shortcomings of the text layer (such as recognition errors) than the calculation of the documents’ proximity using the complete vocabulary of the corpus. We determined the factors that can affect the quality of the decision: the way of compiling a terminology vocabulary, the choice of the range of n-grams for the vocabulary, the correctness of the wording of specific terms and the validity of their inclusion in the glossary of the document. The findings can be used to solve applied problems related to the search for documents that are close in meaning, such as semantic search, taking into account the subject area, corporate search in multi-user mode, detection of hidden plagiarism, identification of contradictions in a collection of documents, determination of novelty in documents when building a knowledge base.

Identifying Periampullary Regions in MRI Images Using Deep Learning

BackgroundDevelopment and validation of a deep learning method to automatically segment the peri-ampullary (PA) region in magnetic resonance imaging (MRI) images.MethodsA group of patients with or without periampullary carcinoma (PAC) was included. The PA regions were manually annotated in MRI images by experts. Patients were randomly divided into one training set, one validation set, and one test set. Deep learning methods were developed to automatically segment the PA region in MRI images. The segmentation performance of the methods was compared in the validation set. The model with the highest intersection over union (IoU) was evaluated in the test set.ResultsThe deep learning algorithm achieved optimal accuracies in the segmentation of the PA regions in both T1 and T2 MRI images. The value of the IoU was 0.68, 0.68, and 0.64 for T1, T2, and combination of T1 and T2 images, respectively.ConclusionsDeep learning algorithm is promising with accuracies of concordance with manual human assessment in segmentation of the PA region in MRI images. This automated non-invasive method helps clinicians to identify and locate the PA region using preoperative MRI scanning.

Automatic Summarization of Cricket Highlights using Audio Processing

The problem of automatic excitement detection in cricket videos is considered and applied for highlight generation. This paper focuses on detecting exciting events in video using complementary information from the audio and video domains. First, a method of audio and video elements separation is proposed. Thereafter, the “level-of-excitement” is measured using features such as amplitude, and spectral center of gravity extracted from the commentators speech’s amplitude to decide the threshold. Our experiments using actual cricket videos show that these features are well correlated with human assessment of excitability. Finally, audio/video information is fused according to time-order scenes which has “excitability” in order to generate highlights of cricket. The techniques described in this paper are generic and applicable to a variety of topic and video/acoustic domains.

Automatic identification of bird females using egg phenotype

Identification of individuals greatly contributes to understanding animal ecology and evolution, and in many cases can only be achieved using expensive and invasive techniques. Advances in computing technology offer alternative cost-effective techniques which are less invasive and can discriminate between individuals based on visual and/or acoustic cues. Here, we employ human assessment and an automatic analytical approach to predict the identity of common cuckoo (Cuculus canorus) females based on the appearance of their eggs. The cuckoo’s secretive brood parasitic strategy makes studying its life history very challenging. Eggs were analysed using calibrated digital photography for quantifying spotting pattern, size and shape, and spectrometry for measuring colour. Cuckoo females were identified from genetic sampling of their nestlings, allowing us to determine the accuracy of human and automatic female assignment. Finally, we used a novel ‘same-different’ approach that uses both genetic and phenotypic information to assign eggs that were not genetically analysed.Our results supported the ‘constant egg-type hypothesis’, showing that individual cuckoo females lay eggs with a relatively constant appearance and that eggs laid by different females differ more than eggs laid by the same female. The accuracy of unsupervised hierarchical clustering was comparable to assessments of experienced human observers. Supervised random forest analysis showed better results, with higher cluster accuracy. Same-different analysis was able to assign 22 of 87 unidentified cuckoo eggs to seven already known females.Our study showed that egg appearance on its own is not sufficient for identification of individual cuckoo females. We therefore advocate genetic analysis to be used for this purpose. However, supervised analytical methods reliably assigned a relatively high number of eggs without genetic data to their mothers which can be used in conjunction with genetic testing as a cost-effective method for increasing sample sizes for eggs where genetic samples could not be obtained.

Macular hole morphology and measurement using an automated three-dimensional image segmentation algorithm

ObjectiveFull-thickness macular holes (MH) are classified principally by size, which is one of the strongest predictors of anatomical and visual success. Using a three-dimensional (3D) automated image processing algorithm, we analysed optical coherence tomography (OCT) images of 104 MH of patients, comparing MH dimensions and morphology with clinician-acquired two-dimensional measurements.Methods and AnalysisAll patients underwent a high-density central horizontal scanning OCT protocol. Two independent clinicians measured the minimum linear diameter (MLD) and maximum base diameter. OCT images were also analysed using an automated 3D segmentation algorithm which produced key parameters including the respective maximum and minimum diameter of the minimum area (MA) of the MH, as well as volume and surface area.ResultsUsing the algorithm-derived values, MH were found to have significant asymmetry in all dimensions. The minima of the MA were typically approximately 90° to the horizontal, and differed from their maxima by 55 μm. The minima of the MA differed from the human-measured MLD by a mean of nearly 50 μm, with significant interobserver variability. The resultant differences led to reclassification using the International Vitreomacular Traction Study Group classification in a quarter of the patients (p=0.07).ConclusionMH are complex shapes with significant asymmetry in all dimensions. We have shown how 3D automated analysis of MH describes their dimensions more accurately and repeatably than human assessment. This could be used in future studies investigating hole progression and outcome to help guide optimum treatments.