scholarly journals Performance and limitation of machine learning algorithms for diabetic retinopathy screening: A meta-analysis (Preprint)

2020 ◽  
Author(s):  
Jo-Hsuan Wu ◽  
Tin-Yan Alvin Liu ◽  
Wan-Ting Hsu ◽  
Jennifer Hui-Chun Ho ◽  
Chien-Chang Lee
Keyword(s):  
Machine Learning ◽  
Diabetic Retinopathy ◽  
Diagnostic Accuracy ◽  
State Of The Art ◽  
Meta Analysis ◽  
External Validation ◽  
Screening Method ◽  
Machine Learning Algorithms ◽  
Diabetic Retinopathy Screening ◽  
Fundus Photographs

BACKGROUND Standardly diagnosed by human experts, the high prevalence of diabetic retinopathy (DR) warrants a more efficient screening method. Although machine learning (ML)-based automated DR diagnosis has gained attention due to recent approval of IDx-DR, performance of this tool has not be examined systematically, and the best ML technique for utilization in real-world setting has not been discussed. OBJECTIVE To examine systematically the overall diagnostic accuracy of ML in diagnosing DR of different categories based on color fundus photographs and to determine the state-of-the-art ML approach. METHODS Published studies in PubMed and EMBASE were searched from inception to June, 2020. Studies were screened for relevant outcomes, publication types, and data sufficiency, and a total of 60 (2.8%) out of 2128 studies were retrieved after study selection. Extraction of data was performed by 2 authors according to PRISMA, and the quality assessment was performed according to QUADUS-2. Meta-analysis of diagnostic accuracy was pooled using a bivariate random-effects model. The main outcomes included diagnostic accuracy, sensitivity, and specificity of ML in diagnosing DR based on color fundus photographs, as well as the performances of different major types of ML algorithms. RESULTS The primary meta-analysis included 60 color fundus photograph studies (445,175 interpretations). Overall, ML demonstrated high accuracy in diagnosing DR of various categories, with a pooled AUROC from 0.97 (95% CI: 0.96, 0.99) to 0.99 (95%CI: 0.98, 1.00). The performance of ML in detecting more-than-mild DR (mtmDR) was robust (Sen: 0.95, AUROC: 0.97), and by subgroup analyses, we observed that robust performance of ML was not limited to benchmark datasets (Sen: 0.92; AUROC: 0.96) but could be generalized to images collected in clinical practice (Sen: 0.97; AUROC: 097). Neural network was the most widely utilized method, and the subgroup analysis revealed a pooled AUROC of 0.98 (95% CI: 0.96, 0.99) for studies that utilized neural networks to diagnose mtmDR. CONCLUSIONS This meta-analysis demonstrated high diagnostic accuracy of ML algorithms in detecting diabetic retinopathy on color fundus photographs, suggesting that state-of-the-art, ML-based DR screening algorithms are likely ready for clinical applications. However, a significant portion of the earlier published studies had methodology flaws, such as the lack of external validation and presence of spectrum bias. The results of these studies should be interpreted with caution.

scholarly journals Automated analysis of retinal imaging using machine learning techniques for computer vision

F1000Research ◽  
2016 ◽  
Vol 5 ◽  
pp. 1573 ◽  
Author(s):  
Jeffrey De Fauw ◽  
Pearse Keane ◽  
Nenad Tomasev ◽  
Daniel Visentin ◽  
George van den Driessche ◽  
...  
Keyword(s):  
Machine Learning ◽  
Diabetic Retinopathy ◽  
Macular Degeneration ◽  
Human Error ◽  
Machine Learning Algorithms ◽  
Age Related Macular Degeneration ◽  
Machine Learning Techniques ◽  
Sound Waves ◽  
Age Related ◽  
Fundus Photographs

There are almost two million people in the United Kingdom living with sight loss, including around 360,000 people who are registered as blind or partially sighted. Sight threatening diseases, such as diabetic retinopathy and age related macular degeneration have contributed to the 40% increase in outpatient attendances in the last decade but are amenable to early detection and monitoring. With early and appropriate intervention, blindness may be prevented in many cases. Ophthalmic imaging provides a way to diagnose and objectively assess the progression of a number of pathologies including neovascular (“wet”) age-related macular degeneration (wet AMD) and diabetic retinopathy. Two methods of imaging are commonly used: digital photographs of the fundus (the ‘back’ of the eye) and Optical Coherence Tomography (OCT, a modality that uses light waves in a similar way to how ultrasound uses sound waves). Changes in population demographics and expectations and the changing pattern of chronic diseases creates a rising demand for such imaging. Meanwhile, interrogation of such images is time consuming, costly, and prone to human error. The application of novel analysis methods may provide a solution to these challenges. This research will focus on applying novel machine learning algorithms to automatic analysis of both digital fundus photographs and OCT in Moorfields Eye Hospital NHS Foundation Trust patients. Through analysis of the images used in ophthalmology, along with relevant clinical and demographic information, Google DeepMind Health will investigate the feasibility of automated grading of digital fundus photographs and OCT and provide novel quantitative measures for specific disease features and for monitoring the therapeutic success.

scholarly journals Automated analysis of retinal imaging using machine learning techniques for computer vision

F1000Research ◽  
2017 ◽  
Vol 5 ◽  
pp. 1573 ◽  
Author(s):  
Jeffrey De Fauw ◽  
Pearse Keane ◽  
Nenad Tomasev ◽  
Daniel Visentin ◽  
George van den Driessche ◽  
...  
Keyword(s):  
Machine Learning ◽  
Diabetic Retinopathy ◽  
Macular Degeneration ◽  
Human Error ◽  
Machine Learning Algorithms ◽  
Age Related Macular Degeneration ◽  
Machine Learning Techniques ◽  
Sound Waves ◽  
Age Related ◽  
Fundus Photographs

There are almost two million people in the United Kingdom living with sight loss, including around 360,000 people who are registered as blind or partially sighted. Sight threatening diseases, such as diabetic retinopathy and age related macular degeneration have contributed to the 40% increase in outpatient attendances in the last decade but are amenable to early detection and monitoring. With early and appropriate intervention, blindness may be prevented in many cases. Ophthalmic imaging provides a way to diagnose and objectively assess the progression of a number of pathologies including neovascular (“wet”) age-related macular degeneration (wet AMD) and diabetic retinopathy. Two methods of imaging are commonly used: digital photographs of the fundus (the ‘back’ of the eye) and Optical Coherence Tomography (OCT, a modality that uses light waves in a similar way to how ultrasound uses sound waves). Changes in population demographics and expectations and the changing pattern of chronic diseases creates a rising demand for such imaging. Meanwhile, interrogation of such images is time consuming, costly, and prone to human error. The application of novel analysis methods may provide a solution to these challenges. This research will focus on applying novel machine learning algorithms to automatic analysis of both digital fundus photographs and OCT in Moorfields Eye Hospital NHS Foundation Trust patients. Through analysis of the images used in ophthalmology, along with relevant clinical and demographic information, DeepMind Health will investigate the feasibility of automated grading of digital fundus photographs and OCT and provide novel quantitative measures for specific disease features and for monitoring the therapeutic success.

scholarly journals Accuracy of Machine Learning Algorithms for the Classification of Molecular Features of Gliomas on MRI: A Systematic Literature Review and Meta-Analysis

Cancers ◽  
2021 ◽  
Vol 13 (11) ◽  
pp. 2606
Author(s):  
Evi J. van Kempen ◽  
Max Post ◽  
Manoj Mannil ◽  
Benno Kusters ◽  
Mark ter Laan ◽  
...  
Keyword(s):  
Machine Learning ◽  
Large Scale ◽  
Meta Analysis ◽  
Learning Algorithms ◽  
External Validation ◽  
Machine Learning Algorithms ◽  
Molecular Characteristics ◽  
Aggregated Data ◽  
Molecular Features

Treatment planning and prognosis in glioma treatment are based on the classification into low- and high-grade oligodendroglioma or astrocytoma, which is mainly based on molecular characteristics (IDH1/2- and 1p/19q codeletion status). It would be of great value if this classification could be made reliably before surgery, without biopsy. Machine learning algorithms (MLAs) could play a role in achieving this by enabling glioma characterization on magnetic resonance imaging (MRI) data without invasive tissue sampling. The aim of this study is to provide a performance evaluation and meta-analysis of various MLAs for glioma characterization. Systematic literature search and meta-analysis were performed on the aggregated data, after which subgroup analyses for several target conditions were conducted. This study is registered with PROSPERO, CRD42020191033. We identified 724 studies; 60 and 17 studies were eligible to be included in the systematic review and meta-analysis, respectively. Meta-analysis showed excellent accuracy for all subgroups, with the classification of 1p/19q codeletion status scoring significantly poorer than other subgroups (AUC: 0.748, p = 0.132). There was considerable heterogeneity among some of the included studies. Although promising results were found with regard to the ability of MLA-tools to be used for the non-invasive classification of gliomas, large-scale, prospective trials with external validation are warranted in the future.

scholarly journals Identification of Diabetic Retinopathy through Machine Learning

2021 ◽  
Vol 2021 ◽  
pp. 1-8
Author(s):  
Malik Bader Alazzam ◽  
Fawaz Alassery ◽  
Ahmed Almulihi
Keyword(s):  
Machine Learning ◽  
Diabetic Retinopathy ◽  
Diagnostic Accuracy ◽  
Cross Sectional Study ◽  
Disease Classification ◽  
Ophthalmological Examination ◽  
Restricted Boltzmann Machines ◽  
Cross Sectional ◽  
Diabetic Retinopathy Screening

A cross-sectional study of patients with suspected diabetic retinopathy (DR) who had an ophthalmological examination and a retinal scan is the focus of this research. Specialized retinal images were analyzed and classified using OPF and RBM models (restricted Boltzmann machines). Classification of retinographs was based on the presence or absence of disease-related retinopathy (DR). The RBM and OPF models extracted 500 and 1000 characteristics from the images for disease classification after the system training phase for the recognition of retinopathy and normality patterns. There were a total of fifteen different experiment series, each with a repetition rate of 30 cycles. The study included 73 diabetics (a total of 122 eyes), with 50.7% of them being men and 49.3% being women. The population was on the older side, at 59.7 years old on average. The RBM-1000 had the highest overall diagnostic accuracy (89.47) of any of the devices evaluated. The RBM-500 had a better autodetection system for DR signals in fundus images than the competition (100% sensitivity). In terms of specificity, RBM-1000 and OPF-1000 correctly identified all of the images that lacked DR signs. In particular, the RBM model of machine learning automatic disease detection performed well in terms of diagnostic accuracy, sensitivity, and application in diabetic retinopathy screening.

scholarly journals Performance of deep neural network-based artificial intelligence method in diabetic retinopathy screening: a systematic review and meta-analysis of diagnostic test accuracy

2020 ◽  
Vol 183 (1) ◽  
pp. 41-49 ◽  
Author(s):  
Shirui Wang ◽  
Yuelun Zhang ◽  
Shubin Lei ◽  
Huijuan Zhu ◽  
Jianqiang Li ◽  
...  
Keyword(s):  
Neural Networks ◽  
Diabetic Retinopathy ◽  
Diagnostic Accuracy ◽  
Meta Analysis ◽  
Cochrane Library ◽  
Test Accuracy ◽  
Sample Sizes ◽  
Diabetic Retinopathy Screening ◽  
Retinopathy Screening ◽  
Training Sets

Objective Automatic diabetic retinopathy screening system based on neural networks has been used to detect diabetic retinopathy (DR). However, there is no quantitative synthesis of performance of these methods. We aimed to estimate the sensitivity and specificity of neural networks in DR grading. Methods Medline, Embase, IEEE Xplore, and Cochrane Library were searched up to 23 July 2019. Studies that evaluated performance of neural networks in detection of moderate or worse DR or diabetic macular edema using retinal fundus images with ophthalmologists’ judgment as reference standard were included. Two reviewers extracted data independently. Risk of bias of eligible studies was assessed using QUDAS-2 tool. Results Twenty-four studies involving 235 235 subjects were included. Quantitative random-effects meta-analysis using the Rutter and Gatsonis hierarchical summary receiver operating characteristics (HSROC) model revealed a pooled sensitivity of 91.9% (95% CI: 89.6% to 94.3%) and specificity of 91.3% (95% CI: 89.0% to 93.5%). Subgroup analyses and meta-regression did not provide any statistically significant findings for the heterogeneous diagnostic accuracy in studies with different image resolutions, sample sizes of training sets, architecture of convolutional neural networks, or diagnostic criteria. Conclusions State-of-the-art neural networks could effectively detect clinical significant DR. To further improve diagnostic accuracy of neural networks, researchers might need to develop new algorithms rather than simply enlarge sample sizes of training sets or optimize image quality.

Diagnostic test accuracy for use of machine learning in diagnosis of autism spectrum disorder: A Systematic Review and Meta-Analysis (Preprint)

2019 ◽  
Author(s):  
Sun Jae Moon ◽  
Jin Seub Hwang ◽  
Rajesh Kana ◽  
John Torous ◽  
Jung Won Kim
Keyword(s):  
Machine Learning ◽  
Systematic Review ◽  
Autism Spectrum Disorder ◽  
Meta Analysis ◽  
Learning Algorithms ◽  
Structural Mri ◽  
Autism Spectrum ◽  
Machine Learning Algorithms ◽  
Spectrum Disorder ◽  
Test Accuracy

BACKGROUND Over the recent years, machine learning algorithms have been more widely and increasingly applied in biomedical fields. In particular, its application has been drawing more attention in the field of psychiatry, for instance, as diagnostic tests/tools for autism spectrum disorder. However, given its complexity and potential clinical implications, there is ongoing need for further research on its accuracy. OBJECTIVE The current study aims to summarize the evidence for the accuracy of use of machine learning algorithms in diagnosing autism spectrum disorder (ASD) through systematic review and meta-analysis. METHODS MEDLINE, Embase, CINAHL Complete (with OpenDissertations), PsyINFO and IEEE Xplore Digital Library databases were searched on November 28th, 2018. Studies, which used a machine learning algorithm partially or fully in classifying ASD from controls and provided accuracy measures, were included in our analysis. Bivariate random effects model was applied to the pooled data in meta-analysis. Subgroup analysis was used to investigate and resolve the source of heterogeneity between studies. True-positive, false-positive, false negative and true-negative values from individual studies were used to calculate the pooled sensitivity and specificity values, draw SROC curves, and obtain area under the curve (AUC) and partial AUC. RESULTS A total of 43 studies were included for the final analysis, of which meta-analysis was performed on 40 studies (53 samples with 12,128 participants). A structural MRI subgroup meta-analysis (12 samples with 1,776 participants) showed the sensitivity at 0.83 (95% CI-0.76 to 0.89), specificity at 0.84 (95% CI -0.74 to 0.91), and AUC/pAUC at 0.90/0.83. An fMRI/deep neural network (DNN) subgroup meta-analysis (five samples with 1,345 participants) showed the sensitivity at 0.69 (95% CI- 0.62 to 0.75), the specificity at 0.66 (95% CI -0.61 to 0.70), and AUC/pAUC at 0.71/0.67. CONCLUSIONS Machine learning algorithms that used structural MRI features in diagnosis of ASD were shown to have accuracy that is similar to currently used diagnostic tools.

scholarly journals Accelerating organic solar cell material's discovery: high-throughput screening and big data

2021 ◽  
Author(s):  
Xabier Rodríguez-Martínez ◽  
Enrique Pascual-San-José ◽  
Mariano Campoy-Quiles
Keyword(s):  
Machine Learning ◽  
Big Data ◽  
High Throughput ◽  
Organic Solar Cells ◽  
High Throughput Screening ◽  
Organic Solar Cell ◽  
State Of The Art ◽  
Review Article ◽  
Machine Learning Algorithms ◽  
Device Optimization

This review article presents the state-of-the-art in high-throughput computational and experimental screening routines with application in organic solar cells, including materials discovery, device optimization and machine-learning algorithms.

scholarly journals Systematic literature review of machine learning methods used in the analysis of real-world data for patient-provider decision making

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Alan Brnabic ◽  
Lisa M. Hess
Keyword(s):  
Machine Learning ◽  
Decision Making ◽  
Literature Review ◽  
Systematic Literature Review ◽  
Real World ◽  
Learning Algorithms ◽  
External Validation ◽  
Machine Learning Algorithms ◽  
Learning Methods ◽  
Machine Learning Methods

Abstract Background Machine learning is a broad term encompassing a number of methods that allow the investigator to learn from the data. These methods may permit large real-world databases to be more rapidly translated to applications to inform patient-provider decision making. Methods This systematic literature review was conducted to identify published observational research of employed machine learning to inform decision making at the patient-provider level. The search strategy was implemented and studies meeting eligibility criteria were evaluated by two independent reviewers. Relevant data related to study design, statistical methods and strengths and limitations were identified; study quality was assessed using a modified version of the Luo checklist. Results A total of 34 publications from January 2014 to September 2020 were identified and evaluated for this review. There were diverse methods, statistical packages and approaches used across identified studies. The most common methods included decision tree and random forest approaches. Most studies applied internal validation but only two conducted external validation. Most studies utilized one algorithm, and only eight studies applied multiple machine learning algorithms to the data. Seven items on the Luo checklist failed to be met by more than 50% of published studies. Conclusions A wide variety of approaches, algorithms, statistical software, and validation strategies were employed in the application of machine learning methods to inform patient-provider decision making. There is a need to ensure that multiple machine learning approaches are used, the model selection strategy is clearly defined, and both internal and external validation are necessary to be sure that decisions for patient care are being made with the highest quality evidence. Future work should routinely employ ensemble methods incorporating multiple machine learning algorithms.

Sign in / Sign up

Export Citation Format

Share Document