scholarly journals Brain metastasis detection using machine learning: a systematic review and meta-analysis

2020 ◽  
Author(s):  
Se Jin Cho ◽  
Leonard Sunwoo ◽  
Sung Hyun Baik ◽  
Yun Jung Bae ◽  
Byung Se Choi ◽  
...  
Keyword(s):  
Machine Learning ◽  
Brain Metastasis ◽  
False Positive ◽  
Data Augmentation ◽  
Meta Analysis ◽  
False Positive Rate ◽  
Relevant Literature ◽  
Inverse Variance ◽  
Positive Rate

Abstract Background Accurate detection of brain metastasis (BM) is important for cancer patients. We aimed to systematically review the performance and quality of machine-learning-based BM detection on MRI in the relevant literature. Methods A systematic literature search was performed for relevant studies reported before April 27, 2020. We assessed the quality of the studies using modified tailored questionnaires of the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) criteria and the Checklist for Artificial Intelligence in Medical Imaging (CLAIM). Pooled detectability was calculated using an inverse-variance weighting model. Results A total of 12 studies were included, which showed a clear transition from classical machine learning (cML) to deep learning (DL) after 2018. The studies on DL used a larger sample size than those on cML. The cML and DL groups also differed in the composition of the dataset, and technical details such as data augmentation. The pooled proportions of detectability of BM were 88.7% (95% CI, 84–93%) and 90.1% (95% CI, 84–95%) in the cML and DL groups, respectively. The false-positive rate per person was lower in the DL group than the cML group (10 vs 135, P < 0.001). In the patient selection domain of QUADAS-2, three studies (25%) were designated as high risk due to non-consecutive enrollment and arbitrary exclusion of nodules. Conclusion A comparable detectability of BM with a low false-positive rate per person was found in the DL group compared with the cML group. Improvements are required in terms of quality and study design.

Identification of and Correction for Publication Bias: Comment

2019 ◽  
Author(s):  
Amanda Kvarven ◽  
Eirik Strømland ◽  
Magnus Johannesson
Keyword(s):  
Publication Bias ◽  
False Positive ◽  
Large Scale ◽  
Meta Analysis ◽  
False Positive Rate ◽  
Effect Sizes ◽  
Replication Studies ◽  
Moderate Reduction ◽  
Positive Rate ◽  
Meta Analyses

Andrews & Kasy (2019) propose an approach for adjusting effect sizes in meta-analysis for publication bias. We use the Andrews-Kasy estimator to adjust the result of 15 meta-analyses and compare the adjusted results to 15 large-scale multiple labs replication studies estimating the same effects. The pre-registered replications provide precisely estimated effect sizes, which do not suffer from publication bias. The Andrews-Kasy approach leads to a moderate reduction of the inflated effect sizes in the meta-analyses. However, the approach still overestimates effect sizes by a factor of about two or more and has an estimated false positive rate of between 57% and 100%.

scholarly journals Identification of newborns at risk for autism using electronic medical records and machine learning

2019 ◽  
Author(s):  
Rayees Rahman ◽  
Arad Kodesh ◽  
Stephen Z Levine ◽  
Sven Sandin ◽  
Abraham Reichenberg ◽  
...  
Keyword(s):  
Machine Learning ◽  
Autism Spectrum Disorder ◽  
Positive Predictive Value ◽  
Electronic Medical Records ◽  
Predictive Value ◽  
False Positive ◽  
Medical Records ◽  
False Positive Rate ◽  
Autism Spectrum ◽  
Positive Rate

AbstractImportanceCurrent approaches for early identification of individuals at high risk for autism spectrum disorder (ASD) in the general population are limited, where most ASD patients are not identified until after the age of 4. This is despite substantial evidence suggesting that early diagnosis and intervention improves developmental course and outcome.ObjectiveDevelop a machine learning (ML) method predicting the diagnosis of ASD in offspring in a general population sample, using parental electronic medical records (EMR) available before childbirthDesignPrognostic study of EMR data within a single Israeli health maintenance organization, for the parents of 1,397 ASD children (ICD-9/10), and 94,741 non-ASD children born between January 1st, 1997 through December 31st, 2008. The complete EMR record of the parents was used to develop various ML models to predict the risk of having a child with ASD.Main outcomes and measuresRoutinely available parental sociodemographic information, medical histories and prescribed medications data until offspring’s birth were used to generate features to train various machine learning algorithms, including multivariate logistic regression, artificial neural networks, and random forest. Prediction performance was evaluated with 10-fold cross validation, by computing C statistics, sensitivity, specificity, accuracy, false positive rate, and precision (positive predictive value, PPV).ResultsAll ML models tested had similar performance, achieving an average C statistics of 0.70, sensitivity of 28.63%, specificity of 98.62%, accuracy of 96.05%, false positive rate of 1.37%, and positive predictive value of 45.85% for predicting ASD in this dataset.Conclusion and relevanceML algorithms combined with EMR capture early life ASD risk. Such approaches may be able to enhance the ability for accurate and efficient early detection of ASD in large populations of children.Key pointsQuestionCan autism risk in children be predicted using the pre-birth electronic medical record (EMR) of the parents?FindingsIn this population-based study that included 1,397 children with autism spectrum disorder (ASD) and 94,741 non-ASD children, we developed a machine learning classifier for predicting the likelihood of childhood diagnosis of ASD with an average C statistic of 0.70, sensitivity of 28.63%, specificity of 98.62%, accuracy of 96.05%, false positive rate of 1.37%, and positive predictive value of 45.85%.MeaningThe results presented serve as a proof-of-principle of the potential utility of EMR for the identification of a large proportion of future children at a high-risk of ASD.

scholarly journals Estimating the statistical performance of different approaches to meta-analysis of data from animal studies in identifying the impact of aspects of study design

2018 ◽  
Author(s):  
Qianying Wang ◽  
Jing Liao ◽  
Kaitlyn Hair ◽  
Alexandra Bannach-Brown ◽  
Zsanett Bahor ◽  
...  
Keyword(s):  
Effect Size ◽  
False Positive ◽  
Latent Variable ◽  
Statistical Power ◽  
Animal Studies ◽  
Meta Analysis ◽  
False Positive Rate ◽  
Mean Difference ◽  
Positive Rate ◽  
Meta Regression

AbstractBackgroundMeta-analysis is increasingly used to summarise the findings identified in systematic reviews of animal studies modelling human disease. Such reviews typically identify a large number of individually small studies, testing efficacy under a variety of conditions. This leads to substantial heterogeneity, and identifying potential sources of this heterogeneity is an important function of such analyses. However, the statistical performance of different approaches (normalised compared with standardised mean difference estimates of effect size; stratified meta-analysis compared with meta-regression) is not known.MethodsUsing data from 3116 experiments in focal cerebral ischaemia to construct a linear model predicting observed improvement in outcome contingent on 25 independent variables. We used stochastic simulation to attribute these variables to simulated studies according to their prevalence. To ascertain the ability to detect an effect of a given variable we introduced in addition this “variable of interest” of given prevalence and effect. To establish any impact of a latent variable on the apparent influence of the variable of interest we also introduced a “latent confounding variable” with given prevalence and effect, and allowed the prevalence of the variable of interest to be different in the presence and absence of the latent variable.ResultsGenerally, the normalised mean difference (NMD) approach had higher statistical power than the standardised mean difference (SMD) approach. Even when the effect size and the number of studies contributing to the meta-analysis was small, there was good statistical power to detect the overall effect, with a low false positive rate. For detecting an effect of the variable of interest, stratified meta-analysis was associated with a substantial false positive rate with NMD estimates of effect size, while using an SMD estimate of effect size had very low statistical power. Univariate and multivariable meta-regression performed substantially better, with low false positive rate for both NMD and SMD approaches; power was higher for NMD than for SMD. The presence or absence of a latent confounding variables only introduced an apparent effect of the variable of interest when there was substantial asymmetry in the prevalence of the variable of interest in the presence or absence of the confounding variable.ConclusionsIn meta-analysis of data from animal studies, NMD estimates of effect size should be used in preference to SMD estimates, and meta-regression should, where possible, be chosen over stratified meta-analysis. The power to detect the influence of the variable of interest depends on the effect of the variable of interest and its prevalence, but unless effects are very large adequate power is only achieved once at least 100 experiments are included in the meta-analysis.

Machine Learning for Automated Polyp Detection in Computed Tomography Colonography

2012 ◽  
pp. 830-850
Author(s):  
Abhilash Alexander Miranda ◽  
Olivier Caelen ◽  
Gianluca Bontempi
Keyword(s):  
Machine Learning ◽  
Computed Tomography ◽  
False Positive ◽  
False Positive Rate ◽  
Learning Algorithms ◽  
Colorectal Polyps ◽  
Machine Learning Algorithms ◽  
Computed Tomography Colonography ◽  
Positive Rate ◽  
Independent Features

This chapter presents a comprehensive scheme for automated detection of colorectal polyps in computed tomography colonography (CTC) with particular emphasis on robust learning algorithms that differentiate polyps from non-polyp shapes. The authors’ automated CTC scheme introduces two orientation independent features which encode the shape characteristics that aid in classification of polyps and non-polyps with high accuracy, low false positive rate, and low computations making the scheme suitable for colorectal cancer screening initiatives. Experiments using state-of-the-art machine learning algorithms viz., lazy learning, support vector machines, and naïve Bayes classifiers reveal the robustness of the two features in detecting polyps at 100% sensitivity for polyps with diameter greater than 10 mm while attaining total low false positive rates, respectively, of 3.05, 3.47 and 0.71 per CTC dataset at specificities above 99% when tested on 58 CTC datasets. The results were validated using colonoscopy reports provided by expert radiologists.

scholarly journals Identification of newborns at risk for autism using electronic medical records and machine learning

2020 ◽  
Vol 63 (1) ◽  
Author(s):  
Rayees Rahman ◽  
Arad Kodesh ◽  
Stephen Z. Levine ◽  
Sven Sandin ◽  
Abraham Reichenberg ◽  
...  
Keyword(s):  
Machine Learning ◽  
General Population ◽  
Electronic Medical Records ◽  
False Positive ◽  
Medical Records ◽  
False Positive Rate ◽  
Autism Spectrum ◽  
Health Maintenance ◽  
C Statistic ◽  
Positive Rate

Abstract Background. Current approaches for early identification of individuals at high risk for autism spectrum disorder (ASD) in the general population are limited, and most ASD patients are not identified until after the age of 4. This is despite substantial evidence suggesting that early diagnosis and intervention improves developmental course and outcome. The aim of the current study was to test the ability of machine learning (ML) models applied to electronic medical records (EMRs) to predict ASD early in life, in a general population sample. Methods. We used EMR data from a single Israeli Health Maintenance Organization, including EMR information for parents of 1,397 ASD children (ICD-9/10) and 94,741 non-ASD children born between January 1st, 1997 and December 31st, 2008. Routinely available parental sociodemographic information, parental medical histories, and prescribed medications data were used to generate features to train various ML algorithms, including multivariate logistic regression, artificial neural networks, and random forest. Prediction performance was evaluated with 10-fold cross-validation by computing the area under the receiver operating characteristic curve (AUC; C-statistic), sensitivity, specificity, accuracy, false positive rate, and precision (positive predictive value [PPV]). Results. All ML models tested had similar performance. The average performance across all models had C-statistic of 0.709, sensitivity of 29.93%, specificity of 98.18%, accuracy of 95.62%, false positive rate of 1.81%, and PPV of 43.35% for predicting ASD in this dataset. Conclusions. We conclude that ML algorithms combined with EMR capture early life ASD risk as well as reveal previously unknown features to be associated with ASD-risk. Such approaches may be able to enhance the ability for accurate and efficient early detection of ASD in large populations of children.

The Effects of Payoff on the Visual Processing of Dental Radiographs

1978 ◽  
Vol 20 (4) ◽  
pp. 385-390 ◽  
Author(s):  
William H. Mobley ◽  
Irwin L. Goldstein
Keyword(s):  
Visual Processing ◽  
False Positive ◽  
False Positive Rate ◽  
Dental Students ◽  
Technical Quality ◽  
Observer Error ◽  
Positive Rate ◽  
Dental Radiographs ◽  
Payoff Condition

The objective of the present study was to reassess the magnitude of observer error by dental students observing dental radiographs. Additionally, the effects of different payoff conditions on observer responses were examined. These data indicated high false positive rates as well as an inability to judge which radiographs were most difficult to assess. Further analyses indicated that false positive rates remained high even when payoff conditions penalized such responses. A payoff condition specifically designed to lower the false positive rate instead resulted in subjects lowering their hit rate and thus raising the miss rate. It is apparent that control of observer error must not only consider the technical quality of the radiograph but also the decision processes of the observer.

Application of Deep Learning to Detect Ground Deformation in InSAR Data

2020 ◽  
Author(s):  
Pui Anantrasirichai ◽  
Juliet Biggs ◽  
Fabien Albino ◽  
David Bull
Keyword(s):  
Deep Learning ◽  
False Positive ◽  
Data Augmentation ◽  
Ground Deformation ◽  
Signal To Noise Ratio ◽  
Matrix Completion ◽  
False Positive Rate ◽  
Synthetic Approach ◽  
Positive Rate ◽  
Machine Learning Applications

<p>Satellite interferometric synthetic aperture radar (InSAR) can be used for measuring surface deformation for a variety of applications. Recent satellite missions, such as Sentinel-1, produce a large amount of data, meaning that visual inspection is impractical. Here we use deep learning, which has proved successful at object detection, to overcome this problem. Initially we present the use of convolutional neural networks (CNNs) for detecting rapid deformation events, which we test on a global dataset of over 30,000 wrapped interferograms at 900 volcanoes. We compare two potential training datasets: data augmentation applied to archive examples and synthetic models. Both are able to detect true positive results, but the data augmentation approach has a false positive rate of 0.205% and the synthetic approach has a false positive rate of 0.036%.  Then, I will present an enhanced technique for measuring slow, sustained deformation over a range of scales from volcanic unrest to urban sources of deformation such as coalfields. By rewrapping cumulative time series, the detection performance is improved when the deformation rate is slow, as more fringes are generated without altering the signal to noise ratio. We adapt the method to use persistent scatterer InSAR data, which is sparse in nature,  by using spatial interpolation methods such as modified matrix completion Finally, future perspectives for machine learning applications on InSAR data will be discussed.</p>

scholarly journals Data Mining Validation of Fluconazole Breakpoints Established by the European Committee on Antimicrobial Susceptibility Testing

2009 ◽  
Vol 53 (7) ◽  
pp. 2949-2954 ◽  
Author(s):  
Isabel Cuesta ◽  
Concha Bielza ◽  
Pedro Larrañaga ◽  
Manuel Cuenca-Estrella ◽  
Fernando Laguna ◽  
...  
Keyword(s):  
Machine Learning ◽  
Antimicrobial Susceptibility ◽  
Roc Curve ◽  
False Positive ◽  
Statistical Power ◽  
Susceptibility Testing ◽  
False Positive Rate ◽  
Antimicrobial Susceptibility Testing ◽  
European Committee ◽  
Positive Rate

ABSTRACT European Committee on Antimicrobial Susceptibility Testing (EUCAST) breakpoints classify Candida strains with a fluconazole MIC ≤ 2 mg/liter as susceptible, those with a fluconazole MIC of 4 mg/liter as representing intermediate susceptibility, and those with a fluconazole MIC > 4 mg/liter as resistant. Machine learning models are supported by complex statistical analyses assessing whether the results have statistical relevance. The aim of this work was to use supervised classification algorithms to analyze the clinical data used to produce EUCAST fluconazole breakpoints. Five supervised classifiers (J48, Correlation and Regression Trees [CART], OneR, Naïve Bayes, and Simple Logistic) were used to analyze two cohorts of patients with oropharyngeal candidosis and candidemia. The target variable was the outcome of the infections, and the predictor variables consisted of values for the MIC or the proportion between the dose administered and the MIC of the isolate (dose/MIC). Statistical power was assessed by determining values for sensitivity and specificity, the false-positive rate, the area under the receiver operating characteristic (ROC) curve, and the Matthews correlation coefficient (MCC). CART obtained the best statistical power for a MIC > 4 mg/liter for detecting failures (sensitivity, 87%; false-positive rate, 8%; area under the ROC curve, 0.89; MCC index, 0.80). For dose/MIC determinations, the target was >75, with a sensitivity of 91%, a false-positive rate of 10%, an area under the ROC curve of 0.90, and an MCC index of 0.80. Other classifiers gave similar breakpoints with lower statistical power. EUCAST fluconazole breakpoints have been validated by means of machine learning methods. These computer tools must be incorporated in the process for developing breakpoints to avoid researcher bias, thus enhancing the statistical power of the model.

OS08.6.A Glioblastoma treatment response machine learning monitoring biomarkers: a systematic review and meta-analysis

2021 ◽  
Vol 23 (Supplement_2) ◽  
pp. ii11-ii12
Author(s):  
T C Booth ◽  
A Chelliah ◽  
A Roman ◽  
A Al Busaidi ◽  
H Shuaib ◽  
...  
Keyword(s):  
Machine Learning ◽  
Treatment Response ◽  
Meta Analysis ◽  
False Positive Rate ◽  
Risk Of Bias ◽  
Response Monitoring ◽  
Reference Standard ◽  
Positive Rate ◽  
Test Sets

Abstract BACKGROUND The aim of the systematic review was to assess recently published studies on diagnostic test accuracy of glioblastoma treatment response monitoring biomarkers in adults, developed through machine learning (ML). MATERIAL AND METHODS PRISMA methodology was followed. Articles published 09/2018-01/2021 (since previous reviews) were searched for using MEDLINE, EMBASE, and the Cochrane Register by two reviewers independently. Included study participants were adult patients with high grade glioma who had undergone standard treatment (maximal resection, radiotherapy with concomitant and adjuvant temozolomide) and subsequently underwent follow-up imaging to determine treatment response status (specifically, distinguishing progression/recurrence from progression/recurrence mimics - the target condition). Risk of bias and applicability was assessed with QUADAS 2. A third reviewer arbitrated any discrepancy. Contingency tables were created for hold-out test sets and recall, specificity, precision, F1-score, balanced accuracy calculated. A meta-analysis was performed using a bivariate model for recall, false positive rate and area-under the receiver operator characteristic curve (AUC). RESULTS Eighteen studies were included with 1335 patients in training sets and 384 in test sets. To determine whether there was progression or a mimic, the reference standard combination of follow-up imaging and histopathology at re-operation was applied in 67% (13/18) of studies. The small numbers of patient included in studies, the high risk of bias and concerns of applicability in the study designs (particularly in relation to the reference standard and patient selection due to confounding), and the low level of evidence, suggest that limited conclusions can be drawn from the data. Ten studies (10/18, 56%) had internal or external hold-out test set data that could be included in a meta-analysis of monitoring biomarker studies. The pooled sensitivity was 0.77 (0.65–0.86). The pooled false positive rate (1-specificity) was 0.35 (0.25–0.47). The summary point estimate for the AUC was 0.77. CONCLUSION There is likely good diagnostic performance of machine learning models that use MRI features to distinguish between progression and mimics. The diagnostic performance of ML using implicit features did not appear to be superior to ML using explicit features. There are a range of ML-based solutions poised to become treatment response monitoring biomarkers for glioblastoma. To achieve this, the development and validation of ML models require large, well-annotated datasets where the potential for confounding in the study design has been carefully considered. Therefore, multidisciplinary efforts and multicentre collaborations are necessary.

Sign in / Sign up

Export Citation Format

Share Document