Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study

ObjectiveThe aim of the study is to explore the potential value of S-Detect for residents-in-training, a computer-assisted diagnosis system based on deep learning (DL) algorithm.MethodsThe study was designed as a cross-sectional study. Routine breast ultrasound examinations were conducted by an experienced radiologist. The ultrasonic images of the lesions were retrospectively assessed by five residents-in-training according to the Breast Imaging Report and Data System (BI-RADS) lexicon, and a dichotomic classification of the lesions was provided by S-Detect. The diagnostic performances of S-Detect and the five residents were measured and compared using the pathological results as the gold standard. The category 4a lesions assessed by the residents were downgraded to possibly benign as classified by S-Detect. The diagnostic performance of the integrated results was compared with the original results of the residents.ParticipantsA total of 195 focal breast lesions were consecutively enrolled, including 82 malignant lesions and 113 benign lesions.ResultsS-Detect presented higher specificity (77.88%) and area under the curve (AUC) (0.82) than the residents (specificity: 19.47%–48.67%, AUC: 0.62–0.74). A total of 24, 31, 38, 32 and 42 identified as BI-RADS 4a lesions by residents 1, 2, 3, 4 and 5 were downgraded to possibly benign lesions by S-Detect, respectively. Among these downgraded lesions, 24, 28, 35, 30 and 40 lesions were proven to be pathologically benign, respectively. After combining the residents' results with the results of the software in category 4a lesions, the specificity and AUC of the five residents significantly improved (specificity: 46.02%–76.11%, AUC: 0.71–0.85, p<0.001). The intraclass correlation coefficient of the five residents also increased after integration (from 0.480 to 0.643).ConclusionsWith the help of the DL software, the specificity, overall diagnostic performance and interobserver agreement of the residents greatly improved. The software can be used as adjunctive tool for residents-in-training, downgrading 4a lesions to possibly benign and reducing unnecessary biopsies.

Download Full-text

Modality-Specific Deep Learning Model Ensembles Toward Improving TB Detection in Chest Radiographs

IEEE Access ◽

10.1109/access.2020.2971257 ◽

2020 ◽

Vol 8 ◽

pp. 27318-27326 ◽

Cited By ~ 6

Author(s):

Sivaramakrishnan Rajaraman ◽

Sameer K. Antani

Keyword(s):

Deep Learning ◽

Learning Model ◽

Chest Radiographs ◽

Model Ensembles ◽

Deep Learning Model

Download Full-text

Asbestos-related findings in chest radiographs of the male population of the county of Telemark, Norway--a cross-sectional study.

Scandinavian Journal of Work Environment & Health ◽

10.5271/sjweh.2105 ◽

1986 ◽

Vol 12 (6) ◽

pp. 567-573 ◽

Cited By ~ 10

Author(s):

Bjørn Hilt ◽

J T Lien ◽

P G Lund-Larsen ◽

K Lund ◽

S Langard

Keyword(s):

Cross Sectional Study ◽

Chest Radiographs ◽

Sectional Study ◽

Male Population ◽

Cross Sectional

Download Full-text

Moderate to severe OSA screening based on support vector machine of the Chinese population faciocervical measurements dataset: a cross-sectional study

BMJ Open ◽

10.1136/bmjopen-2020-048482 ◽

2021 ◽

Vol 11 (9) ◽

pp. e048482

Author(s):

Liu Zhang ◽

Ya Ru Yan ◽

Shi Qi Li ◽

Hong Peng Li ◽

Ying Ni Lin ◽

...

Keyword(s):

Machine Learning ◽

Chinese Population ◽

Learning Model ◽

Cross Sectional Study ◽

Neck Circumference ◽

Secondary Outcome ◽

Support Vector ◽

Sectional Study ◽

Cross Sectional ◽

Machine Learning Model

ObjectivesObstructive sleep apnoea (OSA) has received much attention as a risk factor for perioperative complications and 68.5% of OSA patients remain undiagnosed before surgery. Faciocervical characteristics may screen OSA for Asians due to smaller upper airways compared with Caucasians. Thus, our study aimed to explore a machine-learning model to screen moderate to severe OSA based on faciocervical and anthropometric measurements.DesignA cross-sectional study.SettingData were collected from the Shanghai Jiao Tong University School of Medicine affiliated Ruijin Hospital between February 2019 and August 2020.ParticipantsA total of 481 Chinese participants were included in the study.Primary and secondary outcome(1) Identification of moderate to severe OSA with apnoea–hypopnoea index 15 events/hour and (2) Verification of the machine-learning model.ResultsSex-Age-Body mass index (BMI)-maximum Interincisal distance-ratio of Height to thyrosternum distance-neck Circumference-waist Circumference (SABIHC2) model was set up. The SABIHC2 model could screen moderate to severe OSA with an area under the curve (AUC)=0.832, the sensitivity of 0.916 and specificity of 0.749, and performed better than the STOP-BANG (snoring, tiredness, observed apnea, high blood pressure, BMI, age, neck circumference, and male gender) questionnaire, which showed AUC=0.631, the sensitivity of 0.487 and specificity of 0.772. Especially for asymptomatic patients (Epworth Sleepiness Scale <10), the SABIHC2 model demonstrated better predictive ability compared with the STOP-BANG questionnaire, with AUC (0.824 vs 0.530), sensitivity (0.892 vs 0.348) and specificity (0.755 vs 0.809).ConclusionThe SABIHC2 machine-learning model provides a simple and accurate assessment of moderate to severe OSA in the Chinese population, especially for those without significant daytime sleepiness.

Download Full-text

Automated Radiology Alert System for Pneumothorax Detection on Chest Radiographs Improves Efficiency and Diagnostic Performance

Diagnostics ◽

10.3390/diagnostics11071182 ◽

2021 ◽

Vol 11 (7) ◽

pp. 1182

Author(s):

Cheng-Yi Kao ◽

Chiao-Yun Lin ◽

Cheng-Chen Chao ◽

Han-Sheng Huang ◽

Hsing-Yu Lee ◽

...

Keyword(s):

Deep Learning ◽

Diagnostic Performance ◽

Medical Center ◽

Characteristic Curve ◽

Learning Model ◽

Chest Radiographs ◽

Alert System ◽

Before And After ◽

Set Up ◽

Deep Learning Model

We aimed to set up an Automated Radiology Alert System (ARAS) for the detection of pneumothorax in chest radiographs by a deep learning model, and to compare its efficiency and diagnostic performance with the existing Manual Radiology Alert System (MRAS) at the tertiary medical center. This study retrospectively collected 1235 chest radiographs with pneumothorax labeling from 2013 to 2019, and 337 chest radiographs with negative findings in 2019 were separated into training and validation datasets for the deep learning model of ARAS. The efficiency before and after using the model was compared in terms of alert time and report time. During parallel running of the two systems from September to October 2020, chest radiographs prospectively acquired in the emergency department with age more than 6 years served as the testing dataset for comparison of diagnostic performance. The efficiency was improved after using the model, with mean alert time improving from 8.45 min to 0.69 min and the mean report time from 2.81 days to 1.59 days. The comparison of the diagnostic performance of both systems using 3739 chest radiographs acquired during parallel running showed that the ARAS was better than the MRAS as assessed in terms of sensitivity (recall), area under receiver operating characteristic curve, and F1 score (0.837 vs. 0.256, 0.914 vs. 0.628, and 0.754 vs. 0.407, respectively), but worse in terms of positive predictive value (PPV) (precision) (0.686 vs. 1.000). This study had successfully designed a deep learning model for pneumothorax detection on chest radiographs and set up an ARAS with improved efficiency and overall diagnostic performance.

Download Full-text

Detecting prodromal Alzheimer’s disease with MRI through deep learning

10.1101/813899 ◽

2019 ◽

Author(s):

Xinyang Feng ◽

Frank A. Provenzano ◽

Scott A. Small ◽

Keyword(s):

Deep Learning ◽

Large Scale ◽

Data Augmentation ◽

Hippocampal Formation ◽

Mechanistic Explanation ◽

Learning Model ◽

Tau Pathology ◽

Cross Sectional ◽

And Performance ◽

Deep Learning Model

ABSTRACTDeep learning applied to MRI for Alzheimer’s classification is hypothesized to improve if the deep learning model implicates disease’s pathophysiology. The challenge in testing this hypothesis is that large-scale data are required to train this type of model. Here, we overcome this challenge by using a novel data augmentation strategy and show that our MRI-based deep learning model classifies Alzheimer’s dementia with high accuracy. Moreover, a class activation map was found dominated by signal from the hippocampal formation, a site where Alzheimer’s pathophysiology begins. Next, we tested the model’s performance in prodromal Alzheimer’s when patients present with mild cognitive impairment (MCI). We retroactively dichotomized a large cohort of MCI patients who were followed for up to 10 years into those with and without prodromal Alzheimer’s at baseline and used the dementia-derived model to generate individual ‘deep learning MRI’ scores. We compared the two groups on these scores, and on other biomarkers of amyloid pathology, tau pathology, and neurodegeneration. The deep learning MRI scores outperformed nearly all other biomarkers, including—unexpectedly—biomarkers of amyloid or tau pathology, in classifying prodromal disease and in predicting clinical progression. Providing a mechanistic explanation, the deep learning MRI scores were found to be linked to regional tau pathology, through investigations using cross-sectional, longitudinal, premortem and postmortem data. Our findings validate that a disease’s known pathophysiology can improve the design and performance of deep learning models. Moreover, by showing that deep learning can extract useful biomarker information from conventional MRIs, the advantages of this model extend practically, potentially reducing patient burden, risk, and cost.

Download Full-text

A chest radiography-based artificial intelligence deep-learning model to predict severe Covid-19 patient outcomes: the CAPE (Covid-19 AI Predictive Engine) Model

10.1101/2020.05.25.20113084 ◽

2020 ◽

Author(s):

Charlene Liew ◽

Jessica Quah ◽

Han Leong Goh ◽

Narayan Venkataraman

Keyword(s):

Deep Learning ◽

Patient Outcomes ◽

Chest Radiography ◽

Scoring Systems ◽

Learning Model ◽

Chest Radiographs ◽

Icu Admission ◽

Risk Scoring ◽

Deep Learning Model ◽

Severe Patient

AbstractBackgroundChest radiography may be used together with deep-learning models to prognosticate COVID-19 patient outcomesPurposeT o evaluate the performance of a deep-learning model for the prediction of severe patient outcomes from COVID-19 pneumonia on chest radiographs.MethodsA deep-learning model (CAPE: Covid-19 AI Predictive Engine) was trained on 2337 CXR images including 2103 used only for validation while training. The prospective test set consisted of CXR images (n=70) obtained from RT-PCR confirmed COVID-19 pneumonia patients between 1 January and 30 April 2020 in a single center. The radiographs were analyzed by the AI model. Model performance was obtained by receiver operating characteristic curve analysis.ResultsIn the prospective test set, the mean age of the patients was 46 (+/-16.2) years (84.2% male). The deep-learning model accurately predicted outcomes of ICU admission/mortality from COVID-19 pneumonia with an AUC of 0.79 (95% CI 0.79-0.96). Compared to traditional risk scoring systems for pneumonia based upon laboratory and clinical parameters, the model matched the EWS and MulBTSA risk scoring systems and outperformed CURB-65.ConclusionsA deep-learning model was able to predict severe patient outcomes (ICU admission and mortality) from COVID-19 on chest radiographs.Key ResultsA deep-learning model was able to predict severe patient outcomes (ICU admission and mortality) from COVID-19 from chest radiographs with an AUC of 0.79, which is comparable to traditional risk scoring systems for pneumonia.Summary StatementThis is a chest radiography-based AI model to prognosticate the risk of severe COVID-19 pneumonia outcomes.

Download Full-text