Validation of expert system enhanced deep learning algorithm for automated screening for COVID-Pneumonia on chest X-rays

AbstractSARS-CoV2 pandemic exposed the limitations of artificial intelligence based medical imaging systems. Earlier in the pandemic, the absence of sufficient training data prevented effective deep learning (DL) solutions for the diagnosis of COVID-19 based on X-Ray data. Here, addressing the lacunae in existing literature and algorithms with the paucity of initial training data; we describe CovBaseAI, an explainable tool using an ensemble of three DL models and an expert decision system (EDS) for COVID-Pneumonia diagnosis, trained entirely on pre-COVID-19 datasets. The performance and explainability of CovBaseAI was primarily validated on two independent datasets. Firstly, 1401 randomly selected CxR from an Indian quarantine center to assess effectiveness in excluding radiological COVID-Pneumonia requiring higher care. Second, curated dataset; 434 RT-PCR positive cases and 471 non-COVID/Normal historical scans, to assess performance in advanced medical settings. CovBaseAI had an accuracy of 87% with a negative predictive value of 98% in the quarantine-center data. However, sensitivity was 0.66–0.90 taking RT-PCR/radiologist opinion as ground truth. This work provides new insights on the usage of EDS with DL methods and the ability of algorithms to confidently predict COVID-Pneumonia while reinforcing the established learning; that benchmarking based on RT-PCR may not serve as reliable ground truth in radiological diagnosis. Such tools can pave the path for multi-modal high throughput detection of COVID-Pneumonia in screening and referral.

Download Full-text

Validation of expert system enhanced deep learning algorithm for automated screening for COVID-Pneumonia on chest X-rays

10.1101/2020.10.20.20213793 ◽

2020 ◽

Author(s):

Prashant Sadashiv Gidde ◽

Shyam Sunder Prasad ◽

Ajay Pratap Singh ◽

Nitin Bhatheja ◽

Satyartha Prakash ◽

...

Keyword(s):

Deep Learning ◽

Learning Algorithm ◽

Image Interpretation ◽

Ground Truth ◽

Training Data ◽

Screening Tools ◽

X Rays ◽

Rt Pcr ◽

Middle Income ◽

Deep Learning Algorithm

AbstractThe coronavirus disease of 2019 (COVID-19) pandemic exposed a limitation of artificial intelligence (AI) based medical image interpretation systems. Early in the pandemic, when need was greatest, the absence of sufficient training data prevented effective deep learning (DL) solutions. Even now, there is a need for Chest-X-ray (CxR) screening tools in low and middle income countries (LMIC), when RT-PCR is delayed, to exclude COVID-19 pneumonia (Cov-Pneum) requiring transfer to higher care. In absence of local LMIC data and poor portability of CxR DL algorithms, a new approach is needed. Axiomatically, it is faster to repurpose existing data than to generate new datasets. Here, we describe CovBaseAI, an explainable tool which uses an ensemble of three DL models and an expert decision system (EDS) for Cov-Pneum diagnosis, trained entirely on datasets from the pre-COVID-19 period. Portability, performance, and explainability of CovBaseAI was primarily validated on two independent datasets. First, 1401 randomly selected CxR from an Indian quarantine-center to assess effectiveness in excluding radiologic Cov-Pneum that may require higher care. Second, a curated dataset with 434 RT-PCR positive cases of varying levels of severity and 471 historical scans containing normal studies and non-COVID pathologies, to assess performance in advanced medical settings. CovBaseAI had accuracy of 87% with negative predictive value of 98% in the quarantine-center data for Cov-Pneum. However, sensitivity varied from 0.66 to 0.90 depending on whether RT-PCR or radiologist opinion was set as ground truth. This tool with explainability feature has better performance than publicly available algorithms trained on COVID-19 data but needs further improvement.

Download Full-text

Using artificial intelligence to assist radiologists in distinguishing COVID-19 from other pulmonary infections

Journal of X-Ray Science and Technology ◽

10.3233/xst-200735 ◽

2020 ◽

pp. 1-17

Author(s):

Yanhong Yang ◽

Fleming Y.M. Lure ◽

Hengyuan Miao ◽

Ziqi Zhang ◽

Stefan Jaeger ◽

...

Keyword(s):

Deep Learning ◽

Learning Algorithm ◽

Model Development ◽

Training Data ◽

Test Accuracy ◽

Pulmonary Infections ◽

Deep Learning Algorithm ◽

Average Accuracy ◽

Independent Test ◽

Comparable Performance

Background: Accurate and rapid diagnosis of coronavirus disease (COVID-19) is crucial for timely quarantine and treatment. Purpose: In this study, a deep learning algorithm-based AI model using ResUNet network was developed to evaluate the performance of radiologists with and without AI assistance in distinguishing COVID-19 infected pneumonia patients from other pulmonary infections on CT scans. Methods: For model development and validation, a total number of 694 cases with 111,066 CT slides were retrospectively collected as training data and independent test data in the study. Among them, 118 are confirmed COVID-19 infected pneumonia cases and 576 are other pulmonary infections cases (e.g. tuberculosis cases, common pneumonia cases and non-COVID-19 viral pneumonia cases). The cases were divided into training and testing datasets. The independent test was performed by evaluating and comparing the performance of three radiologists with different years of practice experience in distinguishing COVID-19 infected pneumonia cases with and without the AI assistance. Results: Our final model achieved an overall test accuracy of 0.914 with an area of the receiver operating characteristic (ROC) curve (AUC) of 0.903 in which the sensitivity and specificity are 0.918 and 0.909, respectively. The deep learning-based model then achieved a comparable performance by improving the radiologists’ performance in distinguish COVOD-19 from other pulmonary infections, yielding better average accuracy and sensitivity, from 0.941 to 0.951 and from 0.895 to 0.942, respectively, when compared to radiologists without using AI assistance. Conclusion: A deep learning algorithm-based AI model developed in this study successfully improved radiologists’ performance in distinguishing COVID-19 from other pulmonary infections using chest CT images.

Download Full-text

Performance of a deep learning CNN model for the automated detection of 13 common conditions on Chest X-rays

10.36227/techrxiv.16725727 ◽

2021 ◽

Author(s):

Tirupathi Karthik ◽

Vijayalakshmi Kasiraman ◽

Bhavani Paski ◽

Kashyap Gurram ◽

Amit Talwar ◽

...

Keyword(s):

Deep Learning ◽

Learning Algorithm ◽

Delayed Diagnosis ◽

Cost Effective ◽

Radiology Workflow ◽

X Rays ◽

Term Care ◽

X Ray ◽

Intelligence System ◽

Deep Learning Algorithm

Background and aims: Chest X-rays are widely used, non-invasive, cost effective imaging tests. However, the complexity of interpretation and global shortage of radiologists have led to reporting backlogs, delayed diagnosis and a compromised quality of care. A fully automated, reliable artificial intelligence system that can quickly triage abnormal images for urgent radiologist review would be invaluable in the clinical setting. The aim was to develop and validate a deep learning Convoluted Neural Network algorithm to automate the detection of 13 common abnormalities found on Chest X-rays. Method: In this retrospective study, a VGG 16 deep learning model was trained on images from the Chest-ray 14, a large publicly available Chest X-ray dataset, containing over 112,120 images with annotations. Images were split into training, validation and testing sets and trained to identify 13 specific abnormalities. The primary performance measures were accuracy and precision. Results: The model demonstrated an overall accuracy of 88% in the identification of abnormal X-rays and 87% in the detection of 13 common chest conditions with no model bias. Conclusion: This study demonstrates that a well-trained deep learning algorithm can accurately identify multiple abnormalities on X-ray images. As such models get further refined, they can be used to ease radiology workflow bottlenecks and improve reporting efficiency. Napier Healthcare’s team that developed this model consists of medical IT professionals who specialize in AI and its practical application in acute & long-term care settings. This is currently being piloted in a few hospitals and diagnostic labs on a commercial basis.

Download Full-text

TRACKING AND PREDICTING COVID-19 RADIOLOGICAL TRAJECTORY USING DEEP LEARNING ON CHEST X-RAYS: INITIAL ACCURACY TESTING

10.1101/2020.05.01.20086207 ◽

2020 ◽

Author(s):

S. Duchesne ◽

D. Gourdeau ◽

P. Archambault ◽

C. Chartrand-Lefebvre ◽

L. Dieumegarde ◽

...

Keyword(s):

Deep Learning ◽

Learning Algorithm ◽

Roc Curves ◽

Quantitative Information ◽

Lung Lesion ◽

Imaging Features ◽

X Rays ◽

Disease Trajectory ◽

Deep Learning Algorithm ◽

Deep Learning Features

ABSTRACTBackgroundDecision scores and ethically mindful algorithms are being established to adjudicate mechanical ventilation in the context of potential resources shortage due to the current onslaught of COVID-19 cases. There is a need for a reproducible and objective method to provide quantitative information for those scores.PurposeTowards this goal, we present a retrospective study testing the ability of a deep learning algorithm at extracting features from chest x-rays (CXR) to track and predict radiological evolution.Materials and MethodsWe trained a repurposed deep learning algorithm on the CheXnet open dataset (224,316 chest X-ray images of 65,240 unique patients) to extract features that mapped to radiological labels. We collected CXRs of COVID-19-positive patients from two open-source datasets (last accessed on April 9, 2020)(Italian Society for Medical and Interventional Radiology and MILA). Data collected form 60 pairs of sequential CXRs from 40 COVID patients (mean age ± standard deviation: 56 ± 13 years; 23 men, 10 women, seven not reported) and were categorized in three categories: “Worse”, “Stable”, or “Improved” on the basis of radiological evolution ascertained from images and reports. Receiver operating characteristic analyses, Mann-Whitney tests were performed.ResultsOn patients from the CheXnet dataset, the area under ROC curves ranged from 0.71 to 0.93 for seven imaging features and one diagnosis. Deep learning features between “Worse” and “Improved” outcome categories were significantly different for three radiological signs and one diagnostic (“Consolidation”, “Lung Lesion”, “Pleural effusion” and “Pneumonia”; all P < 0.05). Features from the first CXR of each pair could correctly predict the outcome category between “Worse” and “Improved” cases with 82.7% accuracy.ConclusionCXR deep learning features show promise for classifying the disease trajectory. Once validated in studies incorporating clinical data and with larger sample sizes, this information may be considered to inform triage decisions.

Download Full-text

Evaluasi Performasi Ruang Warna pada Klasifikasi Diabetic Retinophaty Menggunakan Convolution Neural Network

Jurnal Teknologi Informasi dan Ilmu Komputer ◽

10.25126/jtiik.2021834459 ◽

2021 ◽

Vol 8 (3) ◽

pp. 619

Author(s):

Candra Dewi ◽

Andri Santoso ◽

Indriati Indriati ◽

Nadia Artha Dewi ◽

Yoke Kusuma Arbawa

Keyword(s):

Neural Network ◽

Diabetic Retinopathy ◽

Deep Learning ◽

Convolutional Neural Network ◽

Learning Algorithm ◽

Color Space ◽

Training Data ◽

Color Spaces ◽

Deep Learning Algorithm ◽

Optimal Accuracy

Semakin meningkatnya jumlah penderita diabetes menjadi salah satu faktor penyebab semakin tingginya penderita penyakit diabetic retinophaty. Salah satu citra yang digunakan oleh dokter mata untuk mengidentifikasi diabetic retinophaty adalah foto retina. Dalam penelitian ini dilakukan pengenalan penyakit diabetic retinophaty secara otomatis menggunakan citra fundus retina dan algoritme Convolutional Neural Network (CNN) yang merupakan variasi dari algoritme Deep Learning. Kendala yang ditemukan dalam proses pengenalan adalah warna retina yang cenderung merah kekuningan sehingga ruang warna RGB tidak menghasilkan akurasi yang optimal. Oleh karena itu, dalam penelitian ini dilakukan pengujian pada berbagai ruang warna untuk mendapatkan hasil yang lebih baik. Dari hasil uji coba menggunakan 1000 data pada ruang warna RGB, HSI, YUV dan L*a*b* memberikan hasil yang kurang optimal pada data seimbang dimana akurasi terbaik masih dibawah 50%. Namun pada data tidak seimbang menghasilkan akurasi yang cukup tinggi yaitu 83,53% pada ruang warna YUV dengan pengujian pada data latih dan akurasi 74,40% dengan data uji pada semua ruang warna. AbstractIncreasing the number of people with diabetes is one of the factors causing the high number of people with diabetic retinopathy. One of the images used by ophthalmologists to identify diabetic retinopathy is a retinal photo. In this research, the identification of diabetic retinopathy is done automatically using retinal fundus images and the Convolutional Neural Network (CNN) algorithm, which is a variation of the Deep Learning algorithm. The obstacle found in the recognition process is the color of the retina which tends to be yellowish red so that the RGB color space does not produce optimal accuracy. Therefore, in this research, various color spaces were tested to get better results. From the results of trials using 1000 images data in the color space of RGB, HSI, YUV and L * a * b * give suboptimal results on balanced data where the best accuracy is still below 50%. However, the unbalanced data gives a fairly high accuracy of 83.53% with training data on the YUV color space and 74,40% with testing data on all color spaces.

Download Full-text

Deep learning algorithm evaluation of hypertension classification in less photoplethysmography signals conditions

Measurement and Control ◽

10.1177/00202940211001904 ◽

2021 ◽

Vol 54 (3-4) ◽

pp. 439-445

Author(s):

Chih-Ta Yen ◽

Sheng-Nan Chang ◽

Cheng-Hong Liao

Keyword(s):

Deep Learning ◽

Short Term Memory ◽

Learning Algorithm ◽

Training Data ◽

Short Term ◽

Deep Learning Algorithm ◽

Testing Dataset ◽

Data Points ◽

Optimal Classification ◽

Long Short Term Memory

This study used photoplethysmography signals to classify hypertensive into no hypertension, prehypertension, stage I hypertension, and stage II hypertension. There are four deep learning models are compared in the study. The difficulties in the study are how to find the optimal parameters such as kernel, kernel size, and layers in less photoplethysmographyt (PPG) training data condition. PPG signals were used to train deep residual network convolutional neural network (ResNetCNN) and bidirectional long short-term memory (BILSTM) to determine the optimal operating parameters when each dataset consisted of 2100 data points. During the experiment, the proportion of training and testing datasets was 8:2. The model demonstrated an optimal classification accuracy of 76% when the testing dataset was used.

Download Full-text

Classification of User Comment Using Word2vec and Deep Learning

International Journal of Emerging Technology and Advanced Engineering ◽

10.46338/ijetae0521_01 ◽

2021 ◽

Vol 11 (5) ◽

pp. 1-8

Author(s):

Rafly Indra Kurnia ◽

◽

Abba Suganda Girsang

Keyword(s):

Social Media ◽

Deep Learning ◽

User Satisfaction ◽

Rating Scale ◽

Short Term Memory ◽

Learning Algorithm ◽

Training Data ◽

Deep Learning Algorithm ◽

User Comments

This study will classify the text based on the rating of the provider application on the Google Play Store. This research is classification of user comments using Word2vec and the deep learning algorithm in this case is Long Short Term Memory (LSTM) based on the rating given with a rating scale of 1-5 with a detailed rating 1 is the lowest and rating 5 is the highest data and a rating scale of 1-3 with a detailed rating, 1 as a negative is a combination of ratings 1 and 2, rating 2 as a neutral is rating 3, and rating 3 as a positive is a combination of ratings 4 and 5 to get sentiment from users using SMOTE oversampling to handle the imbalance data. The data used are 16369 data. The training data and the testing data will be taken from user comments MyTelkomsel’s application from the play.google.com site where each comment has a rating in Indonesian Language. This review data will be very useful for companies to make business decisions. This data can be obtained from social media, but social media does not provide a rating feature for every user comment. This research goal is that data from social media such as Twitter or Facebook can also quickly find out the total of the user satisfaction based from the rating from the comment given. The best f1 scores and precisions obtained using 5 classes with LSTM and SMOTE were 0.62 and 0.70 and the best f1 scores and precisions obtained using 3 classes with LSTM and SMOTE were 0.86 and 0.87

Download Full-text

Performance of a deep-learning algorithm for referable thoracic abnormalities on chest radiographs: A multicenter study of a health screening cohort

PLoS ONE ◽

10.1371/journal.pone.0246472 ◽

2021 ◽

Vol 16 (2) ◽

pp. e0246472

Author(s):

Eun Young Kim ◽

Young Jae Kim ◽

Won-Jun Choi ◽

Gi Pyo Lee ◽

Ye Ra Choi ◽

...

Keyword(s):

Deep Learning ◽

Operating Characteristic ◽

Learning Algorithm ◽

Ground Truth ◽

Health Screening ◽

Health Care Clinics ◽

Radiologic Findings ◽

Deep Learning Algorithm ◽

Chest X Ray ◽

Good Stand

Purpose This study evaluated the performance of a commercially available deep-learning algorithm (DLA) (Insight CXR, Lunit, Seoul, South Korea) for referable thoracic abnormalities on chest X-ray (CXR) using a consecutively collected multicenter health screening cohort. Methods and materials A consecutive health screening cohort of participants who underwent both CXR and chest computed tomography (CT) within 1 month was retrospectively collected from three institutions’ health care clinics (n = 5,887). Referable thoracic abnormalities were defined as any radiologic findings requiring further diagnostic evaluation or management, including DLA-target lesions of nodule/mass, consolidation, or pneumothorax. We evaluated the diagnostic performance of the DLA for referable thoracic abnormalities using the area under the receiver operating characteristic (ROC) curve (AUC), sensitivity, and specificity using ground truth based on chest CT (CT-GT). In addition, for CT-GT-positive cases, three independent radiologist readings were performed on CXR and clear visible (when more than two radiologists called) and visible (at least one radiologist called) abnormalities were defined as CXR-GTs (clear visible CXR-GT and visible CXR-GT, respectively) to evaluate the performance of the DLA. Results Among 5,887 subjects (4,329 males; mean age 54±11 years), referable thoracic abnormalities were found in 618 (10.5%) based on CT-GT. DLA-target lesions were observed in 223 (4.0%), nodule/mass in 202 (3.4%), consolidation in 31 (0.5%), pneumothorax in one 1 (<0.1%), and DLA-non-target lesions in 409 (6.9%). For referable thoracic abnormalities based on CT-GT, the DLA showed an AUC of 0.771 (95% confidence interval [CI], 0.751–0.791), a sensitivity of 69.6%, and a specificity of 74.0%. Based on CXR-GT, the prevalence of referable thoracic abnormalities decreased, with visible and clear visible abnormalities found in 405 (6.9%) and 227 (3.9%) cases, respectively. The performance of the DLA increased significantly when using CXR-GTs, with an AUC of 0.839 (95% CI, 0.829–0.848), a sensitivity of 82.7%, and s specificity of 73.2% based on visible CXR-GT and an AUC of 0.872 (95% CI, 0.863–0.880, P <0.001 for the AUC comparison of GT-CT vs. clear visible CXR-GT), a sensitivity of 83.3%, and a specificity of 78.8% based on clear visible CXR-GT. Conclusion The DLA provided fair-to-good stand-alone performance for the detection of referable thoracic abnormalities in a multicenter consecutive health screening cohort. The DLA showed varied performance according to the different methods of ground truth.

Download Full-text

Explainable COVID-19 Detection on Chest X-rays Using an End-to-End Deep Convolutional Neural Network Architecture

Big Data and Cognitive Computing ◽

10.3390/bdcc5040073 ◽

2021 ◽

Vol 5 (4) ◽

pp. 73

Author(s):

Mohamed Chetoui ◽

Moulay A. Akhloufi ◽

Bardia Yousefi ◽

El Mostafa Bouattane

Keyword(s):

Deep Learning ◽

Medical Imaging ◽

Network Architecture ◽

Learning Algorithm ◽

Binary Classification ◽

X Rays ◽

Deep Learning Algorithm ◽

Binary Model ◽

Chest X Ray ◽

Multi Class Classification

The coronavirus pandemic is spreading around the world. Medical imaging modalities such as radiography play an important role in the fight against COVID-19. Deep learning (DL) techniques have been able to improve medical imaging tools and help radiologists to make clinical decisions for the diagnosis, monitoring and prognosis of different diseases. Computer-Aided Diagnostic (CAD) systems can improve work efficiency by precisely delineating infections in chest X-ray (CXR) images, thus facilitating subsequent quantification. CAD can also help automate the scanning process and reshape the workflow with minimal patient contact, providing the best protection for imaging technicians. The objective of this study is to develop a deep learning algorithm to detect COVID-19, pneumonia and normal cases on CXR images. We propose two classifications problems, (i) a binary classification to classify COVID-19 and normal cases and (ii) a multiclass classification for COVID-19, pneumonia and normal. Nine datasets and more than 3200 COVID-19 CXR images are used to assess the efficiency of the proposed technique. The model is trained on a subset of the National Institute of Health (NIH) dataset using swish activation, thus improving the training accuracy to detect COVID-19 and other pneumonia. The models are tested on eight merged datasets and on individual test sets in order to confirm the degree of generalization of the proposed algorithms. An explainability algorithm is also developed to visually show the location of the lung-infected areas detected by the model. Moreover, we provide a detailed analysis of the misclassified images. The obtained results achieve high performances with an Area Under Curve (AUC) of 0.97 for multi-class classification (COVID-19 vs. other pneumonia vs. normal) and 0.98 for the binary model (COVID-19 vs. normal). The average sensitivity and specificity are 0.97 and 0.98, respectively. The sensitivity of the COVID-19 class achieves 0.99. The results outperformed the comparable state-of-the-art models for the detection of COVID-19 on CXR images. The explainability model shows that our model is able to efficiently identify the signs of COVID-19.

Download Full-text

COVID-DETECT: A DEEP LEARNING APPROACH FOR CLASSIFICATION OF COVID-19 PNEUMONIA FROM LUNG SEGMENTED CHEST X-RAYS

Biomedical Engineering Applications Basis and Communications ◽

10.4015/s1016237221500101 ◽

2020 ◽

pp. 2150010

Author(s):

S. Rajkumar ◽

P. V. Rajaraman ◽

Haree Shankar Meganathan ◽

V. Sapthagirivasan ◽

K. Tejaswinee ◽

...

Keyword(s):

Deep Learning ◽

Learning Algorithm ◽

Rapid Screening ◽

X Rays ◽

X Ray ◽

Deep Learning Algorithm ◽

X Ray Imaging ◽

Before And After ◽

Wide Availability ◽

Novel Coronavirus

The novel coronavirus (COVID-19) was first reported in the Wuhan City of China in 2019 and became a pandemic. The outbreak has caused shocking effects to the people across the globe. It is important to screen a majority of the population in every country and for the respective governments to take appropriate action. There is a need for a rapid screening system to triage and recommend the patients for appropriate treatment. Chest X-ray imaging is one of the potential modalities, which has ample advantages such as wide availability even in the villages, portability, fast data sharing option from the point of capturing to the point of investigation, etc. The aim of the proposed work is to develop a deep learning algorithm for screening COVID-19 cases by leveraging the widely available X-ray imaging. We have built a deep learning Convolutional Neural Network model utilizing a combination of the public domain (open-source COVID-19) and private data (pneumonia and normal cases). The dataset was used before and after the segmentation of the lung region for training and testing. The outcome of the classification after lung segmentation resulted in significant superiority. The average accuracy achieved by the proposed system was 96%. The heat maps incorporated in the system were helpful for our radiologists to cross-verify whether the appropriate features are identified. This system (COVID-Detect) can be used in remote places in the countries affected by COVID-19 for mass screening of suspected cases and suggesting appropriate actions, such as recommending confirmatory tests.

Download Full-text