Nuances of Interpreting X-ray Analysis by Deep Learning and Lessons for Reporting Experimental Findings

With the increase in the availability of annotated X-ray image data, there has been an accompanying and consequent increase in research on machine-learning-based, and ion particular deep-learning-based, X-ray image analysis. A major problem with this body of work lies in how newly proposed algorithms are evaluated. Usually, comparative analysis is reduced to the presentation of a single metric, often the area under the receiver operating characteristic curve (AUROC), which does not provide much clinical value or insight and thus fails to communicate the applicability of proposed models. In the present paper, we address this limitation of previous work by presenting a thorough analysis of a state-of-the-art learning approach and hence illuminate various weaknesses of similar algorithms in the literature, which have not yet been fully acknowledged and appreciated. Our analysis was performed on the ChestX-ray14 dataset, which has 14 lung disease labels and metainfo such as patient age, gender, and the relative X-ray direction. We examined the diagnostic significance of different metrics used in the literature including those proposed by the International Medical Device Regulators Forum, and present the qualitative assessment of the spatial information learned by the model. We show that models that have very similar AUROCs can exhibit widely differing clinical applicability. As a result, our work demonstrates the importance of detailed reporting and analysis of the performance of machine-learning approaches in this field, which is crucial both for progress in the field and the adoption of such models in practice.

Download Full-text

Machine learning techniques as an eﬃcient alternative diagnostic tool for COVID-19 cases

10.22514/sv.2021.110 ◽

2021 ◽

Keyword(s):

Machine Learning ◽

Deep Learning ◽

Sensitivity And Specificity ◽

Predictive Power ◽

Characteristic Curve ◽

False Negative ◽

Machine Learning Techniques ◽

Support Vector ◽

X Ray ◽

Learning Techniques

Background: The SARS-CoV-2 virus has demonstrated the weakness of many health systems worldwide, creating a saturation and lack of access to treatments. A bottleneck to fight this pandemic relates to the lack of diagnostic infrastructure for early detection of positive cases, particularly in rural and impoverished areas of developing countries. In this context, less costly and fast machine learning (ML) diagnosis-based systems are helpful. However, most of the research has focused on deep-learning techniques for diagnosis, which are computationally and technologically expensive. ML models have been mainly used as a benchmark and are not entirely explored in the existing literature on the topic of this paper. Objective: To analyze the capabilities of ML techniques (compared to deep learning) to diagnose COVID-19 cases based on X-ray images, assessing the performance of these techniques and using their predictive power for such a diagnosis. Methods: A factorial experiment was designed to establish this power with X-ray chest images of healthy, pneumonia, and COVID-19 infected patients. This design considers data-balancing methods, feature extraction approaches, different algorithms, and hyper-parameter optimization. The ML techniques were evaluated based on classification metrics, including accuracy, the area under the receiver operating characteristic curve (AUROC), F1-score, sensitivity, and specificity. Results: The design of experiment provided the mean and its confidence intervals for the predictive capability of different ML techniques, which reached AUROC values as high as 90% with suitable sensitivity and specificity. Among the learning algorithms, support vector machines and random forest performed best. The down-sampling method for unbalanced data improved the predictive power significantly for the images used in this study. Conclusions: Our investigation demonstrated that ML techniques are able to identify COVID-19 infected patients. The results provided suitable values of sensitivity and specificity, minimizing the false-positive or false-negative rates. The models were trained with significantly low computational resources, which helps to provide access and deployment in rural and impoverished areas.

Download Full-text

Using Machine Learning via Deep Learning Algorithms to Diagnose the Lung Disease Based on Chest Imaging: A Survey

International Journal of Interactive Mobile Technologies (iJIM) ◽

10.3991/ijim.v15i16.24191 ◽

2021 ◽

Vol 15 (16) ◽

pp. 95

Author(s):

Shaymaa Taha Ahmed ◽

Suhad Malallah Kadhem

Keyword(s):

Machine Learning ◽

Deep Learning ◽

Lung Disease ◽

Lung Diseases ◽

Learning Algorithms ◽

Learning Approaches ◽

Chest Imaging ◽

Training Set ◽

X Ray ◽

State Of Art

<p class="0abstract"><strong>—</strong> Chest imaging diagnostics is crucial in the medical area due to many serious lung diseases like cancers and nodules and particularly with the current pandemic of Covid-19. Machine learning approaches yield prominent results toward the task of diagnosis. Recently, deep learning methods are utilized and recommended by many studies in this domain. The research aims to critically examine the newest lung disease detection procedures using deep learning algorithms that use X-ray and CT scan datasets. Here, the most recent studies in this area (2015-2021) have been reviewed and summarized to provide an overview of the most appropriate methods that should be used or developed in future works, what limitations should be considered, and at what level these techniques help physicians in identifying the disease with better accuracy. The lack of various standard datasets, the huge training set, the high dimensionality of data, and the independence of features have been the main limitations based on the literature. However, different architectures of deep learning are used by many researchers but, Convolutional Neural Networks (CNN) are still state-of-art techniques in dealing with image datasets.</p>

Download Full-text

Artificial Intelligence and COVID-19 Using Chest CT Scan and Chest X-ray Images: Machine Learning and Deep Learning Approaches for Diagnosis and Treatment

Journal of Personalized Medicine ◽

10.3390/jpm11100993 ◽

2021 ◽

Vol 11 (10) ◽

pp. 993

Author(s):

Roberta Fusco ◽

Roberta Grassi ◽

Vincenza Granata ◽

Sergio Venanzio Setola ◽

Francesca Grassi ◽

...

Keyword(s):

Artificial Intelligence ◽

Machine Learning ◽

Deep Learning ◽

Standard Deviation ◽

Mean Value ◽

Learning Approaches ◽

Inclusion Criteria ◽

X Ray ◽

Standard Deviation Range ◽

Chest X Ray

Objective: To report an overview and update on Artificial Intelligence (AI) and COVID-19 using chest Computed Tomography (CT) scan and chest X-ray images (CXR). Machine Learning and Deep Learning Approaches for Diagnosis and Treatment were identified. Methods: Several electronic datasets were analyzed. The search covered the years from January 2019 to June 2021. The inclusion criteria were studied evaluating the use of AI methods in COVID-19 disease reporting performance results in terms of accuracy or precision or area under Receiver Operating Characteristic (ROC) curve (AUC). Results: Twenty-two studies met the inclusion criteria: 13 papers were based on AI in CXR and 10 based on AI in CT. The summarized mean value of the accuracy and precision of CXR in COVID-19 disease were 93.7% ± 10.0% of standard deviation (range 68.4–99.9%) and 95.7% ± 7.1% of standard deviation (range 83.0–100.0%), respectively. The summarized mean value of the accuracy and specificity of CT in COVID-19 disease were 89.1% ± 7.3% of standard deviation (range 78.0–99.9%) and 94.5 ± 6.4% of standard deviation (range 86.0–100.0%), respectively. No statistically significant difference in summarized accuracy mean value between CXR and CT was observed using the Chi square test (p value > 0.05). Conclusions: Summarized accuracy of the selected papers is high but there was an important variability; however, less in CT studies compared to CXR studies. Nonetheless, AI approaches could be used in the identification of disease clusters, monitoring of cases, prediction of the future outbreaks, mortality risk, COVID-19 diagnosis, and disease management.

Download Full-text

Interpretable Deep Learning for Spatial Analysis of Severe Hailstorms

Monthly Weather Review ◽

10.1175/mwr-d-18-0316.1 ◽

2019 ◽

Vol 147 (8) ◽

pp. 2827-2845 ◽

Cited By ~ 23

Author(s):

David John Gagne II ◽

Sue Ellen Haupt ◽

Douglas W. Nychka ◽

Gregory Thompson

Keyword(s):

Neural Network ◽

Machine Learning ◽

Deep Learning ◽

Convolutional Neural Network ◽

Characteristic Curve ◽

Weather Prediction ◽

Principal Component ◽

Receiver Operator Characteristic Curve ◽

Learning Approaches ◽

Learning Models

Abstract Deep learning models, such as convolutional neural networks, utilize multiple specialized layers to encode spatial patterns at different scales. In this study, deep learning models are compared with standard machine learning approaches on the task of predicting the probability of severe hail based on upper-air dynamic and thermodynamic fields from a convection-allowing numerical weather prediction model. The data for this study come from patches surrounding storms identified in NCAR convection-allowing ensemble runs from 3 May to 3 June 2016. The machine learning models are trained to predict whether the simulated surface hail size from the Thompson hail size diagnostic exceeds 25 mm over the hour following storm detection. A convolutional neural network is compared with logistic regressions using input variables derived from either the spatial means of each field or principal component analysis. The convolutional neural network statistically significantly outperforms all other methods in terms of Brier skill score and area under the receiver operator characteristic curve. Interpretation of the convolutional neural network through feature importance and feature optimization reveals that the network synthesized information about the environment and storm morphology that is consistent with our understanding of hail growth, including large lapse rates and a wind shear profile that favors wide updrafts. Different neurons in the network also record different storm modes, and the magnitude of the output of those neurons is used to analyze the spatiotemporal distributions of different storm modes in the NCAR ensemble.

Download Full-text

Machine and Deep Learning Approaches for Flooding Prevention in Distillation and Extraction Columns

10.22541/au.160660624.43119897/v1 ◽

2020 ◽

Author(s):

Jonas Oeing ◽

Laura Neuendorf ◽

Lukas Bittorf ◽

Waldemar Krieger ◽

Norbert Kockmann

Keyword(s):

Artificial Intelligence ◽

Machine Learning ◽

Deep Learning ◽

Image Data ◽

High Potential ◽

Process Industry ◽

Sensor Data ◽

Learning Approaches ◽

Separation Columns ◽

Module Type

Machine Learning (ML) algorithms can be combined with the modular automation protocol (MTP) and recognize the flooding behavior of laboratory fluids separation columns. Hence, artificial intelligence (AI) tools with deep learning (DL) offer a high potential for the process industry and allow to capture operating states that are otherwise difficult to detect or model. However, the advanced methods are only hesitantly applied in practice. This article provides an overview on how artificial intelligence-based algorithms can be implemented in existing laboratory plants. Process sensor data as well as image data are used to model the flooding behavior of distillation and extraction columns and the system is adapted to the existing modular automation standard of the Module Type Package (MTP).

Download Full-text

Early Detection of Coronavirus Cases Using Chest X-ray Images Employing Machine Learning and Deep Learning Approaches

10.1101/2020.06.07.20124594 ◽

2020 ◽

Author(s):

Khair Ahammed ◽

Md. Shahriare Satu ◽

Mohammad Zoynul Abedin ◽

Md. Auhidur Rahaman ◽

Sheikh Mohammed Shariful Islam

Keyword(s):

Machine Learning ◽

Deep Learning ◽

Random Sampling ◽

Learning Approaches ◽

X Ray ◽

Learning Procedure ◽

Community Transmission ◽

Chest X Ray ◽

Single Dataset ◽

F Measure

AbstractThis study aims to investigate if applying machine learning and deep learning approaches on chest X-ray images can detect cases of coronavirus. The chest X-ray datasets were obtained from Kaggle and Github and pre-processed into a single dataset using random sampling. We applied several machine learning and deep learning methods including Convolutional Neural Networks (CNN) along with classical machine learners. In deep learning procedure, several pre-trained models were also employed transfer learning in this dataset. Our proposed CNN model showed the highest accuracy (94.03%), AUC (95.52%), f-measure (94.03%), sensitivity (94.03%) and specificity (97.01%) as well as the lowest fall out (4.48%) and miss rate (2.98%) respectively. We also evaluated specificity and fall out rate along with accuracy to identify non-COVID-19 individuals more accurately. As a result, our new models might help to early detect COVID-19 patients and prevent community transmission compared to traditional methods.

Download Full-text

Deep Learning Approaches for Whiteboard Image Quality Enhancement

Color and Imaging Conference ◽

10.2352/j.imagingsci.technol.2019.63.4.040404 ◽

2019 ◽

Vol 2019 (1) ◽

pp. 360-368

Author(s):

Mekides Assefa Abebe ◽

Jon Yngve Hardeberg

Keyword(s):

Deep Learning ◽

Image Quality ◽

Image Data ◽

Quality Enhancement ◽

Network Architectures ◽

Learning Approaches ◽

Data Set ◽

Image Quality Enhancement ◽

Processing Techniques ◽

White Balancing

Different whiteboard image degradations highly reduce the legibility of pen-stroke content as well as the overall quality of the images. Consequently, different researchers addressed the problem through different image enhancement techniques. Most of the state-of-the-art approaches applied common image processing techniques such as background foreground segmentation, text extraction, contrast and color enhancements and white balancing. However, such types of conventional enhancement methods are incapable of recovering severely degraded pen-stroke contents and produce artifacts in the presence of complex pen-stroke illustrations. In order to surmount such problems, the authors have proposed a deep learning based solution. They have contributed a new whiteboard image data set and adopted two deep convolutional neural network architectures for whiteboard image quality enhancement applications. Their different evaluations of the trained models demonstrated their superior performances over the conventional methods.

Download Full-text

Integrating Machine/Deep Learning Methods and Filtering Techniques for Reliable Mineral Phase Segmentation of 3D X-ray Computed Tomography Images

Energies ◽

10.3390/en14154595 ◽

2021 ◽

Vol 14 (15) ◽

pp. 4595

Author(s):

Parisa Asadi ◽

Lauren E. Beckingham

Keyword(s):

Machine Learning ◽

Deep Learning ◽

Random Forest ◽

Ct Images ◽

Ct Imaging ◽

Learning Method ◽

Learning Methods ◽

X Ray ◽

Machine Learning Methods ◽

Filtering Techniques

X-ray CT imaging provides a 3D view of a sample and is a powerful tool for investigating the internal features of porous rock. Reliable phase segmentation in these images is highly necessary but, like any other digital rock imaging technique, is time-consuming, labor-intensive, and subjective. Combining 3D X-ray CT imaging with machine learning methods that can simultaneously consider several extracted features in addition to color attenuation, is a promising and powerful method for reliable phase segmentation. Machine learning-based phase segmentation of X-ray CT images enables faster data collection and interpretation than traditional methods. This study investigates the performance of several filtering techniques with three machine learning methods and a deep learning method to assess the potential for reliable feature extraction and pixel-level phase segmentation of X-ray CT images. Features were first extracted from images using well-known filters and from the second convolutional layer of the pre-trained VGG16 architecture. Then, K-means clustering, Random Forest, and Feed Forward Artificial Neural Network methods, as well as the modified U-Net model, were applied to the extracted input features. The models’ performances were then compared and contrasted to determine the influence of the machine learning method and input features on reliable phase segmentation. The results showed considering more dimensionality has promising results and all classification algorithms result in high accuracy ranging from 0.87 to 0.94. Feature-based Random Forest demonstrated the best performance among the machine learning models, with an accuracy of 0.88 for Mancos and 0.94 for Marcellus. The U-Net model with the linear combination of focal and dice loss also performed well with an accuracy of 0.91 and 0.93 for Mancos and Marcellus, respectively. In general, considering more features provided promising and reliable segmentation results that are valuable for analyzing the composition of dense samples, such as shales, which are significant unconventional reservoirs in oil recovery.

Download Full-text

A Review of Recent Deep Learning Approaches in Human-Centered Machine Learning

Sensors ◽

10.3390/s21072514 ◽

2021 ◽

Vol 21 (7) ◽

pp. 2514

Author(s):

Tharindu Kaluarachchi ◽

Andrew Reis ◽

Suranga Nanayakkara

Keyword(s):

Machine Learning ◽

Deep Learning ◽

Review Paper ◽

Learning Systems ◽

Learning Approaches ◽

Application Development ◽

Research Gaps ◽

Domain Experts ◽

Working Definition ◽

Real World Application

After Deep Learning (DL) regained popularity recently, the Artificial Intelligence (AI) or Machine Learning (ML) field is undergoing rapid growth concerning research and real-world application development. Deep Learning has generated complexities in algorithms, and researchers and users have raised concerns regarding the usability and adoptability of Deep Learning systems. These concerns, coupled with the increasing human-AI interactions, have created the emerging field that is Human-Centered Machine Learning (HCML). We present this review paper as an overview and analysis of existing work in HCML related to DL. Firstly, we collaborated with field domain experts to develop a working definition for HCML. Secondly, through a systematic literature review, we analyze and classify 162 publications that fall within HCML. Our classification is based on aspects including contribution type, application area, and focused human categories. Finally, we analyze the topology of the HCML landscape by identifying research gaps, highlighting conflicting interpretations, addressing current challenges, and presenting future HCML research opportunities.

Download Full-text

Detection of Malicious Software by Analyzing Distinct Artifacts Using Machine Learning and Deep Learning Algorithms

Electronics ◽

10.3390/electronics10141694 ◽

2021 ◽

Vol 10 (14) ◽

pp. 1694

Author(s):

Mathew Ashik ◽

A. Jyothish ◽

S. Anandaram ◽

P. Vinod ◽

Francesco Mercaldo ◽

...

Keyword(s):

Neural Network ◽

Machine Learning ◽

Deep Learning ◽

Support Vector ◽

Malware Analysis ◽

Learning Approaches ◽

Dynamic Features ◽

System Calls ◽

Prevention Methods ◽

Structural Aspects

Malware is one of the most significant threats in today’s computing world since the number of websites distributing malware is increasing at a rapid rate. Malware analysis and prevention methods are increasingly becoming necessary for computer systems connected to the Internet. This software exploits the system’s vulnerabilities to steal valuable information without the user’s knowledge, and stealthily send it to remote servers controlled by attackers. Traditionally, anti-malware products use signatures for detecting known malware. However, the signature-based method does not scale in detecting obfuscated and packed malware. Considering that the cause of a problem is often best understood by studying the structural aspects of a program like the mnemonics, instruction opcode, API Call, etc. In this paper, we investigate the relevance of the features of unpacked malicious and benign executables like mnemonics, instruction opcodes, and API to identify a feature that classifies the executable. Prominent features are extracted using Minimum Redundancy and Maximum Relevance (mRMR) and Analysis of Variance (ANOVA). Experiments were conducted on four datasets using machine learning and deep learning approaches such as Support Vector Machine (SVM), Naïve Bayes, J48, Random Forest (RF), and XGBoost. In addition, we also evaluate the performance of the collection of deep neural networks like Deep Dense network, One-Dimensional Convolutional Neural Network (1D-CNN), and CNN-LSTM in classifying unknown samples, and we observed promising results using APIs and system calls. On combining APIs/system calls with static features, a marginal performance improvement was attained comparing models trained only on dynamic features. Moreover, to improve accuracy, we implemented our solution using distinct deep learning methods and demonstrated a fine-tuned deep neural network that resulted in an F1-score of 99.1% and 98.48% on Dataset-2 and Dataset-3, respectively.

Download Full-text