Beyond Importance Scores: Interpreting Tabular ML by Visualizing Feature Semantics

Interpretability is becoming an active research topic as machine learning (ML) models are more widely used to make critical decisions. Tabular data are one of the most commonly used modes of data in diverse applications such as healthcare and finance. Much of the existing interpretability methods used for tabular data only report feature-importance scores—either locally (per example) or globally (per model)—but they do not provide interpretation or visualization of how the features interact. We address this limitation by introducing Feature Vectors, a new global interpretability method designed for tabular datasets. In addition to providing feature-importance, Feature Vectors discovers the inherent semantic relationship among features via an intuitive feature visualization technique. Our systematic experiments demonstrate the empirical utility of this new method by applying it to several real-world datasets. We further provide an easy-to-use Python package for Feature Vectors.

Analyzing artificial intelligence systems for the prediction of atrial fibrillation from sinus-rhythm ECGs including demographics and feature visualization

Atrial fibrillation (AF) is an abnormal heart rhythm, asymptomatic in many cases, that causes several health problems and mortality in population. This retrospective study evaluates the ability of different AI-based models to predict future episodes of AF from electrocardiograms (ECGs) recorded during normal sinus rhythm. Patients are divided into two classes according to AF occurrence or sinus rhythm permanence along their several ECGs registry. In the constrained scenario of balancing the age distributions between classes, our best AI model predicts future episodes of AF with area under the curve (AUC) 0.79 (0.72–0.86). Multiple scenarios and age-sex-specific groups of patients are considered, achieving best performance of prediction for males older than 70 years. These results point out the importance of considering different demographic groups in the analysis of AF prediction, showing considerable performance gaps among them. In addition to the demographic analysis, we apply feature visualization techniques to identify the most important portions of the ECG signals in the task of AF prediction, improving this way the interpretability and understanding of the AI models. These results and the simplicity of recording ECGs during check-ups add feasibility to clinical applications of AI-based models.

MCSNet: Channel Synergy-Based Human-Exoskeleton Interface With Surface Electromyogram

The human–robot interface (HRI) based on biological signals can realize the natural interaction between human and robot. It has been widely used in exoskeleton robots recently to help predict the wearer's movement. Surface electromyography (sEMG)-based HRI has mature applications on the exoskeleton. However, the sEMG signals of paraplegic patients' lower limbs are weak, which means that most HRI based on lower limb sEMG signals cannot be applied to the exoskeleton. Few studies have explored the possibility of using upper limb sEMG signals to predict lower limb movement. In addition, most HRIs do not consider the contribution and synergy of sEMG signal channels. This paper proposes a human–exoskeleton interface based on upper limb sEMG signals to predict lower limb movements of paraplegic patients. The interface constructs an channel synergy-based network (MCSNet) to extract the contribution and synergy of different feature channels. An sEMG data acquisition experiment is designed to verify the effectiveness of MCSNet. The experimental results show that our method has a good movement prediction performance in both within-subject and cross-subject situations, reaching an accuracy of 94.51 and 80.75%, respectively. Furthermore, feature visualization and model ablation analysis show that the features extracted by MCSNet are physiologically interpretable.

Spatial-Channel Attention-Based Class Activation Mapping for Interpreting CNN-Based Image Classification Models

Convolutional neural network (CNN) has been applied widely in various fields. However, it is always hindered by the unexplainable characteristics. Users cannot know why a CNN-based model produces certain recognition results, which is a vulnerability of CNN from the security perspective. To alleviate this problem, in this study, the three existing feature visualization methods of CNN are analyzed in detail firstly, and a unified visualization framework for interpreting the recognition results of CNN is presented. Here, class activation weight (CAW) is considered as the most important factor in the framework. Then, the different types of CAWs are further analyzed, and it is concluded that a linear correlation exists between them. Finally, on this basis, a spatial-channel attention-based class activation mapping (SCA-CAM) method is proposed. This method uses different types of CAWs as attention weights and combines spatial and channel attentions to generate class activation maps, which is capable of using richer features for interpreting the results of CNN. Experiments on four different networks are conducted. The results verify the linear correlation between different CAWs. In addition, compared with the existing methods, the proposed method SCA-CAM can effectively improve the visualization effect of the class activation map with higher flexibility on network structure.

Deep Learning Use for Differentiation of Low-grade vs High-Grade Glioma in Intraoperative Squash Smears

ObjectiveAutomated diagnosis using Artificial Intelligence (AI) techniques would be a useful addition to the intraoperative squash smear diagnosis. A robust diagnostic tool would enhance capabilities in centres where there is limited expertise for the diagnosis of intracranial lesions. The study aims to explore possibilities of deep learning technique-based models to classify squash smear images of glioma into high- and low-grade tumors.Methods500 Scanned images of squash smear were obtained intraoperatively and dataset was built. Image dataset was then pre-processed and fed into a CNN (Convolutional Neural Network) model for training and validation. The dataset consisted of 10,000 images of high (6000) and low (4000) grade gliomas, divided into three sets of training, validation and testing. ResultsCNN model based on deep learning algorithm was built and trained on training dataset to get accuracy of 96.2%. On a testing dataset which contains images previously unseen by trained model, it could achieve accuracies of 91% for diagnosing high grade glioma and 77% for low grade glioma. A positive predictive value of 86.6% and F1-score of 0.887 was achieved. Feature visualization technique was applied at the end to visualize regions of interest.ConclusionDeep Learning techniques can be applied as diagnostic tool if proper standardized images are obtained for reporting of squash smears of gliomas. The diagnostic accuracies of such tools can reach up to current standard diagnostic accuracies by conventional ways of reporting. Feature visualization techniques applied which can be used for rapid screening of slides or section of slide to assist in rapid diagnosis.