scholarly journals CrabNet for explainable deep learning in materials science: bridging the gap between academia and industry

2021 ◽  
Author(s):  
Anthony Wang ◽  
Mahamad Salah Mahmoud ◽  
Mathias Czasny ◽  
Aleksander Gurlo
Keyword(s):  
Deep Learning ◽  
Materials Science ◽  
Chemical Space ◽  
Academic Research ◽  
Model Performance ◽  
Materials Informatics ◽  
Specific Element ◽  
Model Interpretation ◽  
Meaningful Material ◽  
Decision Making Processes

Despite recent breakthroughs in deep learning for materials informatics, there exists a disparity between their popularity in academic research and their limited adoption in the industry. A significant contributor to this “interpretability-adoption gap” is the prevalence of black-box models and the lack of built-in methods for model interpretation. While established methods for evaluating model performance exist, an intuitive understanding of the modeling and decision-making processes in models is nonetheless desired in many cases. In this work, we demonstrate several ways of incorporating model interpretability to the structure-agnostic Compositionally Restricted Attention-Based network, CrabNet. We show that CrabNet learns meaningful, material property-specific element representations based solely on the data with no additional supervision. These element representations can then be used to explore element identity, similarity, behavior, and interactions within different chemical environments. Chemical compounds can also be uniquely represented and examined to reveal clear structures and trends within the chemical space. Additionally, visualizations of the attention mechanism can be used in conjunction to further understand the modeling process, identify potential modeling or dataset errors, and hint at further chemical insights leading to a better understanding of the phenomena governing material properties. We feel confident that the interpretability methods introduced in this work for CrabNet will be of keen interest to materials informatics researchers as well as industrial practitioners alike.

scholarly journals Enabling deeper learning on big data for materials informatics applications

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Dipendra Jha ◽  
Vishu Gupta ◽  
Logan Ward ◽  
Zijiang Yang ◽  
Christopher Wolverton ◽  
...  
Keyword(s):  
Neural Networks ◽  
Big Data ◽  
Deep Learning ◽  
Deep Neural Networks ◽  
Materials Science ◽  
Prediction Models ◽  
Model Performance ◽  
Materials Informatics ◽  
Learning Framework ◽  
Significant Attention

AbstractThe application of machine learning (ML) techniques in materials science has attracted significant attention in recent years, due to their impressive ability to efficiently extract data-driven linkages from various input materials representations to their output properties. While the application of traditional ML techniques has become quite ubiquitous, there have been limited applications of more advanced deep learning (DL) techniques, primarily because big materials datasets are relatively rare. Given the demonstrated potential and advantages of DL and the increasing availability of big materials datasets, it is attractive to go for deeper neural networks in a bid to boost model performance, but in reality, it leads to performance degradation due to the vanishing gradient problem. In this paper, we address the question of how to enable deeper learning for cases where big materials data is available. Here, we present a general deep learning framework based on Individual Residual learning (IRNet) composed of very deep neural networks that can work with any vector-based materials representation as input to build accurate property prediction models. We find that the proposed IRNet models can not only successfully alleviate the vanishing gradient problem and enable deeper learning, but also lead to significantly (up to 47%) better model accuracy as compared to plain deep neural networks and traditional ML techniques for a given input materials representation in the presence of big data.

scholarly journals Deep materials informatics: Applications of deep learning in materials science

2019 ◽  
Vol 9 (3) ◽  
pp. 779-792 ◽  
Author(s):  
Ankit Agrawal ◽  
Alok Choudhary
Keyword(s):  
Deep Learning ◽  
Materials Science ◽  
Materials Informatics ◽  
Image Position

Abstract

scholarly journals Effectiveness of transfer learning for enhancing tumor classification with a convolutional neural network on frozen sections

2020 ◽  
Vol 10 (1) ◽  
Author(s):  
Young-Gon Kim ◽  
Sungchul Kim ◽  
Cristina Eunbee Cho ◽  
In Hye Song ◽  
Hee Jin Lee ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Transfer Learning ◽  
Frozen Section ◽  
Medical Center ◽  
External Validation ◽  
Model Performance ◽  
Classification Model ◽  
Training Dataset

AbstractFast and accurate confirmation of metastasis on the frozen tissue section of intraoperative sentinel lymph node biopsy is an essential tool for critical surgical decisions. However, accurate diagnosis by pathologists is difficult within the time limitations. Training a robust and accurate deep learning model is also difficult owing to the limited number of frozen datasets with high quality labels. To overcome these issues, we validated the effectiveness of transfer learning from CAMELYON16 to improve performance of the convolutional neural network (CNN)-based classification model on our frozen dataset (N = 297) from Asan Medical Center (AMC). Among the 297 whole slide images (WSIs), 157 and 40 WSIs were used to train deep learning models with different dataset ratios at 2, 4, 8, 20, 40, and 100%. The remaining, i.e., 100 WSIs, were used to validate model performance in terms of patch- and slide-level classification. An additional 228 WSIs from Seoul National University Bundang Hospital (SNUBH) were used as an external validation. Three initial weights, i.e., scratch-based (random initialization), ImageNet-based, and CAMELYON16-based models were used to validate their effectiveness in external validation. In the patch-level classification results on the AMC dataset, CAMELYON16-based models trained with a small dataset (up to 40%, i.e., 62 WSIs) showed a significantly higher area under the curve (AUC) of 0.929 than those of the scratch- and ImageNet-based models at 0.897 and 0.919, respectively, while CAMELYON16-based and ImageNet-based models trained with 100% of the training dataset showed comparable AUCs at 0.944 and 0.943, respectively. For the external validation, CAMELYON16-based models showed higher AUCs than those of the scratch- and ImageNet-based models. Model performance for slide feasibility of the transfer learning to enhance model performance was validated in the case of frozen section datasets with limited numbers.

scholarly journals Quality control stress test for deep learning-based diagnostic model in digital pathology

2021 ◽  
Author(s):  
Birgid Schömig-Markiefka ◽  
Alexey Pryalukhin ◽  
Wolfgang Hulla ◽  
Andrey Bychkov ◽  
Junya Fukuoka ◽  
...  
Keyword(s):  
Deep Learning ◽  
Cancer Detection ◽  
Stress Testing ◽  
Computational Analysis ◽  
Stress Test ◽  
Digital Pathology ◽  
Model Performance ◽  
Diagnostic Model ◽  
Model Accuracy ◽  
Diagnostic Models

AbstractDigital pathology provides a possibility for computational analysis of histological slides and automatization of routine pathological tasks. Histological slides are very heterogeneous concerning staining, sections’ thickness, and artifacts arising during tissue processing, cutting, staining, and digitization. In this study, we digitally reproduce major types of artifacts. Using six datasets from four different institutions digitized by different scanner systems, we systematically explore artifacts’ influence on the accuracy of the pre-trained, validated, deep learning-based model for prostate cancer detection in histological slides. We provide evidence that any histological artifact dependent on severity can lead to a substantial loss in model performance. Strategies for the prevention of diagnostic model accuracy losses in the context of artifacts are warranted. Stress-testing of diagnostic models using synthetically generated artifacts might be an essential step during clinical validation of deep learning-based algorithms.

scholarly journals A deep-learning method using computed tomography scout images for estimating patient body weight

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Shota Ichikawa ◽  
Misaki Hamada ◽  
Hiroyuki Sugimori
Keyword(s):  
Computed Tomography ◽  
Body Weight ◽  
Deep Learning ◽  
Contrast Medium ◽  
Model Performance ◽  
Ct Scanning ◽  
Drug Dosing ◽  
Body Weights ◽  
Dose Management ◽  
Medium Dose

AbstractBody weight is an indispensable parameter for determination of contrast medium dose, appropriate drug dosing, or management of radiation dose. However, we cannot always determine the accurate patient body weight at the time of computed tomography (CT) scanning, especially in emergency care. Time-efficient methods to estimate body weight with high accuracy before diagnostic CT scans currently do not exist. In this study, on the basis of 1831 chest and 519 abdominal CT scout images with the corresponding body weights, we developed and evaluated deep-learning models capable of automatically predicting body weight from CT scout images. In the model performance assessment, there were strong correlations between the actual and predicted body weights in both chest (ρ = 0.947, p < 0.001) and abdominal datasets (ρ = 0.869, p < 0.001). The mean absolute errors were 2.75 kg and 4.77 kg for the chest and abdominal datasets, respectively. Our proposed method with deep learning is useful for estimating body weights from CT scout images with clinically acceptable accuracy and potentially could be useful for determining the contrast medium dose and CT dose management in adult patients with unknown body weight.

scholarly journals Deep Learning Can Differentiate IDH-Mutant from IDH-Wild GBM

2021 ◽  
Vol 11 (4) ◽  
pp. 290
Author(s):  
Luca Pasquini ◽  
Antonio Napolitano ◽  
Emanuela Tagliente ◽  
Francesco Dellepiane ◽  
Martina Lucignani ◽  
...  
Keyword(s):  
Deep Learning ◽  
Hypoxia Inducible Factor ◽  
Model Performance ◽  
High Grade Glioma ◽  
Learning Model ◽  
Dense Layer ◽  
Idh Mutation ◽  
Diagnostic Challenge ◽  
Who Grade ◽  
Deep Learning Model

Isocitrate dehydrogenase (IDH) mutant and wildtype glioblastoma multiforme (GBM) often show overlapping features on magnetic resonance imaging (MRI), representing a diagnostic challenge. Deep learning showed promising results for IDH identification in mixed low/high grade glioma populations; however, a GBM-specific model is still lacking in the literature. Our aim was to develop a GBM-tailored deep-learning model for IDH prediction by applying convoluted neural networks (CNN) on multiparametric MRI. We selected 100 adult patients with pathologically demonstrated WHO grade IV gliomas and IDH testing. MRI sequences included: MPRAGE, T1, T2, FLAIR, rCBV and ADC. The model consisted of a 4-block 2D CNN, applied to each MRI sequence. Probability of IDH mutation was obtained from the last dense layer of a softmax activation function. Model performance was evaluated in the test cohort considering categorical cross-entropy loss (CCEL) and accuracy. Calculated performance was: rCBV (accuracy 83%, CCEL 0.64), T1 (accuracy 77%, CCEL 1.4), FLAIR (accuracy 77%, CCEL 1.98), T2 (accuracy 67%, CCEL 2.41), MPRAGE (accuracy 66%, CCEL 2.55). Lower performance was achieved on ADC maps. We present a GBM-specific deep-learning model for IDH mutation prediction, with a maximal accuracy of 83% on rCBV maps. Highest predictivity achieved on perfusion images possibly reflects the known link between IDH and neoangiogenesis through the hypoxia inducible factor.

scholarly journals Deep Learning Versus Logistic Regression to Predict Opioid Dose Reduction After Spinal Cord Stimulation

Neurosurgery ◽  
2020 ◽  
Vol 67 (Supplement_1) ◽  
Author(s):  
Syed M Adil ◽  
Lefko T Charalambous ◽  
Kelly R Murphy ◽  
Shervin Rahimpour ◽  
Stephen C Harward ◽  
...  
Keyword(s):  
Neural Network ◽  
Spinal Cord ◽  
Deep Learning ◽  
Dose Reduction ◽  
Spinal Cord Stimulation ◽  
Network Architecture ◽  
Characteristic Curve ◽  
Model Performance ◽  
Health Crisis ◽  
Opioid Dose

Abstract INTRODUCTION Opioid misuse persists as a public health crisis affecting approximately one in four Americans.1 Spinal cord stimulation (SCS) is a neuromodulation strategy to treat chronic pain, with one goal being decreased opioid consumption. Accurate prognostication about SCS success is key in optimizing surgical decision making for both physicians and patients. Deep learning, using neural network models such as the multilayer perceptron (MLP), enables accurate prediction of non-linear patterns and has widespread applications in healthcare. METHODS The IBM MarketScan® (IBM) database was queried for all patients ≥ 18 years old undergoing SCS from January 2010 to December 2015. Patients were categorized into opioid dose groups as follows: No Use, ≤ 20 morphine milligram equivalents (MME), 20–50 MME, 50–90 MME, and &gt;90 MME. We defined “opiate weaning” as moving into a lower opioid dose group (or remaining in the No Use group) during the 12 months following permanent SCS implantation. After pre-processing, there were 62 predictors spanning demographics, comorbidities, and pain medication history. We compared an MLP with four hidden layers to the LR model with L1 regularization. Model performance was assessed using area under the receiver operating characteristic curve (AUC) with 5-fold nested cross-validation. RESULTS Ultimately, 6,124 patients were included, of which 77% had used opioids for &gt;90 days within the 1-year pre-SCS and 72% had used &gt;5 types of medications during the 90 days prior to SCS. The mean age was 56 ± 13 years old. Collectively, 2,037 (33%) patients experienced opiate weaning. The AUC was 0.74 for the MLP and 0.73 for the LR model. CONCLUSION To our knowledge, we present the first use of deep learning to predict opioid weaning after SCS. Model performance was slightly better than regularized LR. Future efforts should focus on optimization of neural network architecture and hyperparameters to further improve model performance. Models should also be calibrated and externally validated on an independent dataset. Ultimately, such tools may assist both physicians and patients in predicting opioid dose reduction after SCS.

scholarly journals Deep Learning for Monitoring Agricultural Drought in South Asia Using Remote Sensing Data

2021 ◽  
Vol 13 (9) ◽  
pp. 1715
Author(s):  
Foyez Ahmed Prodhan ◽  
Jiahua Zhang ◽  
Fengmei Yao ◽  
Lamei Shi ◽  
Til Prasad Pangali Sharma ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Deep Learning ◽  
South Asia ◽  
Model Performance ◽  
Remote Sensing Data ◽  
Agricultural Drought ◽  
Gradient Boosting ◽  
Complex Nature ◽  
Sensing Data

Drought, a climate-related disaster impacting a variety of sectors, poses challenges for millions of people in South Asia. Accurate and complete drought information with a proper monitoring system is very important in revealing the complex nature of drought and its associated factors. In this regard, deep learning is a very promising approach for delineating the non-linear characteristics of drought factors. Therefore, this study aims to monitor drought by employing a deep learning approach with remote sensing data over South Asia from 2001–2016. We considered the precipitation, vegetation, and soil factors for the deep forwarded neural network (DFNN) as model input parameters. The study evaluated agricultural drought using the soil moisture deficit index (SMDI) as a response variable during three crop phenology stages. For a better comparison of deep learning model performance, we adopted two machine learning models, distributed random forest (DRF) and gradient boosting machine (GBM). Results show that the DFNN model outperformed the other two models for SMDI prediction. Furthermore, the results indicated that DFNN captured the drought pattern with high spatial variability across three penology stages. Additionally, the DFNN model showed good stability with its cross-validated data in the training phase, and the estimated SMDI had high correlation coefficient R2 ranges from 0.57~0.90, 0.52~0.94, and 0.49~0.82 during the start of the season (SOS), length of the season (LOS), and end of the season (EOS) respectively. The comparison between inter-annual variability of estimated SMDI and in-situ SPEI (standardized precipitation evapotranspiration index) showed that the estimated SMDI was almost similar to in-situ SPEI. The DFNN model provides comprehensive drought information by producing a consistent spatial distribution of SMDI which establishes the applicability of the DFNN model for drought monitoring.

scholarly journals Diabetes classification application with efficient missing and outliers data handling algorithms

2021 ◽  
Author(s):  
Hanaa Torkey ◽  
Elhossiny Ibrahim ◽  
EZZ El-Din Hemdan ◽  
Ayman El-Sayed ◽  
Marwa A. Shouman
Keyword(s):  
Deep Learning ◽  
Healthcare System ◽  
Wearable Sensors ◽  
Model Performance ◽  
Analysis Model ◽  
Human Errors ◽  
Web Based ◽  
Machine Learning Model ◽  
Complete Dataset ◽  
Smart Wearable

AbstractCommunication between sensors spread everywhere in healthcare systems may cause some missing in the transferred features. Repairing the data problems of sensing devices by artificial intelligence technologies have facilitated the Medical Internet of Things (MIoT) and its emerging applications in Healthcare. MIoT has great potential to affect the patient's life. Data collected from smart wearable devices size dramatically increases with data collected from millions of patients who are suffering from diseases such as diabetes. However, sensors or human errors lead to missing some values of the data. The major challenge of this problem is how to predict this value to maintain the data analysis model performance within a good range. In this paper, a complete healthcare system for diabetics has been used, as well as two new algorithms are developed to handle the crucial problem of missed data from MIoT wearable sensors. The proposed work is based on the integration of Random Forest, mean, class' mean, interquartile range (IQR), and Deep Learning to produce a clean and complete dataset. Which can enhance any machine learning model performance. Moreover, the outliers repair technique is proposed based on dataset class detection, then repair it by Deep Learning (DL). The final model accuracy with the two steps of imputation and outliers repair is 97.41% and 99.71% Area Under Curve (AUC). The used healthcare system is a web-based diabetes classification application using flask to be used in hospitals and healthcare centers for the patient diagnosed with an effective fashion.

Sign in / Sign up

Export Citation Format

Share Document