scholarly journals FLEX: Faithful Linguistic Explanations for Neural Net Based Model Decisions

2019 ◽  
Vol 33 ◽  
pp. 2539-2546
Author(s):  
Sandareka Wickramanayake ◽  
Wynne Hsu ◽  
Mong Li Lee
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
State Of The Art ◽  
Neural Net ◽  
End User ◽  
Learning Network ◽  
Benchmark Datasets ◽  
Deep Learning Network ◽  
Post Hoc ◽  
User Trust

Explaining the decisions of a Deep Learning Network is imperative to safeguard end-user trust. Such explanations must be intuitive, descriptive, and faithfully explain why a model makes its decisions. In this work, we propose a framework called FLEX (Faithful Linguistic EXplanations) that generates post-hoc linguistic justifications to rationalize the decision of a Convolutional Neural Network. FLEX explains a model’s decision in terms of features that are responsible for the decision. We derive a novel way to associate such features to words, and introduce a new decision-relevance metric that measures the faithfulness of an explanation to a model’s reasoning. Experiment results on two benchmark datasets demonstrate that the proposed framework can generate discriminative and faithful explanations compared to state-of-the-art explanation generators. We also show how FLEX can generate explanations for images of unseen classes as well as automatically annotate objects in images.

scholarly journals An Analysis of State-of-the-art Activation Functions For Supervised Deep Neural Network

2021 ◽  
Author(s):  
Anh Nguyen ◽  
Khoa Pham ◽  
Dat Ngo ◽  
Thanh Ngo ◽  
Lam Pham
Keyword(s):  
Neural Network ◽  
Supervised Classification ◽  
Deep Neural Network ◽  
State Of The Art ◽  
Network Architectures ◽  
Activation Functions ◽  
Scene Classification ◽  
Learning Network ◽  
Deep Learning Network

This paper provides an analysis of state-of-the-art activation functions with respect to supervised classification of deep neural network. These activation functions comprise of Rectified Linear Units (ReLU), Exponential Linear Unit (ELU), Scaled Exponential Linear Unit (SELU), Gaussian Error Linear Unit (GELU), and the Inverse Square Root Linear Unit (ISRLU). To evaluate, experiments over two deep learning network architectures integrating these activation functions are conducted. The first model, basing on Multilayer Perceptron (MLP), is evaluated with MNIST dataset to perform these activation functions.Meanwhile, the second model, likely VGGish-based architecture, is applied for Acoustic Scene Classification (ASC) Task 1A in DCASE 2018 challenge, thus evaluate whether these activation functions work well in different datasets as well as different network architectures.

Rolling bearing performance degradation assessment based on singular value decomposition-sliding window linear regression and improved deep learning network in noisy environment

2021 ◽  
Author(s):  
Shaojiang Dong ◽  
Yang Li ◽  
Peng Zhu ◽  
Xuewu Pei ◽  
Xuejiao Pan ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Linear Regression ◽  
Sliding Window ◽  
Rolling Bearing ◽  
Noisy Environment ◽  
Learning Network ◽  
Deep Learning Network ◽  
Value Decomposition ◽  
Degradation State

Abstract It is difficult to evaluate the degradation performance and the degradation state of the rolling bearing in noisy environment. A new method is proposed to solve the problem based on singular value decomposition (SVD)-sliding window linear regression and ResNeXt - multi-attention mechanism's deep neural network (RMADNN). Firstly, the root mean square(RMS) gradient value is calculated on the basis of RMS based on SVD and linear regression of sliding window, which is used as the rolling bearing performance degradation indicator in noisy environment. Secondly, the degradation state of rolling bearing is divided by the RMS gradient. Thirdly, for the part of the deep learning network model, the soft attention mechanism is introduced into the bidirectional long short-term memory network (BiLSTM) to extract more important and deep fault features. At the same time, the ResNeXt layer is added into the convolutional neural network (CNN) to extract more fault features and merge them through multi-scale grouped convolution. Then, the hybrid domain attention mechanism (HDAM) was introduced after the ResNext layer. The HDAM can screen out more important features from the output features of the ResNext in the two dimensions of channel and spatial. Therefore, the improved deep learning network of the ResNeXt - multi-attention mechanism's deep neural network (RMADNN) in this research is established. Finally, the labeled data set is input into the improved model for training, and the Softmax classifier is used to identify the life decline state of the rolling bearing. The result shows that the indicator of RMS gradient proposed has a better characterization, and the RMADNN model can distinguish the life degradation state of rolling bearing better.

scholarly journals XCM: An Explainable Convolutional Neural Network for Multivariate Time Series Classification

Mathematics ◽  
2021 ◽  
Vol 9 (23) ◽  
pp. 3137
Author(s):  
Kevin Fauvel ◽  
Tao Lin ◽  
Véronique Masson ◽  
Élisa Fromont ◽  
Alexandre Termier
Keyword(s):  
Neural Network ◽  
Time Series ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Input Data ◽  
State Of The Art ◽  
Multivariate Time Series ◽  
Learning Approach ◽  
Multiple Domains ◽  
Post Hoc

Multivariate Time Series (MTS) classification has gained importance over the past decade with the increase in the number of temporal datasets in multiple domains. The current state-of-the-art MTS classifier is a heavyweight deep learning approach, which outperforms the second-best MTS classifier only on large datasets. Moreover, this deep learning approach cannot provide faithful explanations as it relies on post hoc model-agnostic explainability methods, which could prevent its use in numerous applications. In this paper, we present XCM, an eXplainable Convolutional neural network for MTS classification. XCM is a new compact convolutional neural network which extracts information relative to the observed variables and time directly from the input data. Thus, XCM architecture enables a good generalization ability on both large and small datasets, while allowing the full exploitation of a faithful post hoc model-specific explainability method (Gradient-weighted Class Activation Mapping) by precisely identifying the observed variables and timestamps of the input data that are important for predictions. We first show that XCM outperforms the state-of-the-art MTS classifiers on both the large and small public UEA datasets. Then, we illustrate how XCM reconciles performance and explainability on a synthetic dataset and show that XCM enables a more precise identification of the regions of the input data that are important for predictions compared to the current deep learning MTS classifier also providing faithful explainability. Finally, we present how XCM can outperform the current most accurate state-of-the-art algorithm on a real-world application while enhancing explainability by providing faithful and more informative explanations.

scholarly journals Target Tracking Algorithm Based on Adaptive Scale Detection Learning

Complexity ◽  
2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
Dawei Yang
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Target Tracking ◽  
Sudden Change ◽  
Tracking Algorithm ◽  
Learning Network ◽  
Template Update ◽  
Deep Learning Network ◽  
Kernel Correlation ◽  
Correlation Filtering

In this paper, to better solve the problem of low tracking accuracy caused by the sudden change of target scale, we design and propose an adaptive scale mutation tracking algorithm using a deep learning network to detect the target first and then track it using the kernel correlation filtering method and verify the effectiveness of the model through experiments. The improvement point of this paper is to change the traditional kernel correlation filtering algorithm to detect and track at the same time and to combine deep learning with traditional kernel correlation filtering tracking to apply in the process of target tracking; the addition of deep learning network not only can learn more accurate feature representation but also can more effectively cope with the low resolution of video sequences, so that the algorithm in the case of scale mutation achieves more accurate target tracking in the case of scale mutation. To verify the effectiveness of this method in the case of scale mutation, four evaluation criteria, namely, average accuracy, cross-ratio accuracy, temporal robustness, and spatial robustness, are combined to demonstrate the effectiveness of the algorithm in the case of scale mutation. The experimental results verify that the joint detection strategy plays a good role in correcting the tracking drift caused by the subsequent abrupt change of the target scale and the effectiveness of the adaptive template update strategy. By adaptively changing the number of interval frames of neural network redetection to improve the tracking performance, the tracking speed is improved after the fusion of correlation filtering and neural network, and the combination of both is promoted for better application in target tracking tasks.

Fast and accurate decoding of Raman spectra-encoded suspension arrays using deep learning

The Analyst ◽  
2019 ◽  
Vol 144 (14) ◽  
pp. 4312-4319 ◽  
Author(s):  
Xuejing Chen ◽  
Luyuan Xie ◽  
Yonghong He ◽  
Tian Guan ◽  
Xuesi Zhou ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Raman Spectra ◽  
Learning Network ◽  
Deep Learning Network

A deep learning network called “residual neural network” (ResNet) was used to decode Raman spectra-encoded suspension arrays (SAs).

scholarly journals Toxicity Prediction Method Based on Multi-Channel Convolutional Neural Network

Molecules ◽  
2019 ◽  
Vol 24 (18) ◽  
pp. 3383 ◽  
Author(s):  
Yuan ◽  
Wei ◽  
Guan ◽  
Jiang ◽  
Wang ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Prediction Method ◽  
Toxicity Prediction ◽  
Learning Network ◽  
Machine Learning Methods ◽  
Molecular Information ◽  
Molecular Toxicity ◽  
Deep Learning Network

Molecular toxicity prediction is one of the key studies in drug design. In this paper, a deep learning network based on a two-dimension grid of molecules is proposed to predict toxicity. At first, the van der Waals force and hydrogen bond were calculated according to different descriptors of molecules, and multi-channel grids were generated, which could discover more detail and helpful molecular information for toxicity prediction. The generated grids were fed into a convolutional neural network to obtain the result. A Tox21 dataset was used for the evaluation. This dataset contains more than 12,000 molecules. It can be seen from the experiment that the proposed method performs better compared to other traditional deep learning and machine learning methods.

scholarly journals Tree Internal Defected Imaging Using Model-Driven Deep Learning Network

2021 ◽  
Vol 11 (22) ◽  
pp. 10935
Author(s):  
Hongju Zhou ◽  
Liping Sun ◽  
Hongwei Zhou ◽  
Man Zhao ◽  
Xinpei Yuan ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Defect Detection ◽  
Data Driven ◽  
Detection Accuracy ◽  
Inversion Algorithm ◽  
Internal Defects ◽  
Learning Network ◽  
Model Driven ◽  
Deep Learning Network

The health of trees has become an important issue in forestry. How to detect the health of trees quickly and accurately has become a key area of research for scholars in the world. In this paper, a living tree internal defect detection model is established and analyzed using model-driven theory, where the theoretical fundamentals and implementations of the algorithm are clarified. The location information of the defects inside the trees is obtained by setting a relative permittivity matrix. The data-driven inversion algorithm is realized using a model-driven algorithm that is used to optimize the deep convolutional neural network, which combines the advantages of model-driven algorithms and data-driven algorithms. The results of the comparison inversion algorithms, the BP neural network inversion algorithm, and the model-driven deep learning network inversion algorithm, are analyzed through simulations. The results shown that the model-driven deep learning network inversion algorithm maintains a detection accuracy of more than 90% for single defects or homogeneous double defects, while it can still have a detection accuracy of 78.3% for heterogeneous multiple defects. In the simulations, the single defect detection time of the model-driven deep learning network inversion algorithm is kept within 0.1 s. Additionally, the proposed method overcomes the high nonlinearity and ill-posedness electromagnetic inverse scattering and reduces the time cost and computational complexity of detecting internal defects in trees. The results show that resolution and accuracy are improved in the inversion image for detecting the internal defects of trees.

scholarly journals ST-TrafficNet: A Spatial-Temporal Deep Learning Network for Traffic Forecasting

Electronics ◽  
2020 ◽  
Vol 9 (9) ◽  
pp. 1474 ◽  
Author(s):  
Huakang Lu ◽  
Dongmin Huang ◽  
Youyi Song ◽  
Dazhi Jiang ◽  
Teng Zhou ◽  
...  
Keyword(s):  
Deep Learning ◽  
Traffic Flow ◽  
Short Term Memory ◽  
State Of The Art ◽  
High Dimensional ◽  
Percentage Error ◽  
Learning Network ◽  
Temporal Features ◽  
Spatial Dependencies ◽  
Deep Learning Network

This paper presents a spatial-temporal deep learning network, termed ST-TrafficNet, for traffic flow forecasting. Recent deep learning methods highly relate accurate predetermined graph structure for the complex spatial dependencies of traffic flow, and ineffectively harvest high dimensional temporal features of the traffic flow. In this paper, a novel multi-diffusion convolution block constructed by an attentive diffusion convolution and bidirectional diffusion convolution is proposed, which is capable to extract precise potential spatial dependencies. Moreover, a stacked Long Short-Term Memory (LSTM) block is adopted to capture high-dimensional temporal features. By integrating the two blocks, the ST-TrafficNet can learn the spatial-temporal dependencies of intricate traffic data accurately. The performance of the ST-TrafficNet has been evaluated on two real-world benchmark datasets by comparing it with three commonly-used methods and seven state-of-the-art ones. The Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Mean Absolute Percentage Error (MAPE) of the proposed method outperform not only the commonly-used methods, but also the state-of-the-art ones in 15 min, 30 min, and 60 min time-steps.

scholarly journals Transferability of Convolutional Neural Network Models for Identifying Damaged Buildings Due to Earthquake

2021 ◽  
Vol 13 (3) ◽  
pp. 504
Author(s):  
Wanting Yang ◽  
Xianfeng Zhang ◽  
Peng Luo
Keyword(s):  
Neural Network ◽  
Remote Sensing ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Network Model ◽  
Remote Sensing Data ◽  
Remote Sensing Images ◽  
Learning Network ◽  
Building Information ◽  
Deep Learning Network

The collapse of buildings caused by earthquakes can lead to a large loss of life and property. Rapid assessment of building damage with remote sensing image data can support emergency rescues. However, current studies indicate that only a limited sample set can usually be obtained from remote sensing images immediately following an earthquake. Consequently, the difficulty in preparing sufficient training samples constrains the generalization of the model in the identification of earthquake-damaged buildings. To produce a deep learning network model with strong generalization, this study adjusted four Convolutional Neural Network (CNN) models for extracting damaged building information and compared their performance. A sample dataset of damaged buildings was constructed by using multiple disaster images retrieved from the xBD dataset. Using satellite and aerial remote sensing data obtained after the 2008 Wenchuan earthquake, we examined the geographic and data transferability of the deep network model pre-trained on the xBD dataset. The result shows that the network model pre-trained with samples generated from multiple disaster remote sensing images can extract accurately collapsed building information from satellite remote sensing data. Among the adjusted CNN models tested in the study, the adjusted DenseNet121 was the most robust. Transfer learning solved the problem of poor adaptability of the network model to remote sensing images acquired by different platforms and could identify disaster-damaged buildings properly. These results provide a solution to the rapid extraction of earthquake-damaged building information based on a deep learning network model.

Sign in / Sign up

Export Citation Format

Share Document