scholarly journals Which Attributes Matter the Most for Loan Origination? A Neural Attention Approach

2020 ◽  
Author(s):  
Antonios Alexos ◽  
Sotirios Chatzis
Keyword(s):  
Deep Learning ◽  
Real World ◽  
Individual Case ◽  
Learning Model ◽  
Attention Mechanism ◽  
Novel Approach ◽  
Specific Individual ◽  
Real World Datasets ◽  
Deep Learning Model

In this paper we address the understanding of the problem, of why a deep learning model decides that an individual is eligible for a loan or not. Here we propose a novel approach for inferring, which attributes matter the most, for making a decision in each specific individual case. Specifically we leverage concepts from neural attention to devise a novel feature wise attention mechanism. As we show, using real world datasets, our approach offers unique insights into the importance of various features, by producing a decision explanation for each specific loan case. At the same time, we observe that our novel mechanism, generates decisions which are much closer to the decisions generated by human experts, compared to the existent competitors.

scholarly journals A Hybrid Deep Learning Model for Predicting Protein Hydroxylation Sites

2018 ◽  
Vol 19 (9) ◽  
pp. 2817 ◽  
Author(s):  
Haixia Long ◽  
Bo Liao ◽  
Xingyu Xu ◽  
Jialiang Yang
Keyword(s):  
Deep Learning ◽  
Short Term Memory ◽  
Learning Model ◽  
New Drugs ◽  
Post Translational Modifications ◽  
Novel Approach ◽  
Benchmark Datasets ◽  
Memory Network ◽  
Scoring Matrix ◽  
Deep Learning Model

Protein hydroxylation is one type of post-translational modifications (PTMs) playing critical roles in human diseases. It is known that protein sequence contains many uncharacterized residues of proline and lysine. The question that needs to be answered is: which residue can be hydroxylated, and which one cannot. The answer will not only help understand the mechanism of hydroxylation but can also benefit the development of new drugs. In this paper, we proposed a novel approach for predicting hydroxylation using a hybrid deep learning model integrating the convolutional neural network (CNN) and long short-term memory network (LSTM). We employed a pseudo amino acid composition (PseAAC) method to construct valid benchmark datasets based on a sliding window strategy and used the position-specific scoring matrix (PSSM) to represent samples as inputs to the deep learning model. In addition, we compared our method with popular predictors including CNN, iHyd-PseAAC, and iHyd-PseCp. The results for 5-fold cross-validations all demonstrated that our method significantly outperforms the other methods in prediction accuracy.

scholarly journals Do Game Data Generalize Well for Remote Sensing Image Segmentation?

2020 ◽  
Vol 12 (2) ◽  
pp. 275 ◽  
Author(s):  
Zhengxia Zou ◽  
Tianyang Shi ◽  
Wenyuan Li ◽  
Zhou Zhang ◽  
Zhenwei Shi
Keyword(s):  
Remote Sensing ◽  
Deep Learning ◽  
Video Game ◽  
Real World ◽  
Remote Sensing Data ◽  
Ground Truth ◽  
Remote Sensing Image ◽  
Learning Model ◽  
Image Translation ◽  
Deep Learning Model

Despite the recent progress in deep learning and remote sensing image interpretation, the adaption of a deep learning model between different sources of remote sensing data still remains a challenge. This paper investigates an interesting question: do synthetic data generalize well for remote sensing image applications? To answer this question, we take the building segmentation as an example by training a deep learning model on the city map of a well-known video game “Grand Theft Auto V” and then adapting the model to real-world remote sensing images. We propose a generative adversarial training based segmentation framework to improve the adaptability of the segmentation model. Our model consists of a CycleGAN model and a ResNet based segmentation network, where the former one is a well-known image-to-image translation framework which learns a mapping of the image from the game domain to the remote sensing domain; and the latter one learns to predict pixel-wise building masks based on the transformed data. All models in our method can be trained in an end-to-end fashion. The segmentation model can be trained without using any additional ground truth reference of the real-world images. Experimental results on a public building segmentation dataset suggest the effectiveness of our adaptation method. Our method shows superiority over other state-of-the-art semantic segmentation methods, for example, Deeplab-v3 and UNet. Another advantage of our method is that by introducing semantic information to the image-to-image translation framework, the image style conversion can be further improved.

scholarly journals Multi-Task Deep Learning Model with an Attention Mechanism for Ship Accident Sentence Prediction

2021 ◽  
Vol 12 (1) ◽  
pp. 233
Author(s):  
Ho-Min Park ◽  
Jae-Hoon Kim
Keyword(s):  
Deep Learning ◽  
Learning Model ◽  
Attention Mechanism ◽  
Maritime Safety ◽  
Proposed Model ◽  
Deep Learning Model

The number of ship accidents occurring in the Korean ocean has been steadily increasing year by year. The Korea Maritime Safety Tribunal (KMST) has published verdicts to ensure that the relevant personnel can share judgment on these accidents. As of 2020, there have been 3156 ship accidents; thus, it is difficult for the relevant personnel to study these various accidents by only reading the verdicts. Therefore, in this study, we propose a multi-task deep learning model with an attention mechanism for predicting the sentencing of ship accidents. The tasks are accident types, applied articles, and the sentencing of ship accidents. The proposed model was tested under verdicts published by the KMST between 2010 and 2019. Through experiments, we show that the proposed model can improve the performance of sentence prediction and can assist the relevant personnel to study these accidents.

scholarly journals Prediction of pandemic risk for animal-origin coronavirus using a deep learning method

2021 ◽  
Vol 10 (1) ◽  
Author(s):  
Zheng Kou ◽  
Yi-Fan Huang ◽  
Ao Shen ◽  
Saeed Kosari ◽  
Xiang-Rong Liu ◽  
...  
Keyword(s):  
Deep Learning ◽  
General Model ◽  
Characteristic Curve ◽  
Learning Model ◽  
Attention Mechanism ◽  
Animal Origin ◽  
Coding Region ◽  
Species Barrier ◽  
Potential Pathogen ◽  
Deep Learning Model

Abstract Background Coronaviruses can be isolated from bats, civets, pangolins, birds and other wild animals. As an animal-origin pathogen, coronavirus can cross species barrier and cause pandemic in humans. In this study, a deep learning model for early prediction of pandemic risk was proposed based on the sequences of viral genomes. Methods A total of 3257 genomes were downloaded from the Coronavirus Genome Resource Library. We present a deep learning model of cross-species coronavirus infection that combines a bidirectional gated recurrent unit network with a one-dimensional convolution. The genome sequence of animal-origin coronavirus was directly input to extract features and predict pandemic risk. The best performances were explored with the use of pre-trained DNA vector and attention mechanism. The area under the receiver operating characteristic curve (AUROC) and the area under precision-recall curve (AUPR) were used to evaluate the predictive models. Results The six specific models achieved good performances for the corresponding virus groups (1 for AUROC and 1 for AUPR). The general model with pre-training vector and attention mechanism provided excellent predictions for all virus groups (1 for AUROC and 1 for AUPR) while those without pre-training vector or attention mechanism had obviously reduction of performance (about 5–25%). Re-training experiments showed that the general model has good capabilities of transfer learning (average for six groups: 0.968 for AUROC and 0.942 for AUPR) and should give reasonable prediction for potential pathogen of next pandemic. The artificial negative data with the replacement of the coding region of the spike protein were also predicted correctly (100% accuracy). With the application of the Python programming language, an easy-to-use tool was created to implements our predictor. Conclusions Robust deep learning model with pre-training vector and attention mechanism mastered the features from the whole genomes of animal-origin coronaviruses and could predict the risk of cross-species infection for early warning of next pandemic. Graphical Abstract

scholarly journals Dynamic Pollution Emission Prediction Method of a Combined Heat and Power System Based on the Hybrid CNN-LSTM Model and Attention Mechanism

2021 ◽  
Author(s):  
Anping Wan ◽  
Jie Yang ◽  
Ting Chen ◽  
Yang Jinxing ◽  
Ke Li ◽  
...  
Keyword(s):  
Deep Learning ◽  
Power System ◽  
Prediction Method ◽  
Learning Model ◽  
Combined Heat And Power ◽  
Attention Mechanism ◽  
Feature Engineering ◽  
Pollution Emission ◽  
Highly Nonlinear ◽  
Deep Learning Model

Abstract The prediction of pollution emission from a combined heat and power (CHP) system is very important for the production regulation and emergency response of a power system. The composition and structure of the CHP equipment are complex, and the production process is cumbersome. The fuel chemical reaction of the pulverized coal in the boiler represents a highly nonlinear and strongly interrelated process that is strongly affected by external environmental factors, which causes a certain level of volatility and uncertainty. In this study, a pollution emission prediction method of CHP systems based on feature engineering and a hybrid deep learning model is proposed. Feature engineering performs multi-step preprocessing on the original data, refines the correlation factors, and removes redundant variables. The hybrid deep learning model has a multi-variable input and is established by combining the convolutional neural network-long short-term memory network with the attention mechanism. The case study is conducted on the collected actual dataset. The influence of the prediction target periodicity on the prediction results is analyzed seasonally to verify the effectiveness of the hybrid model. The results show that the root mean square error of the proposed method is less than one, and the error is reduced compared to the other basic methods, which proves the superiority of the proposed pollution emission prediction method over the existing methods.

scholarly journals Psychological Stress Detection According to ECG Using a Deep Learning Model with Attention Mechanism

2021 ◽  
Vol 11 (6) ◽  
pp. 2848
Author(s):  
Pengfei Zhang ◽  
Fenghua Li ◽  
Lidong Du ◽  
Rongjian Zhao ◽  
Xianxiang Chen ◽  
...  
Keyword(s):  
Deep Learning ◽  
Short Term Memory ◽  
Learning Model ◽  
Attention Mechanism ◽  
Stress Detection ◽  
Short Term ◽  
Term Memory ◽  
Average Accuracy ◽  
Long Short Term Memory ◽  
Deep Learning Model

To satisfy the need to accurately monitor emotional stress, this paper explores the effectiveness of the attention mechanism based on the deep learning model CNN (Convolutional Neural Networks)-BiLSTM (Bi-directional Long Short-Term Memory) As different attention mechanisms can cause the framework to focus on different positions of the feature map, this discussion adds attention mechanisms to the CNN layer and the BiLSTM layer separately, and to both the CNN layer and BiLSTM layer simultaneously to generate different CNN–BiLSTM networks with attention mechanisms. ECG (electrocardiogram) data from 34 subjects were collected on the server platform created by the Institute of Psychology of the Chinese Academy of Science and the researches. It verifies that the average accuracy of CNN–BiLSTM is up to 0.865 without any attention mechanism, while the highest average accuracy of 0.868 is achieved using the CNN–attention–based BiLSTM.

Sign in / Sign up

Export Citation Format

Share Document