Deep Neural Networks for Behavioral Credit Rating

Logistic regression is the industry standard in credit risk modeling. Regulatory requirements for model explainability have halted the implementation of more advanced, non-linear machine learning algorithms, even though more accurate predictions would benefit consumers and banks alike. Deep neural networks are certainly some of the most prominent non-linear algorithms. In this paper, we propose a deep neural network model for behavioral credit rating. Behavioral models are used to assess the future performance of a bank’s existing portfolio in order to meet the capital requirements introduced by the Basel regulatory framework, which are designed to increase the banks’ ability to absorb large financial shocks. The proposed deep neural network was trained on two different datasets: the first one contains information on loans between 2009 and 2013 (during the financial crisis) and the second one from 2014 to 2018 (after the financial crisis); combined, they include more than 1.5 million examples. The proposed network outperformed multiple benchmarks and was evenly matched with the XGBoost model. Long-term credit rating performance is also presented, as well as a detailed analysis of the reprogrammed facilities’ impact on model performance.

Download Full-text

Removal of Motion Artifacts from ECG signals by Combination of Recurrent Neural Networks and Deep Neural Networks

10.36227/techrxiv.14526828 ◽

2021 ◽

Author(s):

Muhammad Zubair

Keyword(s):

Neural Network ◽

Neural Networks ◽

Recurrent Neural Networks ◽

Deep Neural Network ◽

Deep Neural Networks ◽

Motion Artifacts ◽

Machine Learning Algorithms ◽

Unique Combination ◽

Ecg Signals ◽

Electrocardiogram Ecg

<div><div><div><p>Electrocardiogram (ECG) is the graphical portrayal of heart usefulness. The ECG signals holds its significance in the discovery of heart irregularities. These ECG signals are frequently tainted by antiques from various sources. It is basic to diminish these curios and improve the exactness just as dependability to show signs of improvement results identified with heart usefulness. The most commonly disturbed artifact in ECG signals is Motion Artifacts (MA). In this paper, we have proposed a new concept on how machine learning algorithms can be used for de-noising the ECG signals. Towards the goal, a unique combination of Recurrent Neural Network (RNN) and Deep Neural Network (DNN) is used to efficiently remove MA. The proposed algorithm is validated using ECG records obtained from the MIT-BIH Arrhythmia Database. To eliminate MA using the proposed method, we have used Adam optimization algorithm to train and fit the contaminated ECG data in RNN and DNN models. Performance evaluation results in terms of SNR and RRMSE show that the proposed algorithm outperforms other existing MA removal methods without significantly distorting the morphologies of ECG signals.</p></div></div></div>

Download Full-text

Removal of Motion Artifacts from ECG signals by Combination of Recurrent Neural Networks and Deep Neural Networks

10.36227/techrxiv.14526828.v1 ◽

2021 ◽

Author(s):

Muhammad Zubair

Keyword(s):

Neural Network ◽

Neural Networks ◽

Recurrent Neural Networks ◽

Deep Neural Network ◽

Deep Neural Networks ◽

Motion Artifacts ◽

Machine Learning Algorithms ◽

Unique Combination ◽

Ecg Signals ◽

Electrocardiogram Ecg

Download Full-text

Deep neural networks using a single neuron: folded-in-time architecture using feedback-modulated delay loops

Nature Communications ◽

10.1038/s41467-021-25427-4 ◽

2021 ◽

Vol 12 (1) ◽

Author(s):

Florian Stelzer ◽

André Röhm ◽

Raul Vicente ◽

Ingo Fischer ◽

Serhiy Yanchuk

Keyword(s):

Neural Network ◽

Neural Networks ◽

Deep Neural Network ◽

Single Neuron ◽

Deep Neural Networks ◽

Back Propagation ◽

Local Network ◽

Multiple Time ◽

Learning Tools ◽

Back Propagation Algorithm

AbstractDeep neural networks are among the most widely applied machine learning tools showing outstanding performance in a broad range of tasks. We present a method for folding a deep neural network of arbitrary size into a single neuron with multiple time-delayed feedback loops. This single-neuron deep neural network comprises only a single nonlinearity and appropriately adjusted modulations of the feedback signals. The network states emerge in time as a temporal unfolding of the neuron’s dynamics. By adjusting the feedback-modulation within the loops, we adapt the network’s connection weights. These connection weights are determined via a back-propagation algorithm, where both the delay-induced and local network connections must be taken into account. Our approach can fully represent standard Deep Neural Networks (DNN), encompasses sparse DNNs, and extends the DNN concept toward dynamical systems implementations. The new method, which we call Folded-in-time DNN (Fit-DNN), exhibits promising performance in a set of benchmark tasks.

Download Full-text

Inclusion of Multiple Cycling of the Potential into Deep Neural Network Classification of Voltammetric Reaction Mechanisms

Faraday Discussions ◽

10.1039/d1fd00050k ◽

2021 ◽

Author(s):

Luke Gundry ◽

Gareth Kennedy ◽

Alan Bond ◽

Jie Zhang

Keyword(s):

Neural Network ◽

Neural Networks ◽

Reaction Mechanisms ◽

Deep Neural Network ◽

Deep Neural Networks ◽

Neural Network Classification ◽

Initial Cycle

The use of Deep Neural Networks (DNNs) for the classification of electrochemical mechanisms based on training with simulations of the initial cycle of potential have been reported. In this paper,...

Download Full-text

EMOTIONS RECOGNITION IN HUMAN SPEECH USING DEEP NEURAL NETWORKS

Vestnik komp iuternykh i informatsionnykh tekhnologii ◽

10.14489/vkit.2021.01.pp.044-051 ◽

2021 ◽

pp. 44-51

Author(s):

E. Yu. Shchetinin

Keyword(s):

Neural Network ◽

Machine Learning ◽

Neural Networks ◽

Convolutional Neural Network ◽

Recurrent Neural Network ◽

Deep Neural Networks ◽

Learning Algorithms ◽

Machine Learning Algorithms ◽

Audio Recordings ◽

Computer Studies

The recognition of human emotions is one of the most relevant and dynamically developing areas of modern speech technologies, and the recognition of emotions in speech (RER) is the most demanded part of them. In this paper, we propose a computer model of emotion recognition based on an ensemble of bidirectional recurrent neural network with LSTM memory cell and deep convolutional neural network ResNet18. In this paper, computer studies of the RAVDESS database containing emotional speech of a person are carried out. RAVDESS-a data set containing 7356 files. Entries contain the following emotions: 0 – neutral, 1 – calm, 2 – happiness, 3 – sadness, 4 – anger, 5 – fear, 6 – disgust, 7 – surprise. In total, the database contains 16 classes (8 emotions divided into male and female) for a total of 1440 samples (speech only). To train machine learning algorithms and deep neural networks to recognize emotions, existing audio recordings must be pre-processed in such a way as to extract the main characteristic features of certain emotions. This was done using Mel-frequency cepstral coefficients, chroma coefficients, as well as the characteristics of the frequency spectrum of audio recordings. In this paper, computer studies of various models of neural networks for emotion recognition are carried out on the example of the data described above. In addition, machine learning algorithms were used for comparative analysis. Thus, the following models were trained during the experiments: logistic regression (LR), classifier based on the support vector machine (SVM), decision tree (DT), random forest (RF), gradient boosting over trees – XGBoost, convolutional neural network CNN, recurrent neural network RNN (ResNet18), as well as an ensemble of convolutional and recurrent networks Stacked CNN-RNN. The results show that neural networks showed much higher accuracy in recognizing and classifying emotions than the machine learning algorithms used. Of the three neural network models presented, the CNN + BLSTM ensemble showed higher accuracy.

Download Full-text

Automated vessel segmentation in lung CT and CTA images via deep neural networks

Journal of X-Ray Science and Technology ◽

10.3233/xst-210955 ◽

2021 ◽

pp. 1-15

Author(s):

Wenjun Tan ◽

Luyu Zhou ◽

Xiaoshuo Li ◽

Xiaoyu Yang ◽

Yufei Chen ◽

...

Keyword(s):

Neural Network ◽

Computed Tomography ◽

Neural Networks ◽

Deep Neural Network ◽

Deep Neural Networks ◽

Vessel Segmentation ◽

Multi Scale ◽

Vascular Segmentation ◽

Study Results ◽

Lung Ct

BACKGROUND: The distribution of pulmonary vessels in computed tomography (CT) and computed tomography angiography (CTA) images of lung is important for diagnosing disease, formulating surgical plans and pulmonary research. PURPOSE: Based on the pulmonary vascular segmentation task of International Symposium on Image Computing and Digital Medicine 2020 challenge, this paper reviews 12 different pulmonary vascular segmentation algorithms of lung CT and CTA images and then objectively evaluates and compares their performances. METHODS: First, we present the annotated reference dataset of lung CT and CTA images. A subset of the dataset consisting 7,307 slices for training and 3,888 slices for testing was made available for participants. Second, by analyzing the performance comparison of different convolutional neural networks from 12 different institutions for pulmonary vascular segmentation, the reasons for some defects and improvements are summarized. The models are mainly based on U-Net, Attention, GAN, and multi-scale fusion network. The performance is measured in terms of Dice coefficient, over segmentation ratio and under segmentation rate. Finally, we discuss several proposed methods to improve the pulmonary vessel segmentation results using deep neural networks. RESULTS: By comparing with the annotated ground truth from both lung CT and CTA images, most of 12 deep neural network algorithms do an admirable job in pulmonary vascular extraction and segmentation with the dice coefficients ranging from 0.70 to 0.85. The dice coefficients for the top three algorithms are about 0.80. CONCLUSIONS: Study results show that integrating methods that consider spatial information, fuse multi-scale feature map, or have an excellent post-processing to deep neural network training and optimization process are significant for further improving the accuracy of pulmonary vascular segmentation.

Download Full-text

Extensive deep neural networks for transferring small scale learning to large scale systems

Chemical Science ◽

10.1039/c8sc04578j ◽

2019 ◽

Vol 10 (15) ◽

pp. 4129-4140 ◽

Cited By ~ 9

Author(s):

Kyle Mills ◽

Kevin Ryczko ◽

Iryna Luchak ◽

Adam Domurad ◽

Chris Beeler ◽

...

Keyword(s):

Neural Network ◽

Neural Networks ◽

Large Scale ◽

Deep Neural Network ◽

Deep Neural Networks ◽

Small Scale ◽

Large Scale Systems ◽

Energy Entropy ◽

Large Systems ◽

Number Of Particles

We present a physically-motivated topology of a deep neural network that can efficiently infer extensive parameters (such as energy, entropy, or number of particles) of arbitrarily large systems, doing so with scaling.

Download Full-text

Development and Validation of a Deep Neural Network Model for Prediction of Postoperative In-hospital Mortality

Anesthesiology ◽

10.1097/aln.0000000000002186 ◽

2018 ◽

Vol 129 (4) ◽

pp. 649-662 ◽

Cited By ~ 48

Author(s):

Christine K. Lee ◽

Ira Hofer ◽

Eilon Gabel ◽

Pierre Baldi ◽

Maxime Cannesson

Keyword(s):

Neural Network ◽

Neural Networks ◽

Hospital Mortality ◽

Deep Neural Network ◽

Deep Neural Networks ◽

Operating Characteristics ◽

Physical Status ◽

Physical Status Classification ◽

Receiver Operating Characteristics Curve ◽

Status Classification

Abstract Editor’s Perspective What We Already Know about This Topic What This Article Tells Us That Is New Background The authors tested the hypothesis that deep neural networks trained on intraoperative features can predict postoperative in-hospital mortality. Methods The data used to train and validate the algorithm consists of 59,985 patients with 87 features extracted at the end of surgery. Feed-forward networks with a logistic output were trained using stochastic gradient descent with momentum. The deep neural networks were trained on 80% of the data, with 20% reserved for testing. The authors assessed improvement of the deep neural network by adding American Society of Anesthesiologists (ASA) Physical Status Classification and robustness of the deep neural network to a reduced feature set. The networks were then compared to ASA Physical Status, logistic regression, and other published clinical scores including the Surgical Apgar, Preoperative Score to Predict Postoperative Mortality, Risk Quantification Index, and the Risk Stratification Index. Results In-hospital mortality in the training and test sets were 0.81% and 0.73%. The deep neural network with a reduced feature set and ASA Physical Status classification had the highest area under the receiver operating characteristics curve, 0.91 (95% CI, 0.88 to 0.93). The highest logistic regression area under the curve was found with a reduced feature set and ASA Physical Status (0.90, 95% CI, 0.87 to 0.93). The Risk Stratification Index had the highest area under the receiver operating characteristics curve, at 0.97 (95% CI, 0.94 to 0.99). Conclusions Deep neural networks can predict in-hospital mortality based on automatically extractable intraoperative data, but are not (yet) superior to existing methods.

Download Full-text

Analysis of the possibilities of using a deep neural networks as an image recognition technology to solve the problem of automating attendance tracking

System Analysis in Science and Education ◽

10.37005/2071-9612-2020-2-15-26 ◽

2020 ◽

pp. 15-26

Author(s):

Anna Ilina ◽

Vladimir Korenkov

Keyword(s):

Neural Network ◽

Neural Networks ◽

Quality Assessment ◽

Image Recognition ◽

Deep Neural Network ◽

Deep Neural Networks ◽

Facial Recognition ◽

Manual Counting

The task of counting the number of people is relevant when conducting various types of events, which may include seminars, lectures, conferences, meetings, etc. Instead of monotonous manual counting of participants, it is much more effective to use facial recognition technology, which makes it possible not only to quickly count those present, but also to recognize each of them, which makes it possible to conduct further analysis of this data, identify patterns in them and predict. The research conducted in this paper determines the quality assessment of the use of facial recognition technology in images andvideo streams, based on the use of a deep neural network, to solve the problem of automating attendance tracking.

Download Full-text

Explainable Deep Learning Reproduces a ‘Professional Eye’ on the Diagnosis of Internal Disorders in Persimmon Fruit

Plant and Cell Physiology ◽

10.1093/pcp/pcaa111 ◽

2020 ◽

Vol 61 (11) ◽

pp. 1967-1973

Author(s):

Takashi Akagi ◽

Masanori Onishi ◽

Kanae Masuda ◽

Ryohei Kuroki ◽

Kohei Baba ◽

...

Keyword(s):

Neural Network ◽

Neural Networks ◽

Deep Neural Network ◽

Deep Neural Networks ◽

Binary Classification ◽

Network Models ◽

Plant Biology ◽

Neural Network Models ◽

Persimmon Fruit

Abstract Recent rapid progress in deep neural network techniques has allowed recognition and classification of various objects, often exceeding the performance of the human eye. In plant biology and crop sciences, some deep neural network frameworks have been applied mainly for effective and rapid phenotyping. In this study, beyond simple optimizations of phenotyping, we propose an application of deep neural networks to make an image-based internal disorder diagnosis that is hard even for experts, and to visualize the reasons behind each diagnosis to provide biological interpretations. Here, we exemplified classification of calyx-end cracking in persimmon fruit by using five convolutional neural network models with various layer structures and examined potential analytical options involved in the diagnostic qualities. With 3,173 visible RGB images from the fruit apex side, the neural networks successfully made the binary classification of each degree of disorder, with up to 90% accuracy. Furthermore, feature visualizations, such as Grad-CAM and LRP, visualize the regions of the image that contribute to the diagnosis. They suggest that specific patterns of color unevenness, such as in the fruit peripheral area, can be indexes of calyx-end cracking. These results not only provided novel insights into indexes of fruit internal disorders but also proposed the potential applicability of deep neural networks in plant biology.

Download Full-text