scholarly journals One-Dimensional Convolutional Neural Networks with Feature Selection for Highly Concise Rule Extraction from Credit Scoring Datasets with Heterogeneous Attributes

Electronics ◽  
2020 ◽  
Vol 9 (8) ◽  
pp. 1318
Author(s):  
Yoichi Hayashi ◽  
Naoki Takano
Keyword(s):  
Neural Networks ◽  
Credit Scoring ◽  
Extraction Methods ◽  
Rule Extraction ◽  
Financial Industry ◽  
New Approach ◽  
New Era ◽  
One Dimensional ◽  
Recursive Rule ◽  
Fully Connected

Convolution neural networks (CNNs) have proven effectiveness, but they are not applicable to all datasets, such as those with heterogeneous attributes, which are often used in the finance and banking industries. Such datasets are difficult to classify, and to date, existing high-accuracy classifiers and rule-extraction methods have not been able to achieve sufficiently high classification accuracies or concise classification rules. This study aims to provide a new approach for achieving transparency and conciseness in credit scoring datasets with heterogeneous attributes by using a one-dimensional (1D) fully-connected layer first CNN combined with the Recursive-Rule Extraction (Re-RX) algorithm with a J48graft decision tree (hereafter 1D FCLF-CNN). Based on a comparison between the proposed 1D FCLF-CNN and existing rule extraction methods, our architecture enabled the extraction of the most concise rules (6.2) and achieved the best accuracy (73.10%), i.e., the highest interpretability–priority rule extraction. These results suggest that the 1D FCLF-CNN with Re-RX with J48graft is very effective for extracting highly concise rules for heterogeneous credit scoring datasets. Although it does not completely overcome the accuracy–interpretability dilemma for deep learning, it does appear to resolve this issue for credit scoring datasets with heterogeneous attributes, and thus, could lead to a new era in the financial industry.

scholarly journals An Empirical Evaluation of Rule Extraction from Recurrent Neural Networks

2018 ◽  
Vol 30 (9) ◽  
pp. 2568-2591 ◽  
Author(s):  
Qinglong Wang ◽  
Kaixuan Zhang ◽  
Alexander G. Ororbia II ◽  
Xinyu Xing ◽  
Xue Liu ◽  
...  
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Credit Scoring ◽  
Empirical Evaluation ◽  
Rule Extraction ◽  
Production Rules ◽  
Recursive Models ◽  
Box Models ◽  
Highly Nonlinear ◽  
Black Box Models

Rule extraction from black box models is critical in domains that require model validation before implementation, as can be the case in credit scoring and medical diagnosis. Though already a challenging problem in statistical learning in general, the difficulty is even greater when highly nonlinear, recursive models, such as recurrent neural networks (RNNs), are fit to data. Here, we study the extraction of rules from second-order RNNs trained to recognize the Tomita grammars. We show that production rules can be stably extracted from trained RNNs and that in certain cases, the rules outperform the trained RNNs.

scholarly journals Topological measurement of deep neural networks using persistent homology

2021 ◽  
Author(s):  
Satoru Watanabe ◽  
Hayato Yamana
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Persistent Homology ◽  
Topological Data Analysis ◽  
Data Sets ◽  
One Dimensional ◽  
Novel Approach ◽  
The One ◽  
Fully Connected ◽  
Fully Connected Networks

AbstractThe inner representation of deep neural networks (DNNs) is indecipherable, which makes it difficult to tune DNN models, control their training process, and interpret their outputs. In this paper, we propose a novel approach to investigate the inner representation of DNNs through topological data analysis (TDA). Persistent homology (PH), one of the outstanding methods in TDA, was employed for investigating the complexities of trained DNNs. We constructed clique complexes on trained DNNs and calculated the one-dimensional PH of DNNs. The PH reveals the combinational effects of multiple neurons in DNNs at different resolutions, which is difficult to be captured without using PH. Evaluations were conducted using fully connected networks (FCNs) and networks combining FCNs and convolutional neural networks (CNNs) trained on the MNIST and CIFAR-10 data sets. Evaluation results demonstrate that the PH of DNNs reflects both the excess of neurons and problem difficulty, making PH one of the prominent methods for investigating the inner representation of DNNs.

scholarly journals An Efficient and Robust Star Identification Algorithm Based on Neural Networks

Sensors ◽  
2021 ◽  
Vol 21 (22) ◽  
pp. 7686
Author(s):  
Bendong Wang ◽  
Hao Wang ◽  
Zhonghe Jin
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Identification Accuracy ◽  
Identification Algorithm ◽  
Rotation Invariant ◽  
One Dimensional ◽  
Identification Rate ◽  
Star Identification ◽  
Space Star ◽  
Fully Connected

A lost-in-space star identification algorithm based on a one-dimensional Convolutional Neural Network (1D CNN) is proposed. The lost-in-space star identification aims to identify stars observed with corresponding catalog stars when there is no prior attitude information. With the help of neural networks, the robustness and the speed of the star identification are improved greatly. In this paper, a modified log-Polar mapping is used to constructed rotation-invariant star patterns. Then a 1D CNN is utilized to classify the star patterns associated with guide stars. In the 1D CNN model, a global average pooling layer is used to replace fully-connected layers to reduce the number of parameters and the risk of overfitting. Experiments show that the proposed algorithm is highly robust to position noise, magnitude noise, and false stars. The identification accuracy is 98.1% with 5 pixels position noise, 97.4% with 5 false stars, and 97.7% with 0.5 Mv magnitude noise, respectively, which is significantly higher than the identification rate of the pyramid, optimized grid and modified log-polar algorithms. Moreover, the proposed algorithm guarantees a reliable star identification under dynamic conditions. The identification accuracy is 82.1% with angular velocity of 10 degrees per second. Furthermore, its identification time is as short as 32.7 miliseconds and the memory required is about 1920 kilobytes. The algorithm proposed is suitable for current embedded systems.

scholarly journals A Two-Step Rule-Extraction Technique for a CNN

Electronics ◽  
2020 ◽  
Vol 9 (6) ◽  
pp. 990
Author(s):  
Guido Bologna ◽  
Silvio Fossati
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Skin Cancer ◽  
Convolutional Neural Networks ◽  
Cancer Diagnosis ◽  
Medical Diagnosis ◽  
Rule Extraction ◽  
High Fidelity ◽  
Max Pooling ◽  
Fully Connected

The explanation of the decisions provided by a model are crucial in a domain such as medical diagnosis. With the advent of deep learning, it is very important to explain why a classification is reached by a model. This work tackles the transparency problem of convolutional neural networks(CNNs). We propose to generate propositional rules from CNNs, because they are intuitive to the way humans reason. Our method considers that a CNN is the union of two subnetworks: a multi-layer erceptron (MLP) in the fully connected layers; and a subnetwork including several 2D convolutional layers and max-pooling layers. Rule extraction exhibits two main steps, with each step generating rules from each subnetwork of the CNN. In practice, we approximate the two subnetworks by two particular MLP models that makes it possible to generate propositional rules. We performed the experiments with two datasets involving images: MNISTdigit recognition; and skin-cancer diagnosis. With high fidelity, the extracted rules designated the location of discriminant pixels, as well as the conditions that had to be met to achieve the classification. We illustrated several examples of rules by their centroids and their discriminant pixels.

Using Sample Selection to Improve Accuracy and Simplicity of Rules Extracted from Neural Networks for Credit Scoring Applications

2015 ◽  
Vol 14 (04) ◽  
pp. 1550021 ◽  
Author(s):  
Rudy Setiono ◽  
Arnulfo Azcarraga ◽  
Yoichi Hayashi
Keyword(s):  
Neural Networks ◽  
Predictive Accuracy ◽  
Credit Scoring ◽  
Sample Selection ◽  
Rule Extraction ◽  
Training Data ◽  
Training Dataset ◽  
Original Dataset ◽  
Extraction Algorithm ◽  
The Neural Networks

In this paper, we present an approach for sample selection using an ensemble of neural networks for credit scoring. The ensemble determines samples that can be considered outliers by checking the classification accuracy of the neural networks on the original training data samples. Those samples that are consistently misclassified by the neural networks in the ensemble are removed from the training dataset. The remaining data samples are then used to train and prune another neural network for rule extraction. Our experimental results on publicly available benchmark credit scoring datasets show that by eliminating the outliers, we obtain neural networks with higher predictive accuracy and simpler in structure compared to the networks that are trained with the original dataset. A rule extraction algorithm is applied to generate comprehensible rules from the neural networks. The extracted rules are more concise than the rules generated from networks that have been trained using the original datasets.

Sign in / Sign up

Export Citation Format

Share Document