Knowledge Extraction of Adaptive Structural Learning of Deep Belief Network for Medical Examination Data

2019 ◽  
Vol 13 (01) ◽  
pp. 67-86 ◽  
Author(s):  
Shin Kamada ◽  
Takumi Ichimura ◽  
Toshihide Harada
Keyword(s):  
Deep Learning ◽  
Network Structure ◽  
Classification Accuracy ◽  
Input Data ◽  
Knowledge Extraction ◽  
Deep Belief Network ◽  
Medical Examination ◽  
Support Vector ◽  
Structural Learning ◽  
Belief Network

Deep learning has a hierarchical network structure to represent multiple features of input data. The adaptive structural learning method of Deep Belief Network (DBN) can reach the high classification capability while searching the optimal network structure during the training. The method can find the optimal number of hidden neurons for given input data in a Restricted Boltzmann Machine (RBM) by neuron generation–annihilation algorithm, and generate a new hidden layer in DBN by the extension of the algorithm. In this paper, the proposed adaptive structural learning of DBN (Adaptive DBN) was applied to the comprehensive medical examination data for cancer prediction. The developed prediction system showed higher classification accuracy for test data (99.5% for the lung cancer and 94.3% for the stomach cancer) than the several learning methods such as traditional RBM, DBN, Non-Linear Support Vector Machine (SVM), and Convolutional Neural Network (CNN). Moreover, the explicit knowledge that makes the inference process of the trained DBN is required in deep learning. The binary patterns of activated neurons for given input in RBM and the hierarchical structure of DBN can represent the relation between input and output signals. These binary patterns were classified by C4.5 for knowledge extraction. Although the extracted knowledge showed slightly lower classification accuracy than the trained DBN network, it was able to improve inference speed by about 1/40. We report that the extracted IF-THEN rules from the trained DBN for medical examination data showed some interesting features related to initial condition of cancer.

scholarly journals A Novel Deep Belief Network Model Constructed by Improved Conditional RBMs and its Application in RUL Prediction for Hydraulic Pumps

2020 ◽  
Vol 25 (3) ◽  
pp. 373-382
Author(s):  
He Yu ◽  
Zaike Tian ◽  
Hongru Li ◽  
Baohua Xu ◽  
Guoqing An
Keyword(s):  
Deep Learning ◽  
Graph Model ◽  
Parameter Tuning ◽  
Deep Belief Network ◽  
Support Vector ◽  
Belief Network ◽  
Deep Feature ◽  
Proposed Model ◽  
Vector Machines ◽  
Deep Feature Extraction

Residual Useful Life (RUL) prediction is a key step of Condition-Based Maintenance (CBM). Deep learning-based techniques have shown wonderful prospects on RUL prediction, although their performances depend on heavy structures and parameter tuning strategies of these deep-learning models. In this paper, we propose a novel Deep Belief Network (DBN) model constructed by improved conditional Restrict Boltzmann Machines (RBMs) and apply it in RUL prediction for hydraulic pumps. DBN is a deep probabilistic digraph neural network that consists of multiple layers of RBMs. Since RBM is an undirected graph model and there is no communication among the nodes of the same layer, the deep feature extraction capability of the original DBN model can hardly ensure the accuracy of modeling continuous data. To address this issue, the DBN model is improved by replacing RBM with the Improved Conditional RBM (ICRBM) that adds timing linkage factors and constraint variables among the nodes of the same layers on the basis of RBM. The proposed model is applied to RUL prediction of hydraulic pumps, and the results show that the prediction model proposed in this paper has higher prediction accuracy compared with traditional DBNs, BP networks, support vector machines and modified DBNs such as DEBN and GC-DBN.

scholarly journals Deep belief network-based drug identification using near infrared spectroscopy

2017 ◽  
Vol 10 (02) ◽  
pp. 1630011 ◽  
Author(s):  
Huihua Yang ◽  
Baichao Hu ◽  
Xipeng Pan ◽  
Shengke Yan ◽  
Yanchun Feng ◽  
...  
Keyword(s):  
Neural Network ◽  
Infrared Spectroscopy ◽  
Near Infrared Spectroscopy ◽  
Bp Neural Network ◽  
Classification Accuracy ◽  
Near Infrared ◽  
Deep Belief Network ◽  
Support Vector ◽  
Belief Network ◽  
Drug Identification

Near infrared spectroscopy (NIRS) analysis technology, combined with chemometrics, can be effectively used in quick and nondestructive analysis of quality and category. In this paper, an effective drug identification method by using deep belief network (DBN) with dropout mechanism (dropout-DBN) to model NIRS is introduced, in which dropout is employed to overcome the overfitting problem coming from the small sample. This paper tests proposed method under datasets of different sizes with the example of near infrared diffuse reflectance spectroscopy of erythromycin ethylsuccinate drugs and other drugs, aluminum and nonaluminum packaged. Meanwhile, it gives experiments to compare the proposed method’s performance with back propagation (BP) neural network, support vector machines (SVMs) and sparse denoising auto-encoder (SDAE). The results show that for both binary classification and multi-classification, dropout mechanism can improve the classification accuracy, and dropout-DBN can achieve best classification accuracy in almost all cases. SDAE is similar to dropout-DBN in the aspects of classification accuracy and algorithm stability, which are higher than that of BP neural network and SVM methods. In terms of training time, dropout-DBN model is superior to SDAE model, but inferior to BP neural network and SVM methods. Therefore, dropout-DBN can be used as a modeling tool with effective binary and multi-class classification performance on a spectrum sample set of small size.

Combined deep belief network in deep learning with affinity propagation clustering algorithm for roller bearings fault diagnosis without data label

2018 ◽  
Vol 25 (2) ◽  
pp. 473-482 ◽  
Author(s):  
Fan Xu ◽  
Peter W. Tse
Keyword(s):  
Deep Learning ◽  
Fault Diagnosis ◽  
Clustering Algorithm ◽  
Deep Belief Network ◽  
Affinity Propagation ◽  
Cluster Center ◽  
Support Vector ◽  
Belief Network ◽  
Roller Bearings ◽  
Clustering Model

Unlike many traditional feature extraction methods of vibration signal such as ensemble empirical mode decomposition (EEMD), deep belief network (DBN) in deep learning can extract the useful information automatically and reduce the reliance on experts, with signal processing technology, and troubleshooting experience. In conventional fault diagnosis, data labels are required for classifiers such as support vector machine, random forest, and artificial neural networks. These are usually based on expert knowledge, for training and testing. But the process is usually tedious. The clustering model, on the other hand, can finish the roller bearings fault diagnosis without data labels, which is more efficient. There are some common clustering models which include fuzzy C-means (FCM), Gustafson–Kessel (GK), Gath–Geva (GG) models, and affinity propagation (AP). Unlike FCM, GK, and GG, which require knowledge or experience to pre-set the number of cluster center points, AP clustering algorithm can obtain the cluster center point according to the responsibility and availability calculations for all data points automatically. To the best of the authors’ knowledge, AP is rarely used for fault diagnosis. In this paper, a method which combines DBN, with several hidden layers, and AP for roller bearings fault diagnosis is proposed. For data visualization, the principal component analysis (PCA) is deployed to reduce the dimension of the extracted feature. The first two principal components are employed as the input of the FCM, GK, GG, and AP models for roller bearings faults diagnosis. Compared with other combination models such as EEMD–PCA–FCM/GK/GG and DBN–PCA–FCM/GK/GG, the proposed method, from the experimental results, is superior to the aforementioned combination models.

scholarly journals Sentimen Analisis Tweet Berbahasa Indonesia Dengan Deep Belief Network

2017 ◽  
Vol 11 (2) ◽  
pp. 187 ◽  
Author(s):  
Ira Zulfa ◽  
Edi Winarko
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Data Representation ◽  
Deep Belief Network ◽  
Training Data ◽  
Classification Model ◽  
Experimental Result ◽  
Support Vector ◽  
Machine Method ◽  
Belief Network

Sentiment analysis is a computational research of opinion sentiment and emotion which is expressed in textual mode. Twitter becomes the most popular communication device among internet users. Deep Learning is a new area of machine learning research. It aims to move machine learning closer to its main goal, artificial intelligence. The purpose of deep learning is to change the manual of engineering with learning. At its growth, deep learning has algorithms arrangement that focus on non-linear data representation. One of the machine learning methods is Deep Belief Network (DBN). Deep Belief Network (DBN), which is included in Deep Learning method, is a stack of several algorithms with some extraction features that optimally utilize all resources. This study has two points. First, it aims to classify positive, negative, and neutral sentiments towards the test data. Second, it determines the classification model accuracy by using Deep Belief Network method so it would be able to be applied into the tweet classification, to highlight the sentiment class of training data tweet in Bahasa Indonesia. Based on the experimental result, it can be concluded that the best method in managing tweet data is the DBN method with an accuracy of 93.31%, compared with  Naive Bayes method which has an accuracy of 79.10%, and SVM (Support Vector Machine) method with an accuracy of 92.18%.

Deep learning-based axial capacity prediction for cold-formed steel channel sections using Deep Belief Network

Structures ◽  
2021 ◽  
Vol 33 ◽  
pp. 2792-2802
Author(s):  
Zhiyuan Fang ◽  
Krishanu Roy ◽  
Jiri Mares ◽  
Chiu-Wing Sham ◽  
Boshan Chen ◽  
...  
Keyword(s):  
Deep Learning ◽  
Deep Belief Network ◽  
Belief Network ◽  
Axial Capacity ◽  
Cold Formed Steel ◽  
Capacity Prediction

scholarly journals Fast outlier detection for high-dimensional data of wireless sensor networks

2020 ◽  
Vol 16 (10) ◽  
pp. 155014772096383
Author(s):  
Yan Qiao ◽  
Xinhong Cui ◽  
Peng Jin ◽  
Wu Zhang
Keyword(s):  
Wireless Sensor Networks ◽  
Support Vector Machine ◽  
Sensor Networks ◽  
Outlier Detection ◽  
Large Scale ◽  
Deep Belief Network ◽  
Wireless Sensor ◽  
High Dimensional ◽  
Support Vector ◽  
Belief Network

This article addresses the problem of outlier detection for wireless sensor networks. As increasing amounts of observational data are tending to be high-dimensional and large scale, it is becoming increasingly difficult for existing techniques to perform outlier detection accurately and efficiently. Although dimensionality reduction tools (such as deep belief network) have been utilized to compress the high-dimensional data to support outlier detection, these methods may not achieve the desired performance due to the special distribution of the compressed data. Furthermore, because most existed classification methods must solve a quadratic optimization problem in their training stage, they cannot perform well in large-scale datasets. In this article, we developed a new form of classification model called “deep belief network online quarter-sphere support vector machine,” which combines deep belief network with online quarter-sphere one-class support vector machine. Based on this model, we first propose a model training method that learns the radius of the quarter sphere by a sorting method. Then, an online testing method is proposed to perform online outlier detection without supervision. Finally, we compare the proposed method with the state of the arts using extensive experiments. The experimental results show that our method not only reduces the computational cost by three orders of magnitude but also improves the detection accuracy by 3%–5%.

Sign in / Sign up

Export Citation Format

Share Document