scholarly journals Deep Restricted Kernel Machines Using Conjugate Feature Duality

2017 ◽  
Vol 29 (8) ◽  
pp. 2123-2163 ◽  
Author(s):  
Johan A. K. Suykens
Keyword(s):  
Deep Learning ◽  
Least Squares ◽  
Principal Component ◽  
Feedforward Neural Networks ◽  
Kernel Principal Component Analysis ◽  
Support Vector ◽  
Kernel Machines ◽  
Continuous Variables ◽  
Restricted Boltzmann Machines ◽  
Boltzmann Machines

The aim of this letter is to propose a theory of deep restricted kernel machines offering new foundations for deep learning with kernel machines. From the viewpoint of deep learning, it is partially related to restricted Boltzmann machines, which are characterized by visible and hidden units in a bipartite graph without hidden-to-hidden connections and deep learning extensions as deep belief networks and deep Boltzmann machines. From the viewpoint of kernel machines, it includes least squares support vector machines for classification and regression, kernel principal component analysis (PCA), matrix singular value decomposition, and Parzen-type models. A key element is to first characterize these kernel machines in terms of so-called conjugate feature duality, yielding a representation with visible and hidden units. It is shown how this is related to the energy form in restricted Boltzmann machines, with continuous variables in a nonprobabilistic setting. In this new framework of so-called restricted kernel machine (RKM) representations, the dual variables correspond to hidden features. Deep RKM are obtained by coupling the RKMs. The method is illustrated for deep RKM, consisting of three levels with a least squares support vector machine regression level and two kernel PCA levels. In its primal form also deep feedforward neural networks can be trained within this framework.

scholarly journals On the Sparseness and Generalization Capability of Least Squares Support Vector Machines

2015 ◽  
Vol 3 (3) ◽  
pp. 279-288 ◽  
Author(s):  
Aijun Yan ◽  
Xiaoqian Huang ◽  
Hongshan Shao
Keyword(s):  
Support Vector Machines ◽  
Least Squares ◽  
Principal Component ◽  
Training Data ◽  
Kernel Principal Component Analysis ◽  
Function Parameter ◽  
Support Vector ◽  
Generalization Capability ◽  
Penalty Factor ◽  
Vector Machines

AbstractCompared with standard support vector machines (SVM), sparseness is lost in the modeling process of least squares support vector machines (LS-SVM), causing limited generalization capability. An improved method using quadratic renyi-entropy pruning is presented to deal with the above problems. First, a kernel principal component analysis (KPCA) is used to denoise the training data. Next, the authors use the genetic algorithm to estimate and optimize the kernel function parameter and penalty factor. Then, pick the subset that has the largest quadratic entropy to train and prune, and repeat this process until the cumulative error rate reaches the condition requirement. Finally, comparing experiments on the data classification and regression indicates that the proposed method is effective and may improve the sparseness and the generalization capability of LS-SVM model.

Deep Learning Associated with Laser-Induced Breakdown Spectroscopy (LIBS) for the Prediction of Lead in Soil

2019 ◽  
Vol 73 (5) ◽  
pp. 565-573 ◽  
Author(s):  
Yun Zhao ◽  
Mahamed Lamine Guindo ◽  
Xing Xu ◽  
Miao Sun ◽  
Jiyu Peng ◽  
...  
Keyword(s):  
Deep Learning ◽  
Confusion Matrix ◽  
Principal Component ◽  
Classification Performance ◽  
Soil Samples ◽  
Laser Induced Breakdown Spectroscopy ◽  
Support Vector ◽  
Breakdown Spectroscopy ◽  
Laser Induced Breakdown ◽  
Different Levels

In this study, a method based on laser-induced breakdown spectroscopy (LIBS) was developed to detect soil contaminated with Pb. Different levels of Pb were added to soil samples in which tobacco was planted over a period of two to four weeks. Principal component analysis and deep learning with a deep belief network (DBN) were implemented to classify the LIBS data. The robustness of the method was verified through a comparison with the results of a support vector machine and partial least squares discriminant analysis. A confusion matrix of the different algorithms shows that the DBN achieved satisfactory classification performance on all samples of contaminated soil. In terms of classification, the proposed method performed better on samples contaminated for four weeks than on those contaminated for two weeks. The results show that LIBS can be used with deep learning for the detection of heavy metals in soil.

scholarly journals An Electronic Component Recognition Algorithm Based on Deep Learning with a Faster SqueezeNet

2020 ◽  
Vol 2020 ◽  
pp. 1-11
Author(s):  
Yuanyuan Xu ◽  
Genke Yang ◽  
Jiliang Luo ◽  
Jianan He
Keyword(s):  
Deep Learning ◽  
Characteristic Curve ◽  
Area Under The Curve ◽  
Principal Component ◽  
Recognition Algorithm ◽  
Electronic Component ◽  
Support Vector ◽  
Learning Networks ◽  
Component Recognition ◽  
And Performance

Electronic component recognition plays an important role in industrial production, electronic manufacturing, and testing. In order to address the problem of the low recognition recall and accuracy of traditional image recognition technologies (such as principal component analysis (PCA) and support vector machine (SVM)), this paper selects multiple deep learning networks for testing and optimizes the SqueezeNet network. The paper then presents an electronic component recognition algorithm based on the Faster SqueezeNet network. This structure can reduce the size of network parameters and computational complexity without deteriorating the performance of the network. The results show that the proposed algorithm performs well, where the Receiver Operating Characteristic Curve (ROC) and Area Under the Curve (AUC), capacitor and inductor, reach 1.0. When the FPR is less than or equal 10 − 6   level, the TPR is greater than or equal to 0.99; its reasoning time is about 2.67 ms, achieving the industrial application level in terms of time consumption and performance.

scholarly journals Research on Recognition Method of Driving Fatigue State Based on Sample Entropy and Kernel Principal Component Analysis

Entropy ◽  
2018 ◽  
Vol 20 (9) ◽  
pp. 701 ◽  
Author(s):  
Beige Ye ◽  
Taorong Qiu ◽  
Xiaoming Bai ◽  
Ping Liu
Keyword(s):  
Principal Component Analysis ◽  
Recognition Accuracy ◽  
Principal Component ◽  
Component Analysis ◽  
Sample Entropy ◽  
Kernel Principal Component Analysis ◽  
Support Vector ◽  
Recognition Method ◽  
State Recognition ◽  
Driving Fatigue

In view of the nonlinear characteristics of electroencephalography (EEG) signals collected in the driving fatigue state recognition research and the issue that the recognition accuracy of the driving fatigue state recognition method based on EEG is still unsatisfactory, this paper proposes a driving fatigue recognition method based on sample entropy (SE) and kernel principal component analysis (KPCA), which combines the advantage of the high recognition accuracy of sample entropy and the advantages of KPCA in dimensionality reduction for nonlinear principal components and the strong non-linear processing capability. By using support vector machine (SVM) classifier, the proposed method (called SE_KPCA) is tested on the EEG data, and compared with those based on fuzzy entropy (FE), combination entropy (CE), three kinds of entropies including SE, FE and CE that merged with KPCA. Experiment results show that the method is effective.

scholarly journals A Novel Design of Classification of Coronary Artery Disease Using Deep Learning and Data Mining Algorithms

2021 ◽  
Vol 35 (3) ◽  
pp. 209-215
Author(s):  
Pratibha Verma ◽  
Vineet Kumar Awasthi ◽  
Sanat Kumar Sahu
Keyword(s):  
Neural Network ◽  
Data Mining ◽  
Deep Learning ◽  
Regression Tree ◽  
Principal Component ◽  
Classification And Regression Tree ◽  
Support Vector ◽  
Data Mining Algorithms ◽  
R Programming ◽  
Hidden Layer

Data mining techniques are included with Ensemble learning and deep learning for the classification. The methods used for classification are, Single C5.0 Tree (C5.0), Classification and Regression Tree (CART), kernel-based Support Vector Machine (SVM) with linear kernel, ensemble (CART, SVM, C5.0), Neural Network-based Fit single-hidden-layer neural network (NN), Neural Networks with Principal Component Analysis (PCA-NN), deep learning-based H2OBinomialModel-Deeplearning (HBM-DNN) and Enhanced H2OBinomialModel-Deeplearning (EHBM-DNN). In this study, experiments were conducted on pre-processed datasets using R programming and 10-fold cross-validation technique. The findings show that the ensemble model (CART, SVM and C5.0) and EHBM-DNN are more accurate for classification, compared with other methods.

scholarly journals Intelligent Prediction of Transmission Line Project Cost Based on Least Squares Support Vector Machine Optimized by Particle Swarm Optimization

2018 ◽  
Vol 2018 ◽  
pp. 1-11 ◽  
Author(s):  
Tao Yi ◽  
Hao Zheng ◽  
Yu Tian ◽  
Jin-peng Liu
Keyword(s):  
Support Vector Machine ◽  
Particle Swarm Optimization ◽  
Prediction Model ◽  
Transmission Line ◽  
Least Squares ◽  
Principal Component ◽  
Support Vector ◽  
Swarm Optimization ◽  
Empirical Parameter ◽  
Parameter Prediction

In order to meet the demand of power supply, the construction of transmission line projects is constantly advancing, and the level of cost control is constantly improving, which puts forward higher requirements for the accuracy of cost prediction. This paper proposes an intelligent cost prediction model based on least squares support vector machine (LSSVM) optimized by particle swarm optimization (PSO). Originally extracting natural, technological, and economic indexes from the perspective of cost composition, principal component analysis (PCA) is used to reduce the dimension of indexes. And PSO is innovatively introduced to optimize the parameters of LSSVM model to obtain the optimal parameters. The obtained principal component data are imported into empirical parameter LSSVM prediction model and the optimized parameter PSO-LSSVM prediction model, respectively, for modeling and prediction, and then comparing the prediction results to analyze the effect of model optimization. The results show that the absolute deviation of the optimized parameter prediction model is less than 9%. And the prediction accuracy of the optimized parameter prediction model is better than that of the empirical parameter model, which can provide a reliable basis for investment decision-making of transmission line projects.

Sign in / Sign up

Export Citation Format

Share Document