Information Theoretic Learning

2011 ◽  
pp. 902-909 ◽  
Author(s):  
Deniz Erdogmus ◽  
Jose C. Principe
Keyword(s):  
Neural Network ◽  
Least Squares ◽  
Adaptive Systems ◽  
Mean Squared Error ◽  
Learning Algorithms ◽  
Optimal Solution ◽  
Principal Component ◽  
Second Order ◽  
Linear Discriminant ◽  
Non Gaussian

Learning systems depend on three interrelated components: topologies, cost/performance functions, and learning algorithms. Topologies provide the constraints for the mapping, and the learning algorithms offer the means to find an optimal solution; but the solution is optimal with respect to what? Optimality is characterized by the criterion and in neural network literature, this is the least addressed component, yet it has a decisive influence in generalization performance. Certainly, the assumptions behind the selection of a criterion should be better understood and investigated. Traditionally, least squares has been the benchmark criterion for regression problems; considering classification as a regression problem towards estimating class posterior probabilities, least squares has been employed to train neural network and other classifier topologies to approximate correct labels. The main motivation to utilize least squares in regression simply comes from the intellectual comfort this criterion provides due to its success in traditional linear least squares regression applications – which can be reduced to solving a system of linear equations. For nonlinear regression, the assumption of Gaussianity for the measurement error combined with the maximum likelihood principle could be emphasized to promote this criterion. In nonparametric regression, least squares principle leads to the conditional expectation solution, which is intuitively appealing. Although these are good reasons to use the mean squared error as the cost, it is inherently linked to the assumptions and habits stated above. Consequently, there is information in the error signal that is not captured during the training of nonlinear adaptive systems under non-Gaussian distribution conditions when one insists on second-order statistical criteria. This argument extends to other linear-second-order techniques such as principal component analysis (PCA), linear discriminant analysis (LDA), and canonical correlation analysis (CCA). Recent work tries to generalize these techniques to nonlinear scenarios by utilizing kernel techniques or other heuristics. This begs the question: what other alternative cost functions could be used to train adaptive systems and how could we establish rigorous techniques for extending useful concepts from linear and second-order statistical techniques to nonlinear and higher-order statistical learning methodologies?

A Lyapunov Theory-Based Neural Network Approach for Face Recognition

2010 ◽  
pp. 23-48 ◽  
Author(s):  
Li-Minn Ang ◽  
King Hann Lim ◽  
Kah Phooi Seng ◽  
Siew Wen Chin
Keyword(s):  
Neural Network ◽  
Face Recognition ◽  
Principal Component ◽  
Lyapunov Stability Theory ◽  
Recognition System ◽  
Lyapunov Theory ◽  
Neural Network Approach ◽  
Linear Discriminant ◽  
Feature Based ◽  
Definition Of

This chapter presents a new face recognition system comprising of feature extraction and the Lyapunov theory-based neural network. It first gives the definition of face recognition which can be broadly divided into (i) feature-based approaches, and (ii) holistic approaches. A general review of both approaches will be given in the chapter. Face features extraction techniques including Principal Component Analysis (PCA) and Fisher’s Linear Discriminant (FLD) are discussed. Multilayered neural network (MLNN) and Radial Basis Function neural network (RBF NN) will be reviewed. Two Lyapunov theory-based neural classifiers: (i) Lyapunov theory-based RBF NN, and (ii) Lyapunov theory-based MLNN classifiers are designed based on the Lyapunov stability theory. The design details will be discussed in the chapter. Experiments are performed on two benchmark databases, ORL and Yale. Comparisons with some of the existing conventional techniques are given. Simulation results have shown good performance for face recognition using the Lyapunov theory-based neural network systems.

scholarly journals A Machine Learning Method for the Detection of Brown Core in the Chinese Pear Variety Huangguan Using a MOS-Based E-Nose

Sensors ◽  
2020 ◽  
Vol 20 (16) ◽  
pp. 4499 ◽  
Author(s):  
Hao Wei ◽  
Yu Gu
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Learning Algorithms ◽  
Economic Value ◽  
Back Propagation ◽  
Principal Component ◽  
Back Propagation Neural Network ◽  
Machine Learning Algorithms ◽  
Oxide Semiconductor ◽  
Learning Machine

The brown core is an internal disorder that significantly affects the palatability and economic value of Chinese pears. In this study, a framework that includes a back-propagation neural network (BPNN) and extreme learning machine (ELM) (BP-ELMNN) was proposed for the detection of brown core in the Chinese pear variety Huangguan. The odor data of pear were collected using a metal oxide semiconductor (MOS) electronic nose (E-nose). Principal component analysis was used to analyze the complexity of the odor emitted by pears with brown cores. The performances of several machine learning algorithms, i.e., radial basis function neural network (RBFNN), BPNN, and ELM, were compared with that of the BP-ELMNN. The experimental results showed that the proposed framework provided the best results for the test samples, with an accuracy of 0.9683, a macro-precision of 0.9688, a macro-recall of 0.9683, and a macro-F1 score of 0.9685. The results demonstrate that the use of machine learning algorithms for the analysis of E-nose data is a feasible and non-destructive method to detect brown core in pears.

PSO-Based Uncorrelated Hybrid Discriminant Analysis Algorithm

2011 ◽  
Vol 109 ◽  
pp. 671-675 ◽  
Author(s):  
Xiao Ping Liu ◽  
Gui Yun Xu
Keyword(s):  
Discriminant Analysis ◽  
Optimization Problem ◽  
Optimal Solution ◽  
Principal Component ◽  
Small Sample ◽  
Model Parameters ◽  
Orthogonal Condition ◽  
Linear Discriminant ◽  
Analysis Algorithm ◽  
Discriminant Information

Hybrid discriminant analysis (HDA) can overcome small sample problems and outperform PCA and LDA by unifying principal component analysis (PCA) and linear discriminant analysis (LDA) in a single framework. However, the existing HDA algorithm can’t extract more discriminant information from dataset, and model parameters are difficult to select. To deal with the above problems, a particle swarm optimal (PSO)-based uncorrelated hybrid discriminant analysis algorithm is presented. The conjugate orthogonal condition is added to optimization problem of HDA, PSO is explored to select optimal HDA parameters and the optimal solution can be achieved by solving eigenvalue problem. Simulation demonstrates merits of the proposed algorithm.

scholarly journals Neural Network Implementations for PCA and Its Extensions

2012 ◽  
Vol 2012 ◽  
pp. 1-19 ◽  
Author(s):  
Jialin Qiu ◽  
Hui Wang ◽  
Jiabin Lu ◽  
Biaobiao Zhang ◽  
K.-L. Du
Keyword(s):  
Neural Network ◽  
Minor Component ◽  
Principal Component ◽  
Component Analysis ◽  
Adaptive Signal Processing ◽  
Singular Value ◽  
Eigenvalue Decomposition ◽  
Blind Signal Separation ◽  
Neural Network Models ◽  
Linear Discriminant

Many information processing problems can be transformed into some form of eigenvalue or singular value problems. Eigenvalue decomposition (EVD) and singular value decomposition (SVD) are usually used for solving these problems. In this paper, we give an introduction to various neural network implementations and algorithms for principal component analysis (PCA) and its various extensions. PCA is a statistical method that is directly related to EVD and SVD. Minor component analysis (MCA) is a variant of PCA, which is useful for solving total least squares (TLSs) problems. The algorithms are typical unsupervised learning methods. Some other neural network models for feature extraction, such as localized methods, complex-domain methods, generalized EVD, and SVD, are also described. Topics associated with PCA, such as independent component analysis (ICA) and linear discriminant analysis (LDA), are mentioned in passing in the conclusion. These methods are useful in adaptive signal processing, blind signal separation (BSS), pattern recognition, and information compression.

BEASF-Based Image Enhancement for Caries Detection Using Multidimensional Projection and Neural Network

2018 ◽  
Vol 8 (2) ◽  
pp. 47-66
Author(s):  
Shashikant Patil ◽  
Vaishali Kulkarni ◽  
Archana Bhise
Keyword(s):  
Neural Network ◽  
Principal Component Analysis ◽  
False Negative ◽  
Research Work ◽  
False Negative Rate ◽  
Principal Component ◽  
Component Analysis ◽  
Oral Disease ◽  
Caries Detection ◽  
Linear Discriminant

Tooth caries or cavities diagnosing are concerned as the most significant research work, as this is the common oral disease suffered by humans. Many approaches have been proposed under the topics including demineralization and decaying as well. However, the imaging modalities often suffer from various critical or complex aspects that struggles the methods to attain accurate diagnosis. This article turns to introduce a new cavity diagnosis model with three phases: (i) pre-processing (ii) feature extraction (iii) classification. In the first phase, a new bi-histogram equalization with adaptive sigmoid functions (BEASF) is introduced to enhance the image quality followed by other enhancements models like grey thresholding and active contour. Then, the features are extracted using multilinear principal component analysis (MPCA). Further, the classification is done via neural network (NN) classifier. After the implementation, the proposed model compares its performance over other conventional methods like principal component analysis (PCA), linear discriminant analysis (LDA) and independent component analysis (ICA) and the performance of the approach is analyzed in terms of measures such as accuracy, sensitivity, specificity, precision, false positive rate (FPR), false negative rate (FNR), negative predictive value (NPV), false discovery rate (FDR), F1Score and Mathews correlation coefficient (MCC), and proves the superiority of proposed work.

scholarly journals Deteksi Penyakit Schistosomiasis Melalui Identifikasi Telur Cacing Pada Feses Manusia Menggunakan Probabilistic Neural Network (PNN)

2020 ◽  
Vol 14 (1) ◽  
pp. 49-56
Author(s):  
Windy Rusma Astuti ◽  
Hayani Anastasia ◽  
R Ratianingsih ◽  
J. W. Puspitaa ◽  
Samarang Samarang
Keyword(s):  
Neural Network ◽  
Principal Component Analysis ◽  
Discriminant Analysis ◽  
Ascaris Lumbricoides ◽  
Probabilistic Neural Network ◽  
Principal Component ◽  
Smoothing Parameter ◽  
Human Feces ◽  
Linear Discriminant ◽  
Ancylostoma Duodenale

Abstract Schistosomiasis is a zoonotic disease caused by a blood worm in the Trematode class of the genus Schistosoma that lives in a vein. This disease is one of the oldest and most important diseases in the world. In Indonesia, Schistosomiasis is caused by Schistosoma Japonicum Sp. This study focused on the detection of Schistosomiasis disease through identification of worm eggs found in human feces. Based on the result of the observations of the Schistosomiasis Laboratory in Kaduwaa and Dodolo Villages in North Lore Subdistrict, Poso Regency it was found the worm eggs of other species in feces of resident in Kaduwaa and Dodolo villages, namely Ascaris Lumbricoides worm eggs and Ancylostoma Duodenale worm eggs. Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) methods are used to extract the egg image for the identification process, while Probabilistic Neural Network (PNN) methods were used to classify the species of the egg. The identification results are influenced by image capture techniques, image cutting techniques, the pixel size in the image, smoothing parameter values, and the number of sample images that used to train and test the data. The average accuracy of worm egg images identification using PNN is 98% with using the value of smoothing parameters 0,2. This result also shows that the Probabilistic Neural Network (PNN) method could be applied to identify the image of worm eggs found in human feces.  Abstrak Schistosomiasis merupakan penyakit zoonosis yang disebabkan oleh cacing darah kelas Trematoda dari genus Schistosoma yang tinggal dalam pembuluh darah vena. Penyakit ini merupakan salah satu penyakit tertua dan paling penting di dunia. Di Indonesia, Schistosomiasis disebabkan oleh cacing Schistosoma Japonicum Sp. Penelitian ini berfokus pada deteksi penyakit Schistosomiasis melalui identifikasi telur cacing yang terdapat pada feses manusia. Hasil observasi di Laboratorium Schistosomiasis desa Kaduwaa dan Desa Dodolo Kecamatan Lore Utara Kabupaten Poso memperlihatkan ditemukannya pula telur cacing dari spesies lain pada feses masyarakat desa Kaduwaa dan Desa Dodolo, yaitu telur cacing Ascaris Lumbricoides dan Ancylostoma Duodenale. Metode Principal Component Analysis (PCA) dan Linear Discriminant Analysis (LDA) digunakan untuk ekstraksi citra telur dalam proses identifikasi, sementara metode Probabilistic Neural Network (PNN) digunakan untuk klasifikasi spesies telur. Hasil identifikasi dipengaruhi oleh teknik pengambilann citra, teknik pemotongan citra, besarnya piksel pada citra, nilai smoothing parameter, serta jumlah citra sampel yang digunakan untuk data pelatihan dan pengujian. Akurasi rata-rata identifikasi citra telur cacing menggunakan PNN tertinggi yaitu  dengan menggunakan nilai smoothing parameter . Hal ini menunjukkan bahwa metode Probabilistic Neural Network (PNN) dapat diterapkan untuk identifikasi citra telur cacing yang terdapat pada feses manusia.

Sign in / Sign up

Export Citation Format

Share Document