scholarly journals A Modified Algorithm for Generalized Discriminant Analysis

2004 ◽  
Vol 16 (6) ◽  
pp. 1283-1297 ◽  
Author(s):  
Wenming Zheng ◽  
Li Zhao ◽  
Cairong Zou
Keyword(s):  
Discriminant Analysis ◽  
Theoretical Analysis ◽  
Face Database ◽  
Kernel Trick ◽  
Linear Discriminant ◽  
Degenerate Eigenvalue ◽  
Discriminant Ability ◽  
Previous Algorithm ◽  
Modified Algorithm ◽  
Generalized Discriminant Analysis

Generalized discriminant analysis (GDA) is an extension of the classical linear discriminant analysis (LDA) from linear domain to a nonlinear domain via the kernel trick. However, in the previous algorithm of GDA, the solutions may suffer from the degenerate eigenvalue problem (i.e., several eigenvectors with the same eigenvalue), which makes them not optimal in terms of the discriminant ability. In this letter, we propose a modified algorithm for GDA (MGDA) to solve this problem. The MGDA method aims to remove the degeneracy of GDA and find the optimal discriminant solutions, which maximize the between-class scatter in the subspace spanned by the degenerate eigenvectors of GDA. Theoretical analysis and experimental results on the ORL face database show that the MGDA method achieves better performance than the GDA method.

scholarly journals Generalized Discriminant Analysis Using a Kernel Approach

2000 ◽  
Vol 12 (10) ◽  
pp. 2385-2404 ◽  
Author(s):  
G. Baudat ◽  
F. Anouar
Keyword(s):  
Discriminant Analysis ◽  
Simulated Data ◽  
Feature Space ◽  
Real Data ◽  
Decision Function ◽  
Support Vector ◽  
Linear Discriminant ◽  
Kernel Approach ◽  
Generalized Discriminant Analysis ◽  
Nonlinear Discriminant Analysis

We present a new method that we call generalized discriminant analysis (GDA) to deal with nonlinear discriminant analysis using kernel function operator. The underlying theory is close to the support vector machines (SVM) insofar as the GDA method provides a mapping of the input vectors into high-dimensional feature space. In the transformed space, linear properties make it easy to extend and generalize the classical linear discriminant analysis (LDA) to nonlinear discriminant analysis. The formulation is expressed as an eigenvalue problem resolution. Using a different kernel, one can cover a wide class of nonlinearities. For both simulated data and alternate kernels, we give classification results, as well as the shape of the decision function. The results are confirmed using real data to perform seed classification.

scholarly journals De-biasing Covariance-Regularized Discriminant Analysis

2018 ◽  
Author(s):  
Haoyi Xiong ◽  
Wei Cheng ◽  
Yanjie Fu ◽  
Wenqing Hu ◽  
Jiang Bian ◽  
...  
Keyword(s):  
Discriminant Analysis ◽  
Theoretical Analysis ◽  
Classification Accuracy ◽  
Asymptotic Properties ◽  
Linear Discriminant ◽  
Classification Feature ◽  
Graphical Lasso ◽  
Inverse Covariance Matrix ◽  
Synthetic Datasets ◽  
Mean Vector

Fisher's Linear Discriminant Analysis (FLD) is a well-known technique for linear classification, feature extraction and dimension reduction. The empirical FLD relies on two key estimations from the data -- the mean vector for each class and the (inverse) covariance matrix. To improve the accuracy of FLD under the High Dimension Low Sample Size (HDLSS) settings, Covariance-Regularized FLD (CRLD) has been proposed to use shrunken covariance estimators, such as Graphical Lasso, to strike a balance between biases and variances. Though CRLD could obtain better classification accuracy, it usually incurs bias and converges to the optimal result with a slower asymptotic rate. Inspired by the recent progress in de-biased Lasso, we propose a novel FLD classifier, DBLD, which improves classification accuracy of CRLD through de-biasing. Theoretical analysis shows that DBLD possesses better asymptotic properties than CRLD. We conduct experiments on both synthetic datasets and real application datasets to confirm the correctness of our theoretical analysis and demonstrate the superiority of DBLD over classical FLD, CRLD and other downstream competitors under HDLSS settings.

scholarly journals A Similar Distribution Discriminant Analysis with Orthogonal and Nearly Statistically Uncorrelated Characteristics

2019 ◽  
Vol 2019 ◽  
pp. 1-10 ◽  
Author(s):  
Zhibo Guo ◽  
Ying Zhang
Keyword(s):  
Discriminant Analysis ◽  
Dimensionality Reduction ◽  
Linear Discriminant Analysis ◽  
High Dimensional Data ◽  
Sensor Data ◽  
High Dimensional ◽  
Similar Distribution ◽  
Face Database ◽  
Linear Discriminant ◽  
Reduction Methods

It is very difficult to process and analyze high-dimensional data directly. Therefore, it is necessary to learn a potential subspace of high-dimensional data through excellent dimensionality reduction algorithms to preserve the intrinsic structure of high-dimensional data and abandon the less useful information. Principal component analysis (PCA) and linear discriminant analysis (LDA) are two popular dimensionality reduction methods for high-dimensional sensor data preprocessing. LDA contains two basic methods, namely, classic linear discriminant analysis and FS linear discriminant analysis. In this paper, a new method, called similar distribution discriminant analysis (SDDA), is proposed based on the similarity of samples’ distribution. Furthermore, the method of solving the optimal discriminant vector is given. These discriminant vectors are orthogonal and nearly statistically uncorrelated. The disadvantages of PCA and LDA are overcome, and the extracted features are more effective by using SDDA. The recognition performance of SDDA exceeds PCA and LDA largely. Some experiments on the Yale face database, FERET face database, and UCI multiple features dataset demonstrate that the proposed method is effective. The results reveal that SDDA obtains better performance than comparison dimensionality reduction methods.

scholarly journals Convolutional 2D LDA for Nonlinear Dimensionality Reduction

2017 ◽  
Author(s):  
Qi Wang ◽  
Zequn Qin ◽  
Feiping Nie ◽  
Yuan Yuan
Keyword(s):  
Neural Networks ◽  
Discriminant Analysis ◽  
Convolutional Neural Networks ◽  
High Volume ◽  
Data Representation ◽  
Nonlinear Dimensionality Reduction ◽  
Complex Data ◽  
Two Dimensional ◽  
Linear Discriminant ◽  
Discriminant Ability

Representing high-volume and high-order data is an essential problem, especially in machine learning field. Although existing two-dimensional (2D) discriminant analysis achieves promising performance, the single and linear projection features make it difficult to analyze more complex data. In this paper, we propose a novel convolutional two-dimensional linear discriminant analysis (2D LDA) method for data representation. In order to deal with nonlinear data, a specially designed Convolutional Neural Networks (CNN) is presented, which can be proved having the equivalent objective function with common 2D LDA. In this way, the discriminant ability can benefit from not only the nonlinearity of Convolutional Neural Networks, but also the powerful learning process. Experiment results on several datasets show that the proposed method performs better than other state-of-the-art methods in terms of classification accuracy.

Face and Ethnical Group Recognition with Images of Different Resolutions

2021 ◽  
Vol 9 (1) ◽  
pp. 140-147
Author(s):  
Chong Lu ◽  
Yan Ren ◽  
Liying Han
Keyword(s):  
Face Recognition ◽  
Discriminant Analysis ◽  
Linear Discriminant Analysis ◽  
Least Squares ◽  
Partial Least Squares ◽  
Two Dimensional ◽  
Face Database ◽  
Linear Discriminant ◽  
Better Than

In this paper, a dataset for Xinjiang minority ethnical groups is introduced, and implementation of two dimensional Linear Discriminant Analysis (2DLDA) and two-dimensional Partial Least Squares (2DPLS) is investigated. Two important topics for face recognition and the ethnicity recognition are investigated for database with different image resolutions. Experiments show that 2DLDA performances better than 2DPLS on our face database.

scholarly journals Potencial Use of Near Infrared Spectroscopy (NIRS) to Categorise Chorizo Sausages from Iberian Pigs According to Several Quality Standards

2021 ◽  
Vol 11 (23) ◽  
pp. 11379
Author(s):  
Alberto Ortiz ◽  
Lucía León ◽  
Rebeca Contador ◽  
David Tejerina
Keyword(s):  
Infrared Spectroscopy ◽  
Discriminant Analysis ◽  
Near Infrared Spectroscopy ◽  
Near Infrared ◽  
External Validation ◽  
Partial Least Square ◽  
Least Square ◽  
Raw Material ◽  
Linear Discriminant ◽  
Discriminant Ability

The ability of Near Infrared Spectroscopy (NIRS) to classify pre-sliced Iberian chorizo modified atmosphere packaged (MAP) according to the animal material used in their production (Black, Red, White) in their production in accordance with the official trade categories (which includes the handling system and the different inter-racial crossbreeds) without opening the package was assayed. Furthermore, various spectra pre-treatments and supervised classification chemometric tools; Partial least square-discriminant analysis (PLS-DA), soft independent modelling of class analogies (SIMCA) and linear discriminant analysis (LDA), were assessed. The highest sensitivity values in both calibration and external validation were achieved with SIMCA followed by PLS-DA approaches, while LDA had more provided values among sensitivity and specificity and between the different commercial categories in both sample sets, thus yielding the highest discriminant ability. These results could be a resource to support the traceability and authentication control of individual pre-sliced MAP Iberian chorizo according to the commercial category of the raw material in a non-destructive way.

Class-Incremental Generalized Discriminant Analysis

2006 ◽  
Vol 18 (4) ◽  
pp. 979-1006 ◽  
Author(s):  
Wenming Zheng
Keyword(s):  
Discriminant Analysis ◽  
Eigenvalue Problem ◽  
Small Sample Size ◽  
Feature Space ◽  
Real Data ◽  
Qr Decomposition ◽  
Small Sample ◽  
Major Drawback ◽  
Linear Discriminant ◽  
Generalized Discriminant Analysis

Generalized discriminant analysis (GDA) is the nonlinear extension of the classical linear discriminant analysis (LDA) via the kernel trick. Mathematically, GDA aims to solve a generalized eigenequation problem, which is always implemented by the use of singular value decomposition (SVD) in the previously proposed GDA algorithms. A major drawback of SVD, however, is the difficulty of designing an incremental solution for the eigenvalue problem. Moreover, there are still numerical problems of computing the eigenvalue problem of large matrices. In this article, we propose another algorithm for solving GDA as for the case of small sample size problem, which applies QR decomposition rather than SVD. A major contribution of the proposed algorithm is that it can incrementally update the discriminant vectors when new classes are inserted into the training set. The other major contribution of this article is the presentation of the modified kernel Gram-Schmidt (MKGS) orthogonalization algorithm for implementing the QR decomposition in the feature space, which is more numerically stable than the kernel Gram-Schmidt (KGS) algorithm. We conduct experiments on both simulated and real data to demonstrate the better performance of the proposed methods.

Sign in / Sign up

Export Citation Format

Share Document