Performance Analysis of Dimensionality Reduction Techniques in the Context of Clustering

Data mining is one of the major areas of research. Clustering is one of the main functionalities of datamining. High dimensionality is one of the main issues of clustering and Dimensionality reduction can be used as a solution to this problem. The present work makes a comparative study of dimensionality reduction techniques such as t-distributed stochastic neighbour embedding and probabilistic principal component analysis in the context of clustering. High dimensional data have been reduced to low dimensional data using dimensionality reduction techniques such as t-distributed stochastic neighbour embedding and probabilistic principal component analysis. Cluster analysis has been performed on the high dimensional data as well as the low dimensional data sets obtained through t-distributed stochastic neighbour embedding and Probabilistic principal component analysis with varying number of clusters. Mean squared error; time and space have been considered as parameters for comparison. The results obtained show that time taken to convert the high dimensional data into low dimensional data using probabilistic principal component analysis is higher than the time taken to convert the high dimensional data into low dimensional data using t-distributed stochastic neighbour embedding.The space required by the data set reduced through Probabilistic principal component analysis is less than the storage space required by the data set reduced through t-distributed stochastic neighbour embedding.

Download Full-text

Principal Component Analysis

ACM Computing Surveys ◽

10.1145/3447755 ◽

2021 ◽

Vol 54 (4) ◽

pp. 1-34

Author(s):

Felipe L. Gewers ◽

Gustavo R. Ferreira ◽

Henrique F. De Arruda ◽

Filipi N. Silva ◽

Cesar H. Comin ◽

...

Keyword(s):

Principal Component Analysis ◽

Dimensionality Reduction ◽

Real World ◽

Principal Component ◽

Component Analysis ◽

Basic Principles ◽

Data Standardization ◽

Reduction Techniques ◽

Dimensionality Reduction Techniques ◽

Real World Datasets

Principal component analysis (PCA) is often applied for analyzing data in the most diverse areas. This work reports, in an accessible and integrated manner, several theoretical and practical aspects of PCA. The basic principles underlying PCA, data standardization, possible visualizations of the PCA results, and outlier detection are subsequently addressed. Next, the potential of using PCA for dimensionality reduction is illustrated on several real-world datasets. Finally, we summarize PCA-related approaches and other dimensionality reduction techniques. All in all, the objective of this work is to assist researchers from the most diverse areas in using and interpreting PCA.

Download Full-text

Dimensionality Reduction Techniques for Visualizing Morphometric Data: Comparing Principal Component Analysis to Nonlinear Methods

Evolutionary Biology ◽

10.1007/s11692-018-9464-9 ◽

2018 ◽

Vol 46 (1) ◽

pp. 106-121 ◽

Cited By ~ 4

Author(s):

Trina Y. Du

Keyword(s):

Principal Component Analysis ◽

Dimensionality Reduction ◽

Principal Component ◽

Component Analysis ◽

Morphometric Data ◽

Nonlinear Methods ◽

Reduction Techniques ◽

Dimensionality Reduction Techniques

Download Full-text

Dimensionality reduction for EEG-based sleep stage detection: comparison of autoencoders, principal component analysis and factor analysis

Biomedical Engineering / Biomedizinische Technik ◽

10.1515/bmt-2020-0139 ◽

2020 ◽

Vol 0 (0) ◽

Author(s):

Alexandra-Maria Tăuţan ◽

Alessandro C. Rossi ◽

Ruben de Francisco ◽

Bogdan Ionescu

Keyword(s):

Principal Component Analysis ◽

Factor Analysis ◽

Dimensionality Reduction ◽

Promising Result ◽

Experimental Testing ◽

Sleep Stage ◽

Principal Component ◽

Component Analysis ◽

Reduction Techniques ◽

Dimensionality Reduction Techniques

AbstractMethods developed for automatic sleep stage detection make use of large amounts of data in the form of polysomnographic (PSG) recordings to build predictive models. In this study, we investigate the effect of several dimensionality reduction techniques, i.e., principal component analysis (PCA), factor analysis (FA), and autoencoders (AE) on common classifiers, e.g., random forests (RF), multilayer perceptron (MLP), long-short term memory (LSTM) networks, for automated sleep stage detection. Experimental testing is carried out on the MGH Dataset provided in the “You Snooze, You Win: The PhysioNet/Computing in Cardiology Challenge 2018”. The signals used as input are the six available (EEG) electoencephalographic channels and combinations with the other PSG signals provided: ECG – electrocardiogram, EMG – electromyogram, respiration based signals – respiratory efforts and airflow. We observe that a similar or improved accuracy is obtained in most cases when using all dimensionality reduction techniques, which is a promising result as it allows to reduce the computational load while maintaining performance and in some cases also improves the accuracy of automated sleep stage detection. In our study, using autoencoders for dimensionality reduction maintains the performance of the model, while using PCA and FA the accuracy of the models is in most cases improved.

Download Full-text

High-Dimensional Data Dimension Reduction Based on KECA

Applied Mechanics and Materials ◽

10.4028/www.scientific.net/amm.303-306.1101 ◽

2013 ◽

Vol 303-306 ◽

pp. 1101-1104 ◽

Cited By ~ 2

Author(s):

Yong De Hu ◽

Jing Chang Pan ◽

Xin Tan

Keyword(s):

Principal Component Analysis ◽

Dimension Reduction ◽

High Dimensional Data ◽

Principal Component ◽

Good Method ◽

Component Analysis ◽

Renyi Entropy ◽

Rényi Entropy ◽

Kernel Principal Component Analysis ◽

High Dimensional

Kernel entropy component analysis (KECA) reveals the original data’s structure by kernel matrix. This structure is related to the Renyi entropy of the data. KECA maintains the invariance of the original data’s structure by keeping the data’s Renyi entropy unchanged. This paper described the original data by several components on the purpose of dimension reduction. Then the KECA was applied in celestial spectra reduction and was compared with Principal Component Analysis (PCA) and Kernel Principal Component Analysis (KPCA) by experiments. Experimental results show that the KECA is a good method in high-dimensional data reduction.

Download Full-text

Principal Component Analysis (PCA) for high-dimensional data. PCA is dead. Long live PCA

Perspectives on Big Data Analysis - Contemporary Mathematics ◽

10.1090/conm/622/12430 ◽

2014 ◽

pp. 1-10 ◽

Cited By ~ 1

Author(s):

Fan Yang ◽

Kjell Doksum ◽

Kam-Wah Tsui

Keyword(s):

Principal Component Analysis ◽

High Dimensional Data ◽

Principal Component ◽

Component Analysis ◽

High Dimensional

Download Full-text

Object Tracking Using Probabilistic Principal Component Analysis Based on Particle Filtering Framework

Advanced Materials Research ◽

10.4028/www.scientific.net/amr.341-342.790 ◽

2011 ◽

Vol 341-342 ◽

pp. 790-797 ◽

Cited By ~ 2

Author(s):

Zhi Yan Xiang ◽

Tie Yong Cao ◽

Peng Zhang ◽

Tao Zhu ◽

Jing Feng Pan

Keyword(s):

Principal Component Analysis ◽

Object Tracking ◽

Particle Filtering ◽

Principal Component ◽

Component Analysis ◽

Dimensional Subspace ◽

Target Object ◽

Scale Invariant ◽

Low Dimensional ◽

Probabilistic Principal Component Analysis

In this paper, an object tracking approach is introduced for color video sequences. The approach presents the integration of color distributions and probabilistic principal component analysis (PPCA) into particle filtering framework. Color distributions are robust to partial occlusion, are rotation and scale invariant and are calculated efficiently. Principal Component Analysis (PCA) is used to update the eigenbasis and the mean, which can reflect the appearance changes of the tracked object. And a low dimensional subspace representation of PPCA efficiently adapts to these changes of appearance of the target object. At the same time, a forgetting factor is incorporated into the updating process, which can be used to economize on processing time and enhance the efficiency of object tracking. Computer simulation experiments demonstrate the effectiveness and the robustness of the proposed tracking algorithm when the target object undergoes pose and scale changes, defilade and complex background.

Download Full-text

Exploring High-Dimensional Biological Data with Sparse Contrastive Principal Component Analysis

10.1101/836650 ◽

2019 ◽

Cited By ~ 1

Author(s):

Philippe Boileau ◽

Nima S. Hejazi ◽

Sandrine Dudoit

Keyword(s):

Principal Component Analysis ◽

Dimensionality Reduction ◽

High Throughput Sequencing ◽

Principal Component ◽

Component Analysis ◽

Biological Data ◽

Sequencing Data ◽

Microarray Gene Expression ◽

Biological Signal ◽

Reduction Techniques

AbstractMotivationStatistical analyses of high-throughput sequencing data have re-shaped the biological sciences. In spite of myriad advances, recovering interpretable biological signal from data corrupted by technical noise remains a prevalent open problem. Several classes of procedures, among them classical dimensionality reduction techniques and others incorporating subject-matter knowledge, have provided effective advances; however, no procedure currently satisfies the dual objectives of recovering stable and relevant features simultaneously.ResultsInspired by recent proposals for making use of control data in the removal of unwanted variation, we propose a variant of principal component analysis, sparse contrastive principal component analysis, that extracts sparse, stable, interpretable, and relevant biological signal. The new methodology is compared to competing dimensionality reduction approaches through a simulation study as well as via analyses of several publicly available protein expression, microarray gene expression, and single-cell transcriptome sequencing datasets.AvailabilityA free and open-source software implementation of the methodology, the scPCA R package, is made available via the Bioconductor Project. Code for all analyses presented in the paper is also available via GitHub.

Download Full-text

Dimensionality Reduction with Sparse Locality for Principal Component Analysis

Mathematical Problems in Engineering ◽

10.1155/2020/9723279 ◽

2020 ◽

Vol 2020 ◽

pp. 1-12

Author(s):

Pei Heng Li ◽

Taeho Lee ◽

Hee Yong Youn

Keyword(s):

Principal Component Analysis ◽

Dimensionality Reduction ◽

Sparse Representation ◽

Principal Component ◽

Original Data ◽

Component Analysis ◽

Reconstruction Error ◽

Dimensional Representation ◽

Weighted Matrix ◽

Low Dimensional

Various dimensionality reduction (DR) schemes have been developed for projecting high-dimensional data into low-dimensional representation. The existing schemes usually preserve either only the global structure or local structure of the original data, but not both. To resolve this issue, a scheme called sparse locality for principal component analysis (SLPCA) is proposed. In order to effectively consider the trade-off between the complexity and efficiency, a robust L2,p-norm-based principal component analysis (R2P-PCA) is introduced for global DR, while sparse representation-based locality preserving projection (SR-LPP) is used for local DR. Sparse representation is also employed to construct the weighted matrix of the samples. Being parameter-free, this allows the construction of an intrinsic graph more robust against the noise. In addition, simultaneous learning of projection matrix and sparse similarity matrix is possible. Experimental results demonstrate that the proposed scheme consistently outperforms the existing schemes in terms of clustering accuracy and data reconstruction error.

Download Full-text