scholarly journals Robust Graph Dimensionality Reduction

2018 ◽  
Author(s):  
Xiaofeng Zhu ◽  
Cong Lei ◽  
Hao Yu ◽  
Yonggang Li ◽  
Jiangzhang Gan ◽  
...  
Keyword(s):  
Dimensionality Reduction ◽  
Optimization Algorithm ◽  
Graph Embedding ◽  
Original Data ◽  
Transformation Matrix ◽  
High Dimensional ◽  
Local Similarity ◽  
Robust Estimators ◽  
Classification Tasks ◽  
Low Dimensional

In this paper, we propose conducting Robust Graph Dimensionality Reduction (RGDR) by learning a transformation matrix to map original high-dimensional data into their low-dimensional intrinsic space without the influence of outliers. To do this, we propose simultaneously 1) adaptively learning three variables, \ie a reverse graph embedding of original data, a transformation matrix, and a graph matrix preserving the local similarity of original data in their low-dimensional intrinsic space; and 2) employing robust estimators to  avoid outliers involving the processes of optimizing these three matrices. As a result, original data are cleaned by two strategies, \ie a prediction of original data based on three resulting variables and robust estimators, so that the transformation matrix can be learnt from accurately estimated intrinsic space with the helping of the reverse graph embedding and the graph matrix. Moreover, we propose a new optimization algorithm to the resulting objective function as well as theoretically prove the convergence of our optimization algorithm. Experimental results indicated that our proposed method outperformed all the comparison methods in terms of different classification tasks.

scholarly journals Unsupervised Text Feature Learning via Deep Variational Auto-encoder

2020 ◽  
Vol 49 (3) ◽  
pp. 421-437
Author(s):  
Genggeng Liu ◽  
Lin Xie ◽  
Chi-Hua Chen
Keyword(s):  
Dimensionality Reduction ◽  
High Dimensional Data ◽  
Image Data ◽  
Original Data ◽  
Feature Representation ◽  
High Dimensional ◽  
Learning To Learn ◽  
Text Feature ◽  
Reduction Methods ◽  
Low Dimensional

Dimensionality reduction plays an important role in the data processing of machine learning and data mining, which makes the processing of high-dimensional data more efficient. Dimensionality reduction can extract the low-dimensional feature representation of high-dimensional data, and an effective dimensionality reduction method can not only extract most of the useful information of the original data, but also realize the function of removing useless noise. The dimensionality reduction methods can be applied to all types of data, especially image data. Although the supervised learning method has achieved good results in the application of dimensionality reduction, its performance depends on the number of labeled training samples. With the growing of information from internet, marking the data requires more resources and is more difficult. Therefore, using unsupervised learning to learn the feature of data has extremely important research value. In this paper, an unsupervised multilayered variational auto-encoder model is studied in the text data, so that the high-dimensional feature to the low-dimensional feature becomes efficient and the low-dimensional feature can retain mainly information as much as possible. Low-dimensional feature obtained by different dimensionality reduction methods are used to compare with the dimensionality reduction results of variational auto-encoder (VAE), and the method can be significantly improved over other comparison methods.

scholarly journals DIMENSIONALITY REDUCTION VIA AN ORTHOGONAL AUTOENCODER APPROACH FOR HYPERSPECTRAL IMAGE CLASSIFICATION

2020 ◽  
Vol XLIII-B3-2020 ◽  
pp. 357-362
Author(s):  
V. H. Ayma ◽  
V. A. Ayma ◽  
J. Gutierrez
Keyword(s):  
Dimensionality Reduction ◽  
Hyperspectral Image ◽  
Original Data ◽  
Data Representation ◽  
Hyperspectral Data ◽  
Attractive Alternative ◽  
Kennedy Space Center ◽  
Classification Tasks ◽  
Multiple Levels ◽  
Kennedy Space

Abstract. Nowadays, the increasing amount of information provided by hyperspectral sensors requires optimal solutions to ease the subsequent analysis of the produced data. A common issue in this matter relates to the hyperspectral data representation for classification tasks. Existing approaches address the data representation problem by performing a dimensionality reduction over the original data. However, mining complementary features that reduce the redundancy from the multiple levels of hyperspectral images remains challenging. Thus, exploiting the representation power of neural networks based techniques becomes an attractive alternative in this matter. In this work, we propose a novel dimensionality reduction implementation for hyperspectral imaging based on autoencoders, ensuring the orthogonality among features to reduce the redundancy in hyperspectral data. The experiments conducted on the Pavia University, the Kennedy Space Center, and Botswana hyperspectral datasets evidence such representation power of our approach, leading to better classification performances compared to traditional hyperspectral dimensionality reduction algorithms.

Capturing discrete latent structures: choose LDs over PCs

Biostatistics ◽  
2021 ◽  
Author(s):  
Theresa A Alexander ◽  
Rafael A Irizarry ◽  
Héctor Corrada Bravo
Keyword(s):  
Dimensionality Reduction ◽  
Biological Data ◽  
Reduction Technique ◽  
Latent Structure ◽  
High Dimensional ◽  
Underlying Structure ◽  
Linear Transformations ◽  
Latent Structures ◽  
Low Dimensional ◽  
Discriminatory Information

Summary High-dimensional biological data collection across heterogeneous groups of samples has become increasingly common, creating high demand for dimensionality reduction techniques that capture underlying structure of the data. Discovering low-dimensional embeddings that describe the separation of any underlying discrete latent structure in data is an important motivation for applying these techniques since these latent classes can represent important sources of unwanted variability, such as batch effects, or interesting sources of signal such as unknown cell types. The features that define this discrete latent structure are often hard to identify in high-dimensional data. Principal component analysis (PCA) is one of the most widely used methods as an unsupervised step for dimensionality reduction. This reduction technique finds linear transformations of the data which explain total variance. When the goal is detecting discrete structure, PCA is applied with the assumption that classes will be separated in directions of maximum variance. However, PCA will fail to accurately find discrete latent structure if this assumption does not hold. Visualization techniques, such as t-Distributed Stochastic Neighbor Embedding (t-SNE) and Uniform Manifold Approximation and Projection (UMAP), attempt to mitigate these problems with PCA by creating a low-dimensional space where similar objects are modeled by nearby points in the low-dimensional embedding and dissimilar objects are modeled by distant points with high probability. However, since t-SNE and UMAP are computationally expensive, often a PCA reduction is done before applying them which makes it sensitive to PCAs downfalls. Also, tSNE is limited to only two or three dimensions as a visualization tool, which may not be adequate for retaining discriminatory information. The linear transformations of PCA are preferable to non-linear transformations provided by methods like t-SNE and UMAP for interpretable feature weights. Here, we propose iterative discriminant analysis (iDA), a dimensionality reduction technique designed to mitigate these limitations. iDA produces an embedding that carries discriminatory information which optimally separates latent clusters using linear transformations that permit post hoc analysis to determine features that define these latent structures.

scholarly journals Complex Moment-Based Supervised Eigenmap for Dimensionality Reduction

2019 ◽  
Vol 33 ◽  
pp. 3910-3918 ◽  
Author(s):  
Akira Imakura ◽  
Momo Matsuda ◽  
Xiucai Ye ◽  
Tetsuya Sakurai
Keyword(s):  
Dimensionality Reduction ◽  
Parallel Implementation ◽  
Dimensional Space ◽  
Recognition Performance ◽  
Optimization Methods ◽  
Original Data ◽  
Dimensional Subspace ◽  
Reduction Methods ◽  
Low Dimensional ◽  
Matrix Trace

Dimensionality reduction methods that project highdimensional data to a low-dimensional space by matrix trace optimization are widely used for clustering and classification. The matrix trace optimization problem leads to an eigenvalue problem for a low-dimensional subspace construction, preserving certain properties of the original data. However, most of the existing methods use only a few eigenvectors to construct the low-dimensional space, which may lead to a loss of useful information for achieving successful classification. Herein, to overcome the deficiency of the information loss, we propose a novel complex moment-based supervised eigenmap including multiple eigenvectors for dimensionality reduction. Furthermore, the proposed method provides a general formulation for matrix trace optimization methods to incorporate with ridge regression, which models the linear dependency between covariate variables and univariate labels. To reduce the computational complexity, we also propose an efficient and parallel implementation of the proposed method. Numerical experiments indicate that the proposed method is competitive compared with the existing dimensionality reduction methods for the recognition performance. Additionally, the proposed method exhibits high parallel efficiency.

scholarly journals Extracting Low-Dimensional Latent Structure from Time Series in the Presence of Delays

2015 ◽  
Vol 27 (9) ◽  
pp. 1825-1856 ◽  
Author(s):  
Karthik C. Lakshmanan ◽  
Patrick T. Sadtler ◽  
Elizabeth C. Tyler-Kabara ◽  
Aaron P. Batista ◽  
Byron M. Yu
Keyword(s):  
Time Series ◽  
Time Delay ◽  
Motor Cortex ◽  
Dimensionality Reduction ◽  
Gaussian Process ◽  
Latent Variables ◽  
Time Delays ◽  
High Dimensional ◽  
Latent Space ◽  
Low Dimensional

Noisy, high-dimensional time series observations can often be described by a set of low-dimensional latent variables. Commonly used methods to extract these latent variables typically assume instantaneous relationships between the latent and observed variables. In many physical systems, changes in the latent variables manifest as changes in the observed variables after time delays. Techniques that do not account for these delays can recover a larger number of latent variables than are present in the system, thereby making the latent representation more difficult to interpret. In this work, we introduce a novel probabilistic technique, time-delay gaussian-process factor analysis (TD-GPFA), that performs dimensionality reduction in the presence of a different time delay between each pair of latent and observed variables. We demonstrate how using a gaussian process to model the evolution of each latent variable allows us to tractably learn these delays over a continuous domain. Additionally, we show how TD-GPFA combines temporal smoothing and dimensionality reduction into a common probabilistic framework. We present an expectation/conditional maximization either (ECME) algorithm to learn the model parameters. Our simulations demonstrate that when time delays are present, TD-GPFA is able to correctly identify these delays and recover the latent space. We then applied TD-GPFA to the activity of tens of neurons recorded simultaneously in the macaque motor cortex during a reaching task. TD-GPFA is able to better describe the neural activity using a more parsimonious latent space than GPFA, a method that has been used to interpret motor cortex data but does not account for time delays. More broadly, TD-GPFA can help to unravel the mechanisms underlying high-dimensional time series data by taking into account physical delays in the system.

scholarly journals A Free Search Krill Herd Algorithm for Functions Optimization

2014 ◽  
Vol 2014 ◽  
pp. 1-21 ◽  
Author(s):  
Liangliang Li ◽  
Yongquan Zhou ◽  
Jian Xie
Keyword(s):  
Optimization Algorithm ◽  
Search Strategy ◽  
Optimal Solution ◽  
High Dimensional ◽  
Opposition Based Learning ◽  
Krill Herd Algorithm ◽  
Krill Herd ◽  
Local Optimal Solution ◽  
Free Search ◽  
Low Dimensional

To simulate the freedom and uncertain individual behavior of krill herd, this paper introduces the opposition based learning (OBL) strategy and free search operator into krill herd optimization algorithm (KH) and proposes a novel opposition-based free search krill herd optimization algorithm (FSKH). In FSKH, each krill individual can search according to its own perception and scope of activities. The free search strategy highly encourages the individuals to escape from being trapped in local optimal solution. So the diversity and exploration ability of krill population are improved. And FSKH can achieve a better balance between local search and global search. The experiment results of fourteen benchmark functions indicate that the proposed algorithm can be effective and feasible in both low-dimensional and high-dimensional cases. And the convergence speed and precision of FSKH are higher. Compared to PSO, DE, KH, HS, FS, and BA algorithms, the proposed algorithm shows a better optimization performance and robustness.

Macro Manifold Learning with Applications to Supervised Classification

2014 ◽  
Vol 556-562 ◽  
pp. 3590-3593
Author(s):  
Hong Bing Huang
Keyword(s):  
Pattern Recognition ◽  
Dimensionality Reduction ◽  
Manifold Learning ◽  
Supervised Classification ◽  
Experimental Results ◽  
Classification Tasks ◽  
Low Dimensional

Manifold learning has made many successful applications in the fields of dimensionality reduction and pattern recognition. However, when it is used for supervised classification, the result is still unsatisfactory. To address this challenge, a novel supervised approach, namely macro manifold learning (MML) is proposed. Based on the proposed approach, the low-dimensional embeddings of the testing samples is more favorable for classification tasks. Experimental results demonstrate the feasibility and effectiveness of our proposed approach.

scholarly journals SCDRHA: A scRNA-Seq Data Dimensionality Reduction Algorithm Based on Hierarchical Autoencoder

2021 ◽  
Vol 12 ◽  
Author(s):  
Jianping Zhao ◽  
Na Wang ◽  
Haiyun Wang ◽  
Chunhou Zheng ◽  
Yansen Su
Keyword(s):  
Dimensionality Reduction ◽  
Data Visualization ◽  
State Of The Art ◽  
Dimensional Space ◽  
High Dimensional ◽  
Reduction Algorithm ◽  
Cell Clustering ◽  
Data Dimensionality Reduction ◽  
Single Cell Rna Sequencing ◽  
Low Dimensional

Dimensionality reduction of high-dimensional data is crucial for single-cell RNA sequencing (scRNA-seq) visualization and clustering. One prominent challenge in scRNA-seq studies comes from the dropout events, which lead to zero-inflated data. To address this issue, in this paper, we propose a scRNA-seq data dimensionality reduction algorithm based on a hierarchical autoencoder, termed SCDRHA. The proposed SCDRHA consists of two core modules, where the first module is a deep count autoencoder (DCA) that is used to denoise data, and the second module is a graph autoencoder that projects the data into a low-dimensional space. Experimental results demonstrate that SCDRHA has better performance than existing state-of-the-art algorithms on dimension reduction and noise reduction in five real scRNA-seq datasets. Besides, SCDRHA can also dramatically improve the performance of data visualization and cell clustering.

scholarly journals Fault Identification Based on Local Feature Correlation

2022 ◽  
pp. 119-146
Author(s):  
Jing Wang ◽  
Jinglin Zhou ◽  
Xiaolu Chen
Keyword(s):  
Data Processing ◽  
High Dimension ◽  
Dimensional Space ◽  
Original Data ◽  
Mapping Method ◽  
Kernel Functions ◽  
High Dimensional ◽  
Monitoring Methods ◽  
Multivariate Statistical ◽  
Low Dimensional

AbstractIndustrial data variables show obvious high dimension and strong nonlinear correlation. Traditional multivariate statistical monitoring methods, such as PCA, PLS, CCA, and FDA, are only suitable for solving the high-dimensional data processing with linear correlation. The kernel mapping method is the most common technique to deal with the nonlinearity, which projects the original data in the low-dimensional space to the high-dimensional space through appropriate kernel functions so as to achieve the goal of linear separability in the new space. However, the space projection from the low dimension to the high dimension is contradictory to the actual requirement of dimensionality reduction of the data. So kernel-based method inevitably increases the complexity of data processing.

scholarly journals DATA DIMENSIONALITY REDUCTION FOR NEURAL BASED CLASSIFICATION OF OPTICAL SURFACES DEFECTS

2014 ◽  
pp. 32-42
Author(s):  
Matthieu Voiry ◽  
Kurosh Madani ◽  
Véronique Véronique Amarger ◽  
Joël Bernier
Keyword(s):  
Dimensionality Reduction ◽  
Harmful Effect ◽  
Optical Devices ◽  
High Dimensional ◽  
Major Step ◽  
Optical Surfaces ◽  
Data Dimensionality Reduction ◽  
Classification Tasks ◽  
Faults Diagnosis

A major step for high-quality optical surfaces faults diagnosis concerns scratches and digs defects characterization in products. This challenging operation is very important since it is directly linked with the produced optical component’s quality. A classification phase is mandatory to complete optical devices diagnosis since a number of correctable defects are usually present beside the potential “abiding” ones. Unfortunately relevant data extracted from raw image during defects detection phase are high dimensional. This can have harmful effect on the behaviors of artificial neural networks which are suitable to perform such a challenging classification. Reducing data dimension to a smaller value can decrease the problems related to high dimensionality. In this paper we compare different techniques which permit dimensionality reduction and evaluate their impact on classification tasks performances.

Sign in / Sign up

Export Citation Format

Share Document