Dimensionality Reduction Techniques for High-dimensional Data in Precision Agriculture

2021 ◽  
pp. 28-39
Author(s):  
Mostafa Reisi-Gahrooei ◽  
James A. Whitehurst ◽  
Yiannis Ampatzidis ◽  
Panos Pardalos
Keyword(s):  
Dimensionality Reduction ◽  
Precision Agriculture ◽  
High Dimensional Data ◽  
High Dimensional ◽  
Reduction Techniques ◽  
Dimensionality Reduction Techniques

scholarly journals Performance Analysis of Dimensionality Reduction Techniques in the Context of Clustering

2019 ◽  
Vol 8 (S3) ◽  
pp. 66-71
Author(s):  
T. Sudha ◽  
P. Nagendra Kumar
Keyword(s):  
Principal Component Analysis ◽  
Dimensionality Reduction ◽  
High Dimensional Data ◽  
Principal Component ◽  
Component Analysis ◽  
High Dimensional ◽  
Reduction Techniques ◽  
Dimensionality Reduction Techniques ◽  
Low Dimensional ◽  
Probabilistic Principal Component Analysis

Data mining is one of the major areas of research. Clustering is one of the main functionalities of datamining. High dimensionality is one of the main issues of clustering and Dimensionality reduction can be used as a solution to this problem. The present work makes a comparative study of dimensionality reduction techniques such as t-distributed stochastic neighbour embedding and probabilistic principal component analysis in the context of clustering. High dimensional data have been reduced to low dimensional data using dimensionality reduction techniques such as t-distributed stochastic neighbour embedding and probabilistic principal component analysis. Cluster analysis has been performed on the high dimensional data as well as the low dimensional data sets obtained through t-distributed stochastic neighbour embedding and Probabilistic principal component analysis with varying number of clusters. Mean squared error; time and space have been considered as parameters for comparison. The results obtained show that time taken to convert the high dimensional data into low dimensional data using probabilistic principal component analysis is higher than the time taken to convert the high dimensional data into low dimensional data using t-distributed stochastic neighbour embedding.The space required by the data set reduced through Probabilistic principal component analysis is less than the storage space required by the data set reduced through t-distributed stochastic neighbour embedding.

scholarly journals A generalization of t-SNE and UMAP to single-cell multimodal omics

2021 ◽  
Vol 22 (1) ◽  
Author(s):  
Van Hoan Do ◽  
Stefan Canzar
Keyword(s):  
Dimensionality Reduction ◽  
Single Cell ◽  
Cell Types ◽  
High Dimensional ◽  
Omics Data ◽  
Relative Contribution ◽  
Reduction Techniques ◽  
Dimensionality Reduction Techniques ◽  
Concise Representation ◽  
Cellular Identity

AbstractEmerging single-cell technologies profile multiple types of molecules within individual cells. A fundamental step in the analysis of the produced high-dimensional data is their visualization using dimensionality reduction techniques such as t-SNE and UMAP. We introduce j-SNE and j-UMAP as their natural generalizations to the joint visualization of multimodal omics data. Our approach automatically learns the relative contribution of each modality to a concise representation of cellular identity that promotes discriminative features but suppresses noise. On eight datasets, j-SNE and j-UMAP produce unified embeddings that better agree with known cell types and that harmonize RNA and protein velocity landscapes.

scholarly journals Parallel Framework for Dimensionality Reduction of Large-Scale Datasets

2015 ◽  
Vol 2015 ◽  
pp. 1-12 ◽  
Author(s):  
Sai Kiranmayee Samudrala ◽  
Jaroslaw Zola ◽  
Srinivas Aluru ◽  
Baskar Ganapathysubramanian
Keyword(s):  
Dimensionality Reduction ◽  
Organic Solar Cells ◽  
Large Scale ◽  
Parallel Implementation ◽  
High Dimensional Data ◽  
Real Life ◽  
Processing Parameters ◽  
High Dimensional ◽  
Morphology Evolution ◽  
Reduction Techniques

Dimensionality reduction refers to a set of mathematical techniques used to reduce complexity of the original high-dimensional data, while preserving its selected properties. Improvements in simulation strategies and experimental data collection methods are resulting in a deluge of heterogeneous and high-dimensional data, which often makes dimensionality reduction the only viable way to gain qualitative and quantitative understanding of the data. However, existing dimensionality reduction software often does not scale to datasets arising in real-life applications, which may consist of thousands of points with millions of dimensions. In this paper, we propose a parallel framework for dimensionality reduction of large-scale data. We identify key components underlying the spectral dimensionality reduction techniques, and propose their efficient parallel implementation. We show that the resulting framework can be used to process datasets consisting of millions of points when executed on a 16,000-core cluster, which is beyond the reach of currently available methods. To further demonstrate applicability of our framework we perform dimensionality reduction of 75,000 images representing morphology evolution during manufacturing of organic solar cells in order to identify how processing parameters affect morphology evolution.

scholarly journals A SURVEY ON THE CURES FOR THE CURSE OF DIMENSIONALITY IN BIG DATA

2017 ◽  
Vol 10 (13) ◽  
pp. 355 ◽  
Author(s):  
Reshma Remesh ◽  
Pattabiraman. V
Keyword(s):  
Dimensionality Reduction ◽  
Input Data ◽  
Principal Component ◽  
Kernel Principal Component Analysis ◽  
High Dimensional ◽  
Data Sets ◽  
Learning Approaches ◽  
Data Set ◽  
Reduction Techniques ◽  
Dimensionality Reduction Techniques

Dimensionality reduction techniques are used to reduce the complexity for analysis of high dimensional data sets. The raw input data set may have large dimensions and it might consume time and lead to wrong predictions if unnecessary data attributes are been considered for analysis. So using dimensionality reduction techniques one can reduce the dimensions of input data towards accurate prediction with less cost. In this paper the different machine learning approaches used for dimensionality reductions such as PCA, SVD, LDA, Kernel Principal Component Analysis and Artificial Neural Network  have been studied.

scholarly journals Analysis of unsupervised dimensionality reduction techniques

2009 ◽  
Vol 6 (2) ◽  
pp. 217-227 ◽  
Author(s):  
Aswani Kumar
Keyword(s):  
Dimensionality Reduction ◽  
Approximation Error ◽  
High Dimensional ◽  
Retrieval Task ◽  
Document Collections ◽  
Noise Effects ◽  
Reduction Techniques ◽  
Dimensionality Reduction Techniques ◽  
Text Images ◽  
High Dimensional Datasets

Domains such as text, images etc contain large amounts of redundancies and ambiguities among the attributes which result in considerable noise effects (i.e. the data is high dimension). Retrieving the data from high dimensional datasets is a big challenge. Dimensionality reduction techniques have been a successful avenue for automatically extracting the latent concepts by removing the noise and reducing the complexity in processing the high dimensional data. In this paper we conduct a systematic study on comparing the unsupervised dimensionality reduction techniques for text retrieval task. We analyze these techniques from the view of complexity, approximation error and retrieval quality with experiments on four testing document collections.

Feature Selection

2011 ◽  
pp. 878-882
Author(s):  
Damien François
Keyword(s):  
Feature Selection ◽  
Time Series Prediction ◽  
High Dimensional Data ◽  
Principal Component ◽  
Point Of View ◽  
High Dimensional ◽  
Feature Subset ◽  
Selection Methods ◽  
Reduction Techniques ◽  
Dimensionality Reduction Techniques

In many applications, like function approximation, pattern recognition, time series prediction, and data mining, one has to build a model relating some features describing the data to some response value. Often, the features that are relevant for building the model are not known in advance. Feature selection methods allow removing irrelevant and/or redundant features to only keep the feature subset that are most useful to build a prediction model. The model is simpler and easier to interpret, reducing the risks of overfitting, non-convergence, etc. By contrast with other dimensionality reduction techniques such as principal component analysis or more recent nonlinear projection techniques (Lee & Verleysen 2007), which build a new, smaller set of features, the features that are selected by feature selection methods preserve their initial meaning, potentially bringing extra information about the process being modeled (Guyon 2006). Recently, the advent of high-dimensional data has raised new challenges for feature selection methods, both from the algorithmic point of view and the conceptual point of view (Liu & Motoda 2007). The problem of feature selection is exponential in nature, and many approximate algorithms are cubic with respect to the initial number of features, which may be intractable when the dimensionality of the data is large. Furthermore, high-dimensional data are often highly redundant, and two distinct subsets of features may have very similar predictive power, which can make it difficult to identify the best subset.

A Review on Dimensionality Reduction Techniques

2019 ◽  
Vol 33 (10) ◽  
pp. 1950017 ◽  
Author(s):  
Xuan Huang ◽  
Lei Wu ◽  
Yinsong Ye
Keyword(s):  
Pattern Recognition ◽  
Dimensionality Reduction ◽  
Implementation Process ◽  
Recognition System ◽  
Research Trend ◽  
High Dimensional ◽  
Open Problems ◽  
Reduction Techniques ◽  
Dimensionality Reduction Techniques ◽  
Scope Of Application

High-dimensional data is ubiquitous in scientific research and industrial production fields. It brings a lot of information to people, at the same time, because of its sparse and redundancy, it also brings great challenges to data mining and pattern recognition. Dimensionality reduction can reduce redundancy and noise, reduce the complexity of learning algorithms, and improve the accuracy of classification, it is an important and key step in pattern recognition system. In this paper, we overview the classical techniques for dimensionality reduction and review their properties, and categorize these techniques according to their implementation process. We deduce each algorithm in detail and intuitively show their underlying mathematical principles. Thereby, the focus is to uncover the optimization process for each technique. We compare the characteristics and limitations of each technique and summarize the scope of application, discussing a number of open problems and a perspective of research trend in future.

Sign in / Sign up

Export Citation Format

Share Document