scholarly journals stepwiseCM: An R Package for Stepwise Classification of Cancer Samples Using Multiple Heterogeneous Data Sets

2014 ◽  
Vol 13 ◽  
pp. CIN.S13075
Author(s):  
Askar Obulkasim ◽  
Mark A van de Wiel
Keyword(s):  
Waiting Times ◽  
High Dimensional Data ◽  
R Package ◽  
Heterogeneous Data ◽  
The Other ◽  
High Dimensional ◽  
Data Sets ◽  
Classification Problems ◽  
Data Types ◽  
Crucial Difference

This paper presents the R/Bioconductor package stepwiseCM, which classifies cancer samples using two heterogeneous data sets in an efficient way. The algorithm is able to capture the distinct classification power of two given data types without actually combining them. This package suits for classification problems where two different types of data sets on the same samples are available. One of these data types has measurements on all samples and the other one has measurements on some samples. One is easy to collect and/or relatively cheap (eg, clinical covariates) compared to the latter (high-dimensional data, eg, gene expression). One additional application for which stepwiseCM is proven to be useful as well is the combination of two high-dimensional data types, eg, DNA copy number and mRNA expression. The package includes functions to project the neighborhood information in one data space to the other to determine a potential group of samples that are likely to benefit most by measuring the second type of covariates. The two heterogeneous data spaces are connected by indirect mapping. The crucial difference between the stepwise classification strategy implemented in this package and the existing packages is that our approach aims to be cost-efficient by avoiding measuring additional covariates, which might be expensive or patient-unfriendly, for a potentially large subgroup of individuals. Moreover, in diagnosis for these individuals test, results would be quickly available, which may lead to reduced waiting times and hence lower the patients’ distress. The improvement described remedies the key limitations of existing packages, and facilitates the use of the stepwiseCM package in diverse applications.

scholarly journals Sparse classification with paired covariates

2019 ◽  
Vol 14 (3) ◽  
pp. 571-588
Author(s):  
Armin Rauschenberger ◽  
Iuliana Ciocănea-Teodorescu ◽  
Marianne A. Jonker ◽  
Renée X. Menezes ◽  
Mark A. van de Wiel
Keyword(s):  
Predictive Models ◽  
R Package ◽  
The Other ◽  
Adaptive Lasso ◽  
High Dimensional ◽  
Classification Problems ◽  
One To One ◽  
Single Response ◽  
Sparse Classification

AbstractThis paper introduces the paired lasso: a generalisation of the lasso for paired covariate settings. Our aim is to predict a single response from two high-dimensional covariate sets. We assume a one-to-one correspondence between the covariate sets, with each covariate in one set forming a pair with a covariate in the other set. Paired covariates arise, for example, when two transformations of the same data are available. It is often unknown which of the two covariate sets leads to better predictions, or whether the two covariate sets complement each other. The paired lasso addresses this problem by weighting the covariates to improve the selection from the covariate sets and the covariate pairs. It thereby combines information from both covariate sets and accounts for the paired structure. We tested the paired lasso on more than 2000 classification problems with experimental genomics data, and found that for estimating sparse but predictive models, the paired lasso outperforms the standard and the adaptive lasso. The R package is available from cran.

scholarly journals Cluster Weighted Model Based on TSNE Algorithm for High-Dimensional Data

2021 ◽  
Author(s):  
Kehinde Olobatuyi
Keyword(s):  
Mixture Models ◽  
Dimensional Space ◽  
High Dimensional Data ◽  
Expectation Maximization Algorithm ◽  
Real Data ◽  
R Package ◽  
High Dimensional ◽  
Data Sets ◽  
Dimensionality Reduction Technique ◽  
Weighted Model

Abstract Similar to many Machine Learning models, both accuracy and speed of the Cluster weighted models (CWMs) can be hampered by high-dimensional data, leading to previous works on a parsimonious technique to reduce the effect of ”Curse of dimensionality” on mixture models. In this work, we review the background study of the cluster weighted models (CWMs). We further show that parsimonious technique is not sufficient for mixture models to thrive in the presence of huge high-dimensional data. We discuss a heuristic for detecting the hidden components by choosing the initial values of location parameters using the default values in the ”FlexCWM” R package. We introduce a dimensionality reduction technique called T-distributed stochastic neighbor embedding (TSNE) to enhance the parsimonious CWMs in high-dimensional space. Originally, CWMs are suited for regression but for classification purposes, all multi-class variables are transformed logarithmically with some noise. The parameters of the model are obtained via expectation maximization algorithm. The effectiveness of the discussed technique is demonstrated using real data sets from different fields.

scholarly journals Assessing the shared variation among high-dimensional data matrices: a modified version of the Procrustean correlation coefficient

2019 ◽  
Author(s):  
E. Coissac ◽  
C. Gonindard-Melodelima
Keyword(s):  
Correlation Coefficient ◽  
Partial Correlation ◽  
High Dimensional Data ◽  
Correlation Coefficients ◽  
R Package ◽  
High Dimensional ◽  
Data Sets ◽  
High Dimension Data ◽  
Correct Estimation ◽  
Shared Variation

AbstractMotivationMolecular biology and ecology studies can produce high dimension data. Estimating correlations and shared variation between such data sets are an important step in disentangling the relationships between different elements of a biological system. Unfortunately, classical approaches are susceptible to producing falsely inferred correlations.ResultsHere we propose a corrected version of the Procrustean correlation coefficient that is robust to high dimensional data. This allows for a correct estimation of the shared variation between two data sets and the partial correlation coefficients between a set of matrix data.AvailabilityThe proposed corrected coefficients are implemented in the ProcMod R package available on CRAN. The git repository is hosted at https://git.metabarcoding.org/lecasofts/[email protected]

scholarly journals A High-Dimensional Modeling System Based on Analytical Hierarchy Process and Information Criteria

2021 ◽  
Vol 2021 ◽  
pp. 1-9
Author(s):  
Tuba Koç
Keyword(s):  
High Dimensional Data ◽  
Information Criteria ◽  
Classification Model ◽  
High Dimensional ◽  
Data Sets ◽  
Suitable Model ◽  
Classification Problems ◽  
Dimensional Modeling ◽  
Novel Approach ◽  
Hierarchy Process

High-dimensional data sets frequently occur in several scientific areas, and special techniques are required to analyze these types of data sets. Especially, it becomes important to apply a suitable model in classification problems. In this study, a novel approach is proposed to estimate a statistical model for high-dimensional data sets. The proposed method uses analytical hierarchical process (AHP) and information criteria for determining the optimal PCs for the classification model. The high-dimensional “colon” and “gravier” datasets were used in evaluation part. Application results demonstrate that the proposed approach can be successfully used for modeling purposes.

scholarly journals A primer on high-dimensional data analysis workflows for studying visual cortex development and plasticity

2019 ◽  
Author(s):  
Justin L. Balsor ◽  
David G. Jones ◽  
Kathryn M. Murphy
Keyword(s):  
Big Data ◽  
Visual Cortex ◽  
Clustering Algorithms ◽  
High Dimensional Data ◽  
R Package ◽  
High Dimensional ◽  
Data Sets ◽  
Data Set ◽  
Dimensional Changes ◽  
Or Genes

AbstractNew techniques for quantifying large numbers of proteins or genes are transforming the study of plasticity mechanisms in visual cortex (V1) into the era of big data. With those changes comes the challenge of applying new analytical methods designed for high-dimensional data. Studies of V1, however, can take advantage of the known functions that many proteins have in regulating experience-dependent plasticity to facilitate linking big data analyses with neurobiological functions. Here we discuss two workflows and provide example R code for analyzing high-dimensional changes in a group of proteins (or genes) using two data sets. The first data set includes 7 neural proteins, 9 visual conditions, and 3 regions in V1 from an animal model for amblyopia. The second data set includes 23 neural proteins and 31 ages (20d-80yrs) from human post-mortem samples of V1. Each data set presents different challenges and we describe using PCA, tSNE, and various clustering algorithms including sparse high-dimensional clustering. Also, we describe a new approach for identifying high-dimensional features and using them to construct a plasticity phenotype that identifies neurobiological differences among clusters. We include an R package “v1hdexplorer” that aggregates the various coding packages and custom visualization scripts written in R Studio.

Mahalanobis distance informed by clustering

2018 ◽  
Vol 8 (2) ◽  
pp. 377-406
Author(s):  
Almog Lahav ◽  
Ronen Talmon ◽  
Yuval Kluger
Keyword(s):  
Mahalanobis Distance ◽  
High Dimensional Data ◽  
Hidden Variables ◽  
Real Data ◽  
Risk Groups ◽  
High Dimensional ◽  
Data Sets ◽  
Kaplan Meier ◽  
Data Points ◽  
Survival Plot

Abstract A fundamental question in data analysis, machine learning and signal processing is how to compare between data points. The choice of the distance metric is specifically challenging for high-dimensional data sets, where the problem of meaningfulness is more prominent (e.g. the Euclidean distance between images). In this paper, we propose to exploit a property of high-dimensional data that is usually ignored, which is the structure stemming from the relationships between the coordinates. Specifically, we show that organizing similar coordinates in clusters can be exploited for the construction of the Mahalanobis distance between samples. When the observable samples are generated by a nonlinear transformation of hidden variables, the Mahalanobis distance allows the recovery of the Euclidean distances in the hidden space. We illustrate the advantage of our approach on a synthetic example where the discovery of clusters of correlated coordinates improves the estimation of the principal directions of the samples. Our method was applied to real data of gene expression for lung adenocarcinomas (lung cancer). By using the proposed metric we found a partition of subjects to risk groups with a good separation between their Kaplan–Meier survival plot.

Sign in / Sign up

Export Citation Format

Share Document