scholarly journals Searching for best lower dimensional visualization angles for high dimensional RNA-Seq data

PeerJ ◽  
2018 ◽  
Vol 6 ◽  
pp. e5199
Author(s):  
Wanli Zhang ◽  
Yanming Di
Keyword(s):  
Selection Criterion ◽  
High Dimensional ◽  
Data Sets ◽  
Complex Data ◽  
Rna Seq ◽  
High Dimensions ◽  
Data Set ◽  
Complex Data Sets ◽  
Hidden Patterns ◽  
Scatterplot Matrix

The accumulation of RNA sequencing (RNA-Seq) gene expression data in recent years has resulted in large and complex data sets of high dimensions. Exploratory analysis, including data mining and visualization, reveals hidden patterns and potential outliers in such data, but is often challenged by the high dimensional nature of the data. The scatterplot matrix is a commonly used tool for visualizing multivariate data, and allows us to view multiple bivariate relationships simultaneously. However, the scatterplot matrix becomes less effective for high dimensional data because the number of bivariate displays increases quadratically with data dimensionality. In this study, we introduce a selection criterion for each bivariate scatterplot and design/implement an algorithm that automatically scan and rank all possible scatterplots, with the goal of identifying the plots in which separation between two pre-defined groups is maximized. By applying our method to a multi-experimentArabidopsisRNA-Seq data set, we were able to successfully pinpoint the visualization angles where genes from two biological pathways are the most separated, as well as identify potential outliers.

Fuzzy C-Means in High Dimensional Spaces

2013 ◽  
pp. 1-16
Author(s):  
Roland Winkler ◽  
Frank Klawonn ◽  
Rudolf Kruse
Keyword(s):  
Objective Function ◽  
Local Minimum ◽  
High Dimensional ◽  
Controlled Environment ◽  
Data Sets ◽  
Local Minima ◽  
High Dimensions ◽  
Data Set ◽  
Fcm Algorithm ◽  
Centre Of Gravity

High dimensions have a devastating effect on the FCM algorithm and similar algorithms. One effect is that the prototypes run into the centre of gravity of the entire data set. The objective function must have a local minimum in the centre of gravity that causes FCM’s behaviour. In this paper, examine this problem. This paper answers the following questions: How many dimensions are necessary to cause an ill behaviour of FCM? How does the number of prototypes influence the behaviour? Why has the objective function a local minimum in the centre of gravity? How must FCM be initialised to avoid the local minima in the centre of gravity? To understand the behaviour of the FCM algorithm and answer the above questions, the authors examine the values of the objective function and develop three test environments that consist of artificially generated data sets to provide a controlled environment. The paper concludes that FCM can only be applied successfully in high dimensions if the prototypes are initialized very close to the cluster centres.

Alternative Clustering

2017 ◽  
pp. 1-12 ◽  
Author(s):  
Avinash Navlani ◽  
V. B. Gupta
Keyword(s):  
Research Problem ◽  
Research Community ◽  
Data Sets ◽  
Complex Data ◽  
High Quality ◽  
Data Set ◽  
Alternative Clustering ◽  
Complex Data Sets ◽  
Data Objects ◽  
Community Clustering

In the last couple of decades, clustering has become a very crucial research problem in the data mining research community. Clustering refers to the partitioning of data objects such as records and documents into groups or clusters of similar characteristics. Clustering is unsupervised learning, because of unsupervised nature there is no unique solution for all problems. Most of the time complex data sets require explanation in multiple clustering sets. All the Traditional clustering approaches generate single clustering. There is more than one pattern in a dataset; each of patterns can be interesting in from different perspectives. Alternative clustering intends to find all unlike groupings of the data set such that each grouping has high quality and distinct from each other. This chapter gives you an overall view of alternative clustering; it's various approaches, related work, comparing with various confusing related terms like subspace, multi-view, and ensemble clustering, applications, issues, and challenges.

Fuzzy C-Means in High Dimensional Spaces

2011 ◽  
Vol 1 (1) ◽  
pp. 1-16 ◽  
Author(s):  
Roland Winkler ◽  
Frank Klawonn ◽  
Rudolf Kruse
Keyword(s):  
Objective Function ◽  
Local Minimum ◽  
High Dimensional ◽  
Controlled Environment ◽  
Data Sets ◽  
Local Minima ◽  
High Dimensions ◽  
Data Set ◽  
Fcm Algorithm ◽  
Centre Of Gravity

High dimensions have a devastating effect on the FCM algorithm and similar algorithms. One effect is that the prototypes run into the centre of gravity of the entire data set. The objective function must have a local minimum in the centre of gravity that causes FCM’s behaviour. In this paper, examine this problem. This paper answers the following questions: How many dimensions are necessary to cause an ill behaviour of FCM? How does the number of prototypes influence the behaviour? Why has the objective function a local minimum in the centre of gravity? How must FCM be initialised to avoid the local minima in the centre of gravity? To understand the behaviour of the FCM algorithm and answer the above questions, the authors examine the values of the objective function and develop three test environments that consist of artificially generated data sets to provide a controlled environment. The paper concludes that FCM can only be applied successfully in high dimensions if the prototypes are initialized very close to the cluster centres.

scholarly journals The Impact of Normalization Methods on RNA-Seq Data Analysis

2015 ◽  
Vol 2015 ◽  
pp. 1-10 ◽  
Author(s):  
J. Zyprych-Walczak ◽  
A. Szabelska ◽  
L. Handschuh ◽  
K. Górczak ◽  
K. Klamecka ◽  
...  
Keyword(s):  
High Throughput Sequencing ◽  
Data Sets ◽  
Complex Data ◽  
Rna Seq ◽  
Medical Problems ◽  
Data Set ◽  
Normalization Methods ◽  
Wide Range ◽  
The Impact ◽  
Selection Of

High-throughput sequencing technologies, such as the Illumina Hi-seq, are powerful new tools for investigating a wide range of biological and medical problems. Massive and complex data sets produced by the sequencers create a need for development of statistical and computational methods that can tackle the analysis and management of data. The data normalization is one of the most crucial steps of data processing and this process must be carefully considered as it has a profound effect on the results of the analysis. In this work, we focus on a comprehensive comparison of five normalization methods related to sequencing depth, widely used for transcriptome sequencing (RNA-seq) data, and their impact on the results of gene expression analysis. Based on this study, we suggest a universal workflow that can be applied for the selection of the optimal normalization procedure for any particular data set. The described workflow includes calculation of the bias and variance values for the control genes, sensitivity and specificity of the methods, and classification errors as well as generation of the diagnostic plots. Combining the above information facilitates the selection of the most appropriate normalization method for the studied data sets and determines which methods can be used interchangeably.

Comparison of Multivariate Data Analysis Strategies for High-Content Screening

2011 ◽  
Vol 16 (3) ◽  
pp. 338-347 ◽  
Author(s):  
Anne Kümmel ◽  
Paul Selzer ◽  
Martin Beibel ◽  
Hanspeter Gubler ◽  
Christian N. Parker ◽  
...  
Keyword(s):  
Data Analysis ◽  
High Content Screening ◽  
Data Sets ◽  
Complex Data ◽  
Data Set ◽  
Processing Strategies ◽  
Analysis Strategies ◽  
Complex Data Sets ◽  
High Degree ◽  
Cell Data

High-content screening (HCS) is increasingly used in biomedical research generating multivariate, single-cell data sets. Before scoring a treatment, the complex data sets are processed (e.g., normalized, reduced to a lower dimensionality) to help extract valuable information. However, there has been no published comparison of the performance of these methods. This study comparatively evaluates unbiased approaches to reduce dimensionality as well as to summarize cell populations. To evaluate these different data-processing strategies, the prediction accuracies and the Z′ factors of control compounds of a HCS cell cycle data set were monitored. As expected, dimension reduction led to a lower degree of discrimination between control samples. A high degree of classification accuracy was achieved when the cell population was summarized on well level using percentile values. As a conclusion, the generic data analysis pipeline described here enables a systematic review of alternative strategies to analyze multiparametric results from biological systems.

scholarly journals Uncovering Proximity of Chromosome Territories using Classical Algebraic Statistics

2015 ◽  
Vol 6 (2) ◽  
Author(s):  
Javier Arsuaga ◽  
Ido Heskia ◽  
Serkan Hosten ◽  
Tatiana Maskalevich
Keyword(s):  
Statistical Significance ◽  
Three Dimensional ◽  
Spatial Proximity ◽  
Published Data ◽  
Algebraic Statistics ◽  
Data Sets ◽  
Complex Data ◽  
Data Set ◽  
Complex Data Sets ◽  
Human Blood Lymphocyte

Exchange type chromosome aberrations (ETCAs) are rearrangements of the genome that occur when chromosomes break and the resulting fragments rejoin with fragments from other chromosomes or from other regions within the same chromosome. ETCAs are commonly observed in cancer cells and in cells exposed to radiation. The frequency of these chromosome rearrangements is correlated with their spatial proximity, therefore it can be used to infer the three dimensional organization of the genome. Extracting statistical significance of spatial proximity from cancer and radiation data has remained somewhat elusive because of the sparsity of the data. We here propose a new approach to study the three dimensional organization of the genome using algebraic statistics. We test our method on a published data set of irradiated human blood lymphocyte cells. We provide a rigorous method for testing the overall organization of the genome, and in agreement with previous results we find a random relative positioning of chromosomes with the exception of the chromosome pairs {1,22} and {13,14} that have a significantly larger number of ETCAs than the rest of the chromosome pairs suggesting their spatial proximity. We conclude that algebraic methods can successfully be used to analyze genetic data and have potential applications to larger and more complex data sets. 

Representing Complex Data Sets as Virtual Structures

2002 ◽  
Vol 46 (17) ◽  
pp. 1641-1644
Author(s):  
Adam R. Richardson ◽  
Marvin J. Dainoff ◽  
Leonard S. Mark ◽  
James L. Smart ◽  
Niles Davis
Keyword(s):  
Relative Size ◽  
Conceptual Structure ◽  
Research Strategy ◽  
Data Sets ◽  
Complex Data ◽  
Initial Condition ◽  
Data Set ◽  
Complex Data Sets ◽  
Viewing Perspective ◽  
Virtual Structures

This paper describes a research strategy in which complex data sets are represented as physical objects in a virtual 3-D environment. The advantage of such a representation is that it allows the observer to actively explore the virtual environment so that potential ambiguities found in typical 3-D projections could be resolved by the transformation resulting from the change in viewing perspective. The study reported here constitutes an initial condition in which subjects compared relative size of virtual cubes from two different viewpoints. These results can serve as a basis for construction of cube-like objects representing the underlying conceptual structure of a data set

scholarly journals Tissue-aware RNA-Seq processing and normalization for heterogeneous and sparse data

2016 ◽  
Author(s):  
Joseph N. Paulson ◽  
Cho-Yi Chen ◽  
Camila M. Lopes-Ramos ◽  
Marieke L Kuijjer ◽  
John Platig ◽  
...  
Keyword(s):  
Quality Control ◽  
Large Scale ◽  
Transcriptional Profiling ◽  
R Package ◽  
Heterogeneous Data ◽  
Data Sets ◽  
List Type ◽  
Complex Data ◽  
Rna Seq ◽  
Data Set

AbstractAlthough ultrahigh-throughput RNA-Sequencing has become the dominant technology for genome-wide transcriptional profiling, the vast majority of RNA-Seq studies typically profile only tens of samples, and most analytical pipelines are optimized for these smaller studies. However, projects are generating ever-larger data sets comprising RNA-Seq data from hundreds or thousands of samples, often collected at multiple centers and from diverse tissues. These complex data sets present significant analytical challenges due to batch and tissue effects, but provide the opportunity to revisit the assumptions and methods that we use to preprocess, normalize, and filter RNA-Seq data – critical first steps for any subsequent analysis. We find analysis of large RNA-Seq data sets requires both careful quality control and that one account for sparsity due to the heterogeneity intrinsic in multi-group studies. An R package instantiating our method for large-scale RNA-Seq normalization and preprocessing, YARN, is available at bioconductor.org/packages/yarn.HighlightsOverview of assumptions used in preprocessing and normalizationPipeline for preprocessing, quality control, and normalization of large heterogeneous dataA Bioconductor package for the YARN pipeline and easy manipulation of count dataPreprocessed GTEx data set using the YARN pipeline available as a resource

scholarly journals Sparse Structures for Multivariate Extremes

2020 ◽  
Vol 8 (1) ◽  
Author(s):  
Sebastian Engelke ◽  
Jevgenijs Ivanovs
Keyword(s):  
Graphical Models ◽  
Rare Events ◽  
Principal Component ◽  
Extreme Value Statistics ◽  
High Dimensional ◽  
Annual Review ◽  
Publication Date ◽  
Data Sets ◽  
Complex Data ◽  
Complex Data Sets

Extreme value statistics provides accurate estimates for the small occurrence probabilities of rare events. While theory and statistical tools for univariate extremes are well developed, methods for high-dimensional and complex data sets are still scarce. Appropriate notions of sparsity and connections to other fields such as machine learning, graphical models, and high-dimensional statistics have only recently been established. This article reviews the new domain of research concerned with the detection and modeling of sparse patterns in rare events. We first describe the different forms of extremal dependence that can arise between the largest observations of a multivariate random vector. We then discuss the current research topics, including clustering, principal component analysis, and graphical modeling for extremes. Identification of groups of variables that can be concomitantly extreme is also addressed. The methods are illustrated with an application to flood risk assessment. Expected final online publication date for the Annual Review of Statistics, Volume 8 is March 8, 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

scholarly journals Using Hadoop Technology to Overcome Big Data Problems by Choosing Proposed Cost-efficient Scheduler Algorithm for Heterogeneous Hadoop System (BD3)

2020 ◽  
pp. 58-84
Author(s):  
Abou_el_ela Abdou Hussein
Keyword(s):  
Big Data ◽  
Data Processing ◽  
Data Storage ◽  
Database Management System ◽  
Data Sets ◽  
Complex Data ◽  
Daily Data ◽  
Complex Data Sets ◽  
Cost Efficient ◽  
Hadoop System

Day by day advanced web technologies have led to tremendous growth amount of daily data generated volumes. This mountain of huge and spread data sets leads to phenomenon that called big data which is a collection of massive, heterogeneous, unstructured, enormous and complex data sets. Big Data life cycle could be represented as, Collecting (capture), storing, distribute, manipulating, interpreting, analyzing, investigate and visualizing big data. Traditional techniques as Relational Database Management System (RDBMS) couldn’t handle big data because it has its own limitations, so Advancement in computing architecture is required to handle both the data storage requisites and the weighty processing needed to analyze huge volumes and variety of data economically. There are many technologies manipulating a big data, one of them is hadoop. Hadoop could be understand as an open source spread data processing that is one of the prominent and well known solutions to overcome handling big data problem. Apache Hadoop was based on Google File System and Map Reduce programming paradigm. Through this paper we dived to search for all big data characteristics starting from first three V's that have been extended during time through researches to be more than fifty six V's and making comparisons between researchers to reach to best representation and the precise clarification of all big data V’s characteristics. We highlight the challenges that face big data processing and how to overcome these challenges using Hadoop and its use in processing big data sets as a solution for resolving various problems in a distributed cloud based environment. This paper mainly focuses on different components of hadoop like Hive, Pig, and Hbase, etc. Also we institutes absolute description of Hadoop Pros and cons and improvements to face hadoop problems by choosing proposed Cost-efficient Scheduler Algorithm for heterogeneous Hadoop system.

Sign in / Sign up

Export Citation Format

Share Document