scholarly journals Embedding to reference t-SNE space addresses batch effects in single-cell classification

2021 ◽  
Author(s):  
Pavlin G. Poličar ◽  
Martin Stražar ◽  
Blaž Zupan
Keyword(s):  
Single Cell ◽  
Single Cells ◽  
Secondary Data ◽  
Machine Learning Techniques ◽  
Primary Data ◽  
Data Sets ◽  
Batch Effects ◽  
Data Set ◽  
Straightforward Application ◽  
Multiple Data Sets

AbstractDimensionality reduction techniques, such as t-SNE, can construct informative visualizations of high-dimensional data. When jointly visualising multiple data sets, a straightforward application of these methods often fails; instead of revealing underlying classes, the resulting visualizations expose dataset-specific clusters. To circumvent these batch effects, we propose an embedding procedure that uses a t-SNE visualization constructed on a reference data set as a scaffold for embedding new data points. Each data instance from a new, unseen, secondary data is embedded independently and does not change the reference embedding. This prevents any interactions between instances in the secondary data and implicitly mitigates batch effects. We demonstrate the utility of this approach by analyzing six recently published single-cell gene expression data sets with up to tens of thousands of cells and thousands of genes. The batch effects in our studies are particularly strong as the data comes from different institutions using different experimental protocols. The visualizations constructed by our proposed approach are clear of batch effects, and the cells from secondary data sets correctly co-cluster with cells of the same type from the primary data. We also show the predictive power of our simple, visual classification approach in t-SNE space matches the accuracy of specialized machine learning techniques that consider the entire compendium of features that profile single cells.

scholarly journals Embedding to Reference t-SNE Space Addresses Batch Effects in Single-Cell Classification

2019 ◽  
Author(s):  
Pavlin G. Poličar ◽  
Martin Stražar ◽  
Blaž Zupan
Keyword(s):  
Single Cell ◽  
Secondary Data ◽  
Primary Data ◽  
Data Sets ◽  
Batch Effects ◽  
Data Set ◽  
Reduction Techniques ◽  
Straightforward Application ◽  
Cell Gene Expression ◽  
Multiple Data Sets

AbstractDimensionality reduction techniques, such as t-SNE, can construct informative visualizations of high-dimensional data. When working with multiple data sets, a straightforward application of these methods often fails; instead of revealing underlying classes, the resulting visualizations expose data set-specific clusters. To circumvent these batch effects, we propose an embedding procedure that takes a t-SNE visualization constructed on a reference data set and uses it as a scaffold for embedding new data. The new, secondary data is embedded one data-point at the time. This prevents any interactions between instances in the secondary data and implicitly mitigates batch effects. We demonstrate the utility of this approach with an analysis of six recently published single-cell gene expression data sets containing up to tens of thousands of cells and thousands of genes. In these data sets, the batch effects are particularly strong as the data comes from different institutions and was obtained using different experimental protocols. The visualizations constructed by our proposed approach are cleared of batch effects, and the cells from secondary data sets correctly co-cluster with cells from the primary data sharing the same cell type.

scholarly journals CIM-seq

2021 ◽  
Author(s):  
Nathanael Andrews ◽  
Martin Enge
Keyword(s):  
Single Cell ◽  
Single Cells ◽  
Likelihood Estimation ◽  
Cell Types ◽  
Data Sets ◽  
Target Tissue ◽  
Data Set ◽  
Rnaseq Data ◽  
The Given ◽  
Cell Data

Abstract CIM-seq is a tool for deconvoluting RNA-seq data from cell multiplets (clusters of two or more cells) in order to identify physically interacting cell in a given tissue. The method requires two RNAseq data sets from the same tissue: one of single cells to be used as a reference, and one of cell multiplets to be deconvoluted. CIM-seq is compatible with both droplet based sequencing methods, such as Chromium Single Cell 3′ Kits from 10x genomics; and plate based methods, such as Smartseq2. The pipeline consists of three parts: 1) Dissociation of the target tissue, FACS sorting of single cells and multiplets, and conventional scRNA-seq 2) Feature selection and clustering of cell types in the single cell data set - generating a blueprint of transcriptional profiles in the given tissue 3) Computational deconvolution of multiplets through a maximum likelihood estimation (MLE) to determine the most likely cell type constituents of each multiplet.

scholarly journals Neural Data Visualization for Scalable and Generalizable Single Cell Analysis

2018 ◽  
Author(s):  
Hyunghoon Cho ◽  
Bonnie Berger ◽  
Jian Peng
Keyword(s):  
Single Cell ◽  
Single Cell Analysis ◽  
Single Cells ◽  
Data Sets ◽  
Cell Analysis ◽  
Data Set ◽  
Unseen Data ◽  
Sequencing Experiment ◽  
Cell Expression

SummarySingle-cell RNA sequencing is becoming effective and accessible as emerging technologies push its scale to millions of cells and beyond. Visualizing the landscape of single cell expression has been a fundamental tool in single cell analysis. However, standard methods for visualization, such as t-stochastic neighbor embedding (t-SNE), not only lack scalability to data sets with millions of cells, but also are unable to generalize to new cells, an important ability for transferring knowledge across fast-accumulating data sets. We introduce net-SNE, which trains a neural network to learn a high quality visualization of single cells that newly generalizes to unseen data. While matching the visualization quality of t-SNE on 14 benchmark data sets of varying sizes, from hundreds to 1.3 million cells, net-SNE also effectively positions previously unseen cells, even when an entire subtype is missing from the initial data set or when the new cells are from a different sequencing experiment. Furthermore, given a “reference” visualization, net-SNE can vastly reduce the computational burden of visualizing millions of single cells from multiple days to just a few minutes of runtime. Our work provides a general framework for newly bootstrapping single cell analysis from existing data sets.

scholarly journals A Distributed Content-Based Video Retrieval System for Large Data-sets

2021 ◽  
Author(s):  
ElMehdi SAOUDI ◽  
Said Jai Andaloussi
Keyword(s):  
Real Time ◽  
Retrieval System ◽  
Video Retrieval ◽  
Multimedia Databases ◽  
Video Data ◽  
Multimedia Data ◽  
Machine Learning Techniques ◽  
Data Sets ◽  
Data Set ◽  
Content Based Video Retrieval

Abstract With the rapid growth of the volume of video data and the development of multimedia technologies, it has become necessary to have the ability to accurately and quickly browse and search through information stored in large multimedia databases. For this purpose, content-based video retrieval ( CBVR ) has become an active area of research over the last decade. In this paper, We propose a content-based video retrieval system providing similar videos from a large multimedia data-set based on a query video. The approach uses vector motion-based signatures to describe the visual content and uses machine learning techniques to extract key-frames for rapid browsing and efficient video indexing. We have implemented the proposed approach on both, single machine and real-time distributed cluster to evaluate the real-time performance aspect, especially when the number and size of videos are large. Experiments are performed using various benchmark action and activity recognition data-sets and the results reveal the effectiveness of the proposed method in both accuracy and processing time compared to state-of-the-art methods.

scholarly journals Panoramic stitching of heterogeneous single-cell transcriptomic data

2018 ◽  
Author(s):  
Brian Hie ◽  
Bryan Bryson ◽  
Bonnie Berger
Keyword(s):  
Single Cell ◽  
Cell Types ◽  
Data Sets ◽  
Cell Type ◽  
Data Set ◽  
Wide Range ◽  
Data Set Integration ◽  
Biological Patterns ◽  
Insight Into ◽  
Comprehensive Reference

AbstractResearchers are generating single-cell RNA sequencing (scRNA-seq) profiles of diverse biological systems1–4 and every cell type in the human body.5 Leveraging this data to gain unprecedented insight into biology and disease will require assembling heterogeneous cell populations across multiple experiments, laboratories, and technologies. Although methods for scRNA-seq data integration exist6,7, they often naively merge data sets together even when the data sets have no cell types in common, leading to results that do not correspond to real biological patterns. Here we present Scanorama, inspired by algorithms for panorama stitching, that overcomes the limitations of existing methods to enable accurate, heterogeneous scRNA-seq data set integration. Our strategy identifies and merges the shared cell types among all pairs of data sets and is orders of magnitude faster than existing techniques. We use Scanorama to combine 105,476 cells from 26 diverse scRNA-seq experiments across 9 different technologies into a single comprehensive reference, demonstrating how Scanorama can be used to obtain a more complete picture of cellular function across a wide range of scRNA-seq experiments.

Multiple Data Set ILI for Mechanical Damage Assessment

2012 ◽  
Author(s):  
Chris Goller ◽  
James Simek ◽  
Jed Ludlow
Keyword(s):  
Damage Assessment ◽  
Public Awareness ◽  
Mechanical Damage ◽  
Data Sets ◽  
Data Set ◽  
Multiple Data ◽  
Pipe Joints ◽  
Multiple Data Sets ◽  
Multiple Field ◽  
Flux Leakage

The purpose of this paper is to present a non-traditional pipeline mechanical damage ranking system using multiple-data-set in-line inspection (ILI) tools. Mechanical damage continues to be a major factor in reportable incidents for hazardous liquid and gas pipelines. While several ongoing programs seek to limit damage incidents through public awareness, encroachment monitoring, and one-call systems, others have focused efforts on the quantification of mechanical damage severity through modeling, the use of ILI tools, and subsequent feature assessment at locations selected for excavation. Current generation ILI tools capable of acquiring multiple-data-sets in a single survey may provide an improved assessment of the severity of damaged zones using methods developed in earlier research programs as well as currently reported information. For magnetic flux leakage (MFL) type tools, using multiple field levels, varied field directions, and high accuracy deformation sensors enables detection and provides the data necessary for enhanced severity assessments. This paper will provide a review of multiple-data-set ILI results from several pipe joints with simulated mechanical damage locations created mimicing right-of-way encroachment events in addition to field results from ILI surveys using multiple-data-set tools.

Improving Quality and Safety Through Use of Secondary Data: Methods Case Study

2016 ◽  
Vol 39 (11) ◽  
pp. 1477-1501 ◽  
Author(s):  
Victoria Goode ◽  
Nancy Crego ◽  
Michael P. Cary ◽  
Deirdre Thornlow ◽  
Elizabeth Merwin
Keyword(s):  
Research Question ◽  
Secondary Data ◽  
Data Access ◽  
Data Sets ◽  
Complex Data ◽  
Management Skills ◽  
Data Set ◽  
Large Numbers ◽  
Need To Evaluate

Researchers need to evaluate the strengths and weaknesses of data sets to choose a secondary data set to use for a health care study. This research method review informs the reader of the major issues necessary for investigators to consider while incorporating secondary data into their repertoire of potential research designs and shows the range of approaches the investigators may take to answer nursing research questions in a variety of context areas. The researcher requires expertise in locating and judging data sets and in the development of complex data management skills for managing large numbers of records. There are important considerations such as firm knowledge of the research question supported by the conceptual framework and the selection of appropriate databases, which guide the researcher in delineating the unit of analysis. Other more complex issues for researchers to consider when conducting secondary data research methods include data access, management and security, and complex variable construction.

scholarly journals Data Fusion Using a Multi-Sensor Sparse-Based Clustering Algorithm

2020 ◽  
Vol 12 (23) ◽  
pp. 4007
Author(s):  
Kasra Rafiezadeh Shahi ◽  
Pedram Ghamisi ◽  
Behnood Rasti ◽  
Robert Jackisch ◽  
Paul Scheunders ◽  
...  
Keyword(s):  
Clustering Algorithm ◽  
Spatial Information ◽  
Clustering Algorithms ◽  
Hyperspectral Data ◽  
Sensor Data ◽  
Data Sets ◽  
Data Types ◽  
Data Set ◽  
Multiple Data Sets ◽  
Imaging Sensors

The increasing amount of information acquired by imaging sensors in Earth Sciences results in the availability of a multitude of complementary data (e.g., spectral, spatial, elevation) for monitoring of the Earth’s surface. Many studies were devoted to investigating the usage of multi-sensor data sets in the performance of supervised learning-based approaches at various tasks (i.e., classification and regression) while unsupervised learning-based approaches have received less attention. In this paper, we propose a new approach to fuse multiple data sets from imaging sensors using a multi-sensor sparse-based clustering algorithm (Multi-SSC). A technique for the extraction of spatial features (i.e., morphological profiles (MPs) and invariant attribute profiles (IAPs)) is applied to high spatial-resolution data to derive the spatial and contextual information. This information is then fused with spectrally rich data such as multi- or hyperspectral data. In order to fuse multi-sensor data sets a hierarchical sparse subspace clustering approach is employed. More specifically, a lasso-based binary algorithm is used to fuse the spectral and spatial information prior to automatic clustering. The proposed framework ensures that the generated clustering map is smooth and preserves the spatial structures of the scene. In order to evaluate the generalization capability of the proposed approach, we investigate its performance not only on diverse scenes but also on different sensors and data types. The first two data sets are geological data sets, which consist of hyperspectral and RGB data. The third data set is the well-known benchmark Trento data set, including hyperspectral and LiDAR data. Experimental results indicate that this novel multi-sensor clustering algorithm can provide an accurate clustering map compared to the state-of-the-art sparse subspace-based clustering algorithms.

scholarly journals Estimating observation and model error variances using multiple data sets

2018 ◽  
Vol 11 (7) ◽  
pp. 4239-4260 ◽  
Author(s):  
Richard Anthes ◽  
Therese Rieckh
Keyword(s):  
Error Variance ◽  
Specific Humidity ◽  
Data Sets ◽  
Data Set ◽  
Multiple Data ◽  
Gfs Model ◽  
Multiple Data Sets ◽  
Using Data ◽  
The Tropics ◽  
Estimated Error

Abstract. In this paper we show how multiple data sets, including observations and models, can be combined using the “three-cornered hat” (3CH) method to estimate vertical profiles of the errors of each system. Using data from 2007, we estimate the error variances of radio occultation (RO), radiosondes, ERA-Interim, and Global Forecast System (GFS) model data sets at four radiosonde locations in the tropics and subtropics. A key assumption is the neglect of error covariances among the different data sets, and we examine the consequences of this assumption on the resulting error estimates. Our results show that different combinations of the four data sets yield similar relative and specific humidity, temperature, and refractivity error variance profiles at the four stations, and these estimates are consistent with previous estimates where available. These results thus indicate that the correlations of the errors among all data sets are small and the 3CH method yields realistic error variance profiles. The estimated error variances of the ERA-Interim data set are smallest, a reasonable result considering the excellent model and data assimilation system and assimilation of high-quality observations. For the four locations studied, RO has smaller error variances than radiosondes, in agreement with previous studies. Part of the larger error variance of the radiosondes is associated with representativeness differences because radiosondes are point measurements, while the other data sets represent horizontal averages over scales of ∼ 100 km.

Moisture estimation within a mine heap: An application of cokriging with assay data and electrical resistivity

Geophysics ◽  
2010 ◽  
Vol 75 (1) ◽  
pp. B11-B23 ◽  
Author(s):  
Dale Rucker
Keyword(s):  
Electrical Resistivity ◽  
Secondary Data ◽  
Estimation Procedure ◽  
Primary Data ◽  
Low Grade ◽  
Least Squares Regression ◽  
Data Set ◽  
Correlation Scale ◽  
The Mean ◽  
Moisture Estimation

Cokriging has been applied to estimate the distribution of moisture within a rock pile of low-grade gold ore, or heap. Along with the primary data set of gravimetric moisture content obtained from drilling, electrical resistivity was used to supplement the estimation procedure by supplying a secondary data set. The effectiveness of the cokriging method was determined by comparing the results to kriging the moisture data alone and through least-squares regression (LSR) modeling of colocated resistivity and moisture. In general, the wells from which moisture data were derived were separated by distances far greater than the horizontal correlation scale. The kriging results showed that regions generally undersampled by drilling reverted to the mean of the moisture data. The LSR technique, which provides a simpletransformation of resistivity to moisture, converted the low resis-tivity to highmoisture, and vice versa. The sparse well locations created a high degree of uncertainty in the transformed data set. Extreme resistivity values produced nonphysical moisture values, either negative for the linear model or values greater than one for the power model. The cokriging application, which considers the correlation scale and secondary data, produced the best results, as indicated through the cross validation. The mean and variance of the cokriged moisture were closer to the measured moisture, and the bias in the residuals was the lowest. The application likely could be improved through optimal well placement, whereby the resistivity results guide the drilling program through gross target characterization, and the moisture estimation could be updated iteratively.

Sign in / Sign up

Export Citation Format

Share Document