Clustering methods for single-cell RNA-sequencing expression data: performance evaluation with varying sample sizes and cell compositions

2019 ◽  
Vol 18 (5) ◽  
Author(s):  
Aslı Suner
Keyword(s):  
Performance Evaluation ◽  
Sample Size ◽  
Single Cell ◽  
Rna Sequencing ◽  
Performance Measures ◽  
Systematic Evaluation ◽  
Expression Data ◽  
Clustering Methods ◽  
Sample Sizes ◽  
Single Cell Rna Sequencing

Abstract A number of specialized clustering methods have been developed so far for the accurate analysis of single-cell RNA-sequencing (scRNA-seq) expression data, and several reports have been published documenting the performance measures of these clustering methods under different conditions. However, to date, there are no available studies regarding the systematic evaluation of the performance measures of the clustering methods taking into consideration the sample size and cell composition of a given scRNA-seq dataset. Herein, a comprehensive performance evaluation study of 11 selected scRNA-seq clustering methods was performed using synthetic datasets with known sample sizes and number of subpopulations, as well as varying levels of transcriptome complexity. The results indicate that the overall performance of the clustering methods under study are highly dependent on the sample size and complexity of the scRNA-seq dataset. In most of the cases, better clustering performances were obtained as the number of cells in a given expression dataset was increased. The findings of this study also highlight the importance of sample size for the successful detection of rare cell subpopulations with an appropriate clustering tool.

scholarly journals A systematic evaluation of single cell RNA-seq analysis pipelines

2019 ◽  
Vol 10 (1) ◽  
Author(s):  
Beate Vieth ◽  
Swati Parekh ◽  
Christoph Ziegenhain ◽  
Wolfgang Enard ◽  
Ines Hellmann
Keyword(s):  
Best Practices ◽  
Sample Size ◽  
Single Cell ◽  
Differential Expression ◽  
Rna Sequencing ◽  
Systematic Evaluation ◽  
Library Preparation ◽  
Rna Seq ◽  
Rapid Spread ◽  
Single Cell Rna Sequencing

Abstract The recent rapid spread of single cell RNA sequencing (scRNA-seq) methods has created a large variety of experimental and computational pipelines for which best practices have not yet been established. Here, we use simulations based on five scRNA-seq library protocols in combination with nine realistic differential expression (DE) setups to systematically evaluate three mapping, four imputation, seven normalisation and four differential expression testing approaches resulting in ~3000 pipelines, allowing us to also assess interactions among pipeline steps. We find that choices of normalisation and library preparation protocols have the biggest impact on scRNA-seq analyses. Specifically, we find that library preparation determines the ability to detect symmetric expression differences, while normalisation dominates pipeline performance in asymmetric DE-setups. Finally, we illustrate the importance of informed choices by showing that a good scRNA-seq pipeline can have the same impact on detecting a biological signal as quadrupling the sample size.

scholarly journals A Systematic Evaluation of Single Cell RNA-Seq Analysis Pipelines

2019 ◽  
Author(s):  
Beate Vieth ◽  
Swati Parekh ◽  
Christoph Ziegenhain ◽  
Wolfgang Enard ◽  
Ines Hellmann
Keyword(s):  
Best Practices ◽  
Sample Size ◽  
Single Cell ◽  
Differential Expression ◽  
Rna Sequencing ◽  
Systematic Evaluation ◽  
Library Preparation ◽  
Rna Seq ◽  
Rapid Spread ◽  
Single Cell Rna Sequencing

AbstractThe recent rapid spread of single cell RNA sequencing (scRNA-seq) methods has created a large variety of experimental and computational pipelines for which best practices have not been established, yet. Here, we use simulations based on five scRNA-seq library protocols in combination with nine realistic differential expression (DE) setups to systematically evaluate three mapping, four imputation, seven normalisation and four differential expression testing approaches resulting in ∼ 3,000 pipelines, allowing us to also assess interactions among pipeline steps. We find that choices of normalisation and library preparation protocols have the biggest impact on scRNA-seq analyses. Specifically, we find that library preparation determines the ability to detect symmetric expression differences, while normalisation dominates pipeline performance in asymmetric DE-setups. Finally, we illustrate the importance of informed choices by showing that a good scRNA-seq pipeline can have the same impact on detecting a biological signal as quadrupling the sample size.

scholarly journals Subpopulation identification for single-cell RNA-sequencing data using functional data analysis

2019 ◽  
Author(s):  
Kyungmin Ahn ◽  
Hironobu Fujiwara
Keyword(s):  
Gene Expression ◽  
Data Analysis ◽  
Single Cell ◽  
Gene Expression Data ◽  
Functional Data Analysis ◽  
Functional Data ◽  
Clustering Algorithms ◽  
Expression Data ◽  
Clustering Methods ◽  
Single Cell Rna Sequencing

AbstractBackgroundIn single-cell RNA-sequencing (scRNA-seq) data analysis, a number of statistical tools in multivariate data analysis (MDA) have been developed to help analyze the gene expression data. This MDA approach is typically focused on examining discrete genomic units of genes that ignores the dependency between the data components. In this paper, we propose a functional data analysis (FDA) approach on scRNA-seq data whereby we consider each cell as a single function. To avoid a large number of dropouts (zero or zero-closed values) and reduce the high dimensionality of the data, we first perform a principal component analysis (PCA) and assign PCs to be the amplitude of the function. Then we use the index of PCs directly from PCA for the phase components. This approach allows us to apply FDA clustering methods to scRNA-seq data analysis.ResultsTo demonstrate the robustness of our method, we apply several existing FDA clustering algorithms to the gene expression data to improve the accuracy of the classification of the cell types against the conventional clustering methods in MDA. As a result, the FDA clustering algorithms achieve superior accuracy on simulated data as well as real data such as human and mouse scRNA-seq data.ConclusionsThis new statistical technique enhances the classification performance and ultimately improves the understanding of stochastic biological processes. This new framework provides an essentially different scRNA-seq data analytical approach, which can complement conventional MDA methods. It can be truly effective when current MDA methods cannot detect or uncover the hidden functional nature of the gene expression dynamics.

scholarly journals Built on sand: the shaky foundations of simulating single-cell RNA sequencing data

2021 ◽  
Author(s):  
Helena L Crowell ◽  
Sarah X Morillo Leonardo ◽  
Charlotte Soneson ◽  
Mark D Robinson
Keyword(s):  
Quality Control ◽  
Single Cell ◽  
Rna Sequencing ◽  
Simulated Data ◽  
Quality Standard ◽  
Data Generation ◽  
Clustering Methods ◽  
Sequencing Data ◽  
Study Results ◽  
Single Cell Rna Sequencing

With the emergence of hundreds of single-cell RNA-sequencing (scRNA-seq) datasets, the number of computational tools to analyse aspects of the generated data has grown rapidly. As a result, there is a recurring need to demonstrate whether newly developed methods are truly performant - on their own as well as in comparison to existing tools. Benchmark studies aim to consolidate the space of available methods for a given task, and often use simulated data that provide a ground truth for evaluations. Thus, demanding a high quality standard for synthetically generated data is critical to make simulation study results credible and transferable to real data. Here, we evaluated methods for synthetic scRNA-seq data generation in their ability to mimic experimental data. Besides comparing gene- and cell-level quality control summaries in both one- and two-dimensional settings, we further quantified these at the batch- and cluster-level. Secondly, we investigate the effect of simulators on clustering and batch correction method comparisons, and, thirdly, which and to what extent quality control summaries can capture reference-simulation similarity. Our results suggest that most simulators are unable to accommodate complex designs without introducing artificial effects; they yield over-optimistic performance of integration, and potentially unreliable ranking of clustering methods; and, it is generally unknown which summaries are important to ensure effective simulation-based method comparisons.

scholarly journals A Systematic Evaluation of Single-cell RNA-sequencing Imputation Methods

2020 ◽  
Author(s):  
Wenpin Hou ◽  
Zhicheng Ji ◽  
Hongkai Ji ◽  
Stephanie C. Hicks
Keyword(s):  
Single Cell ◽  
Rna Sequencing ◽  
Rapid Development ◽  
Systematic Evaluation ◽  
Rna Seq ◽  
Experimental Protocol ◽  
Improve Performance ◽  
Imputation Methods ◽  
Single Cell Rna Sequencing ◽  
Experimental Protocols

ABSTRACTThe rapid development of single-cell RNA-sequencing (scRNA-seq) technology, with increased sparsity compared to bulk RNA-sequencing (RNA-seq), has led to the emergence of many methods for preprocessing, including imputation methods. Here, we systematically evaluate the performance of 18 state-of-the-art scRNA-seq imputation methods using cell line and tissue data measured across experimental protocols. Specifically, we assess the similarity of imputed cell profiles to bulk samples as well as investigate whether methods recover relevant biological signals or introduce spurious noise in three downstream analyses: differential expression, unsupervised clustering, and inferring pseudotemporal trajectories. Broadly, we found significant variability in the performance of the methods across evaluation settings. While most scRNA-seq imputation methods recover biological expression observed in bulk RNA-seq data, the majority of the methods do not improve performance in downstream analyses compared to no imputation, in particular for clustering and trajectory analysis, and thus should be used with caution. Furthermore, we find that the performance of scRNA-seq imputation methods depends on many factors including the experimental protocol, the sparsity of the data, the number of cells in the dataset, and the magnitude of the effect sizes. We summarize our results and provide a key set of recommendations for users and investigators to navigate the current space of scRNA-seq imputation methods.

scholarly journals Simulation, power evaluation and sample size recommendation for single-cell RNA-seq

2020 ◽  
Vol 36 (19) ◽  
pp. 4860-4868
Author(s):  
Kenong Su ◽  
Zhijin Wu ◽  
Hao Wu
Keyword(s):  
Sample Size ◽  
Single Cell ◽  
Rna Sequencing ◽  
Statistical Significance ◽  
Sample Size Calculation ◽  
Supplementary Information ◽  
Design Stage ◽  
Rna Seq ◽  
Single Cell Rna Sequencing ◽  
Power Evaluation

Abstract Motivation Determining the sample size for adequate power to detect statistical significance is a crucial step at the design stage for high-throughput experiments. Even though a number of methods and tools are available for sample size calculation for microarray and RNA-seq in the context of differential expression (DE), this topic in the field of single-cell RNA sequencing is understudied. Moreover, the unique data characteristics present in scRNA-seq such as sparsity and heterogeneity increase the challenge. Results We propose POWSC, a simulation-based method, to provide power evaluation and sample size recommendation for single-cell RNA-sequencing DE analysis. POWSC consists of a data simulator that creates realistic expression data, and a power assessor that provides a comprehensive evaluation and visualization of the power and sample size relationship. The data simulator in POWSC outperforms two other state-of-art simulators in capturing key characteristics of real datasets. The power assessor in POWSC provides a variety of power evaluations including stratified and marginal power analyses for DEs characterized by two forms (phase transition or magnitude tuning), under different comparison scenarios. In addition, POWSC offers information for optimizing the tradeoffs between sample size and sequencing depth with the same total reads. Availability and implementation POWSC is an open-source R package available online at https://github.com/suke18/POWSC. Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals A systematic evaluation of single-cell RNA-sequencing imputation methods

2020 ◽  
Vol 21 (1) ◽  
Author(s):  
Wenpin Hou ◽  
Zhicheng Ji ◽  
Hongkai Ji ◽  
Stephanie C. Hicks
Keyword(s):  
Single Cell ◽  
Rna Sequencing ◽  
Systematic Evaluation ◽  
Imputation Methods ◽  
Single Cell Rna Sequencing

scholarly journals Differential analysis of binarized single-cell RNA sequencing data captures biological variation

2021 ◽  
Vol 3 (4) ◽  
Author(s):  
Gerard A Bouland ◽  
Ahmed Mahfouz ◽  
Marcel J T Reinders
Keyword(s):  
Single Cell ◽  
Rna Sequencing ◽  
Expression Profiles ◽  
Biological Variation ◽  
Expression Data ◽  
Differential Analysis ◽  
Sequencing Data ◽  
Single Cell Rna Sequencing ◽  
Cell Expression ◽  
Zero Counts

Abstract Single-cell RNA sequencing data is characterized by a large number of zero counts, yet there is growing evidence that these zeros reflect biological variation rather than technical artifacts. We propose to use binarized expression profiles to identify the effects of biological variation in single-cell RNA sequencing data. Using 16 publicly available and simulated datasets, we show that a binarized representation of single-cell expression data accurately represents biological variation and reveals the relative abundance of transcripts more robustly than counts.

scholarly journals Benchmarking UMI-based single cell RNA-sequencing preprocessing workflows

2021 ◽  
Author(s):  
Yue You ◽  
Luyi Tian ◽  
Shian Su ◽  
Xueyi Dong ◽  
Jafar Sheikh Jabbari ◽  
...  
Keyword(s):  
Single Cell ◽  
Rna Sequencing ◽  
Rapid Development ◽  
Ground Truth ◽  
Clustering Methods ◽  
Analysis Process ◽  
Single Cell Rna Sequencing ◽  
Systematic Biases ◽  
Downstream Analysis ◽  
Almost All

Single-cell RNA sequencing (scRNA-seq) technologies and associated analysis methods have undergone rapid development in recent years. This includes methods for data preprocessing, which assign sequencing reads to genes to create count matrices for downstream analysis. Several packaged preprocessing workflows have been developed that aim to provide users with convenient tools for handling this process. How different preprocessing workflows compare to one another and influence downstream analysis has been less well studied. Here, we systematically benchmark the performance of 9 end-to-end preprocessing workflows (Cell Ranger, Optimus, salmon alevin, kallisto bustools, dropSeqPipe, scPipe, zUMIs, celseq2 and scruff) using datasets with varying levels of biological complexity generated on the CEL-Seq2 and 10x Chromium platforms. We compare these workflows in terms of their quantification properties directly and their impact on normalization and clustering by evaluating the performance of different method combinations. We find that lowly expressed genes are discordant between workflows and observe that some workflows have systematic biases towards particular classes of genomics features. While the scRNA-seq preprocessing workflows compared varied in their detection and quantification of genes across datasets, after downstream analysis with performant normalization and clustering methods, almost all combinations produced clustering results that agreed well with the known cell type labels that provided the ground truth in our analysis. In summary, the choice of preprocessing method was found to be less influential than other steps in the scRNA-seq analysis process. Our study comprehensively compares common scRNAseq preprocessing workflows and summarizes their characteristics to guide workflow users.

Sign in / Sign up

Export Citation Format

Share Document