Using single-cell cytometry to illustrate integrated multi-perspective evaluation of clustering algorithms using Pareto fronts

2021 ◽  
Author(s):  
Givanna H Putri ◽  
Irena Koprinska ◽  
Thomas M Ashhurst ◽  
Nicholas J C King ◽  
Mark N Read
Keyword(s):  
Single Cell ◽  
Performance Metrics ◽  
Clustering Algorithms ◽  
Latin Hypercube Sampling ◽  
Supplementary Information ◽  
Sequencing Data ◽  
Evaluation Protocol ◽  
Benchmark Datasets ◽  
Pareto Fronts ◽  
Parameter Values

Abstract Motivation Many ‘automated gating’ algorithms now exist to cluster cytometry and single-cell sequencing data into discrete populations. Comparative algorithm evaluations on benchmark datasets rely either on a single performance metric, or a few metrics considered independently of one another. However, single metrics emphasize different aspects of clustering performance and do not rank clustering solutions in the same order. This underlies the lack of consensus between comparative studies regarding optimal clustering algorithms and undermines the translatability of results onto other non-benchmark datasets. Results We propose the Pareto fronts framework as an integrative evaluation protocol, wherein individual metrics are instead leveraged as complementary perspectives. Judged superior are algorithms that provide the best trade-off between the multiple metrics considered simultaneously. This yields a more comprehensive and complete view of clustering performance. Moreover, by broadly and systematically sampling algorithm parameter values using the Latin Hypercube sampling method, our evaluation protocol minimizes (un)fortunate parameter value selections as confounding factors. Furthermore, it reveals how meticulously each algorithm must be tuned in order to obtain good results, vital knowledge for users with novel data. We exemplify the protocol by conducting a comparative study between three clustering algorithms (ChronoClust, FlowSOM and Phenograph) using four common performance metrics applied across four cytometry benchmark datasets. To our knowledge, this is the first time Pareto fronts have been used to evaluate the performance of clustering algorithms in any application domain. Availability and implementation Implementation of our Pareto front methodology and all scripts and datasets to reproduce this article are available at https://github.com/ghar1821/ParetoBench. Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals Selecting single cell clustering parameter values using subsampling-based robustness metrics

2021 ◽  
Vol 22 (1) ◽  
Author(s):  
Ryan B. Patterson-Cross ◽  
Ariel J. Levine ◽  
Vilas Menon
Keyword(s):  
Single Cell ◽  
Optimal Parameter ◽  
Clustering Algorithms ◽  
Cell Types ◽  
Parameter Selection ◽  
Data Set ◽  
Biologically Relevant ◽  
Cell Clustering ◽  
Parameter Values ◽  
Robustness Metrics

Abstract Background Generating and analysing single-cell data has become a widespread approach to examine tissue heterogeneity, and numerous algorithms exist for clustering these datasets to identify putative cell types with shared transcriptomic signatures. However, many of these clustering workflows rely on user-tuned parameter values, tailored to each dataset, to identify a set of biologically relevant clusters. Whereas users often develop their own intuition as to the optimal range of parameters for clustering on each data set, the lack of systematic approaches to identify this range can be daunting to new users of any given workflow. In addition, an optimal parameter set does not guarantee that all clusters are equally well-resolved, given the heterogeneity in transcriptomic signatures in most biological systems. Results Here, we illustrate a subsampling-based approach (chooseR) that simultaneously guides parameter selection and characterizes cluster robustness. Through bootstrapped iterative clustering across a range of parameters, chooseR was used to select parameter values for two distinct clustering workflows (Seurat and scVI). In each case, chooseR identified parameters that produced biologically relevant clusters from both well-characterized (human PBMC) and complex (mouse spinal cord) datasets. Moreover, it provided a simple “robustness score” for each of these clusters, facilitating the assessment of cluster quality. Conclusion chooseR is a simple, conceptually understandable tool that can be used flexibly across clustering algorithms, workflows, and datasets to guide clustering parameter selection and characterize cluster robustness.

scholarly journals breakpointR: an R/Bioconductor package to localize strand state changes in Strand-seq data

2019 ◽  
Author(s):  
David Porubsky ◽  
Ashley D Sanders ◽  
Aaron Taudt ◽  
Maria Colomé-Tatché ◽  
Peter M Lansdorp ◽  
...  
Keyword(s):  
Single Cell ◽  
Sister Chromatid Exchanges ◽  
Supplementary Information ◽  
Specific Information ◽  
Bioconductor Package ◽  
Sequencing Data ◽  
Sequencing Technique ◽  
Important Addition ◽  
State Changes ◽  
Chromatid Exchanges

Abstract Motivation Strand-seq is a specialized single-cell DNA sequencing technique centered around the directionality of single-stranded DNA. Computational tools for Strand-seq analyses must capture the strand-specific information embedded in these data. Results Here we introduce breakpointR, an R/Bioconductor package specifically tailored to process and interpret single-cell strand-specific sequencing data obtained from Strand-seq. We developed breakpointR to detect local changes in strand directionality of aligned Strand-seq data, to enable fine-mapping of sister chromatid exchanges, germline inversion and to support global haplotype assembly. Given the broad spectrum of Strand-seq applications we expect breakpointR to be an important addition to currently available tools and extend the accessibility of this novel sequencing technique. Availability and implementation R/Bioconductor package https://bioconductor.org/packages/breakpointR. Supplementary information Supplementary data are available at Bioinformatics online.

Single-cell RNA-seq data clustering: A survey with performance comparison study

2020 ◽  
Vol 18 (04) ◽  
pp. 2040005
Author(s):  
Ruiyi Li ◽  
Jihong Guan ◽  
Shuigeng Zhou
Keyword(s):  
Single Cell ◽  
Data Clustering ◽  
Performance Metrics ◽  
Clustering Algorithms ◽  
Cell Types ◽  
Performance Comparison ◽  
Cellular Heterogeneity ◽  
Clustering Methods ◽  
Multiple Perspectives ◽  
Underlying Mechanisms

Clustering analysis has been widely applied to single-cell RNA-sequencing (scRNA-seq) data to discover cell types and cell states. Algorithms developed in recent years have greatly helped the understanding of cellular heterogeneity and the underlying mechanisms of biological processes. However, these algorithms often use different techniques, were evaluated on different datasets and compared with some of their counterparts usually using different performance metrics. Consequently, there lacks an accurate and complete picture of their merits and demerits, which makes it difficult for users to select proper algorithms for analyzing their data. To fill this gap, we first do a review on the major existing scRNA-seq data clustering methods, and then conduct a comprehensive performance comparison among them from multiple perspectives. We consider 13 state of the art scRNA-seq data clustering algorithms, and collect 12 publicly available real scRNA-seq datasets from the existing works to evaluate and compare these algorithms. Our comparative study shows that the existing methods are very diverse in performance. Even the top-performance algorithms do not perform well on all datasets, especially those with complex structures. This suggests that further research is required to explore more stable, accurate, and efficient clustering algorithms for scRNA-seq data.

schex avoids overplotting for large single-cell RNA-sequencing datasets

2019 ◽  
Vol 36 (7) ◽  
pp. 2291-2292 ◽  
Author(s):  
Saskia Freytag ◽  
Ryan Lister
Keyword(s):  
Single Cell ◽  
Rna Sequencing ◽  
R Package ◽  
Supplementary Information ◽  
Supplementary Data ◽  
Sequencing Data ◽  
Single Cell Rna Sequencing

Abstract Summary Due to the scale and sparsity of single-cell RNA-sequencing data, traditional plots can obscure vital information. Our R package schex overcomes this by implementing hexagonal binning, which has the additional advantages of improving speed and reducing storage for resulting plots. Availability and implementation schex is freely available from Bioconductor via http://bioconductor.org/packages/release/bioc/html/schex.html and its development version can be accessed on GitHub via https://github.com/SaskiaFreytag/schex. Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals SPsimSeq: semi-parametric simulation of bulk and single cell RNA sequencing data

2019 ◽  
Author(s):  
Alemu Takele Assefa ◽  
Jo Vandesompele ◽  
Olivier Thas
Keyword(s):  
Gene Expression ◽  
Single Cell ◽  
Rna Sequencing ◽  
Empirical Distribution ◽  
Supplementary Information ◽  
Rna Seq ◽  
Sequencing Data ◽  
Actual Distribution ◽  
Wide Range ◽  
Single Cell Rna Sequencing

SummarySPsimSeq is a semi-parametric simulation method for bulk and single cell RNA sequencing data. It simulates data from a good estimate of the actual distribution of a given real RNA-seq dataset. In contrast to existing approaches that assume a particular data distribution, our method constructs an empirical distribution of gene expression data from a given source RNA-seq experiment to faithfully capture the data characteristics of real data. Importantly, our method can be used to simulate a wide range of scenarios, such as single or multiple biological groups, systematic variations (e.g. confounding batch effects), and different sample sizes. It can also be used to simulate different gene expression units resulting from different library preparation protocols, such as read counts or UMI counts.Availability and implementationThe R package and associated documentation is available from https://github.com/CenterForStatistics-UGent/SPsimSeq.Supplementary informationSupplementary data are available at bioRχiv online.

scholarly journals A Multi-center Cross-platform Single-cell RNA Sequencing Reference Dataset

2020 ◽  
Author(s):  
Xin Chen ◽  
Zhaowei Yang ◽  
Wanqiu Chen ◽  
Yongmei Zhao ◽  
Andrew Farmer ◽  
...  
Keyword(s):  
Single Cell ◽  
Rna Sequencing ◽  
Whole Genome Sequencing Data ◽  
Differential Analysis ◽  
Sequencing Data ◽  
Reference Dataset ◽  
Batch Correction ◽  
Distinct Cell ◽  
Single Cell Rna Sequencing ◽  
Benchmark Datasets

AbstractSingle-cell RNA sequencing (scRNA-seq) is developing rapidly, and investigators seeking to use this technology are left with a variety of options for both experimental platform and bioinformatics methods. There is an urgent need for scRNA-seq reference datasets for benchmarking of different scRNA-seq platforms and bioinformatics methods. To be broadly applicable, these should be generated from renewable, well characterized reference samples and processed in multiple centers across different platforms. Here we present a benchmarking scRNA-seq dataset that includes 20 scRNA-seq datasets acquired either as a mixtures or as individual samples from two biologically distinct cell lines for which a large amount of multi-platform whole genome sequencing data are also available. These scRNA-seq datasets were generated from multiple popular platforms across four sequencing centers. Our benchmark datasets provide a resource that we believe will have great value for the single-cell community by serving as a reference dataset for evaluating various bioinformatics methods for scRNA-seq analyses, including but not limited to data preprocessing, imputation, normalization, clustering, batch correction, and differential analysis.

EnImpute: imputing dropout events in single-cell RNA-sequencing data via ensemble learning

2019 ◽  
Vol 35 (22) ◽  
pp. 4827-4829 ◽  
Author(s):  
Xiao-Fei Zhang ◽  
Le Ou-Yang ◽  
Shuo Yang ◽  
Xing-Ming Zhao ◽  
Xiaohua Hu ◽  
...  
Keyword(s):  
Single Cell ◽  
Rna Sequencing ◽  
Ensemble Learning ◽  
R Package ◽  
Supplementary Information ◽  
Sequencing Data ◽  
Single Cell Rna Sequencing ◽  
The Individual ◽  
Downstream Analysis ◽  
Shiny Application

Abstract Summary Imputation of dropout events that may mislead downstream analyses is a key step in analyzing single-cell RNA-sequencing (scRNA-seq) data. We develop EnImpute, an R package that introduces an ensemble learning method for imputing dropout events in scRNA-seq data. EnImpute combines the results obtained from multiple imputation methods to generate a more accurate result. A Shiny application is developed to provide easier implementation and visualization. Experiment results show that EnImpute outperforms the individual state-of-the-art methods in almost all situations. EnImpute is useful for correcting the noisy scRNA-seq data before performing downstream analysis. Availability and implementation The R package and Shiny application are available through Github at https://github.com/Zhangxf-ccnu/EnImpute. Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals Scirpy: a Scanpy extension for analyzing single-cell T-cell receptor-sequencing data

2020 ◽  
Vol 36 (18) ◽  
pp. 4817-4818 ◽  
Author(s):  
Gregor Sturm ◽  
Tamas Szabo ◽  
Georgios Fotakis ◽  
Marlene Haider ◽  
Dietmar Rieder ◽  
...  
Keyword(s):  
T Cell ◽  
Single Cell ◽  
Large Scale ◽  
Single Cells ◽  
Cell Receptor ◽  
Supplementary Information ◽  
Sequencing Data ◽  
Seamless Integration ◽  
T Cell Phenotypes ◽  
Cell Phenotypes

Abstract Summary Advances in single-cell technologies have enabled the investigation of T-cell phenotypes and repertoires at unprecedented resolution and scale. Bioinformatic methods for the efficient analysis of these large-scale datasets are instrumental for advancing our understanding of adaptive immune responses. However, while well-established solutions are accessible for the processing of single-cell transcriptomes, no streamlined pipelines are available for the comprehensive characterization of T-cell receptors. Here, we propose single-cell immune repertoires in Python (Scirpy), a scalable Python toolkit that provides simplified access to the analysis and visualization of immune repertoires from single cells and seamless integration with transcriptomic data. Availability and implementation Scirpy source code and documentation are available at https://github.com/icbi-lab/scirpy. Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals gpart: human genome partitioning and visualization of high-density SNP data by identifying haplotype blocks

2019 ◽  
Vol 35 (21) ◽  
pp. 4419-4421 ◽  
Author(s):  
Sun Ah Kim ◽  
Myriam Brossard ◽  
Delnaz Roshandel ◽  
Andrew D Paterson ◽  
Shelley B Bull ◽  
...  
Keyword(s):  
Clustering Algorithms ◽  
R Package ◽  
Supplementary Information ◽  
Visualization Tool ◽  
Sequencing Data ◽  
Haplotype Blocks ◽  
Snp Data ◽  
Computing Environments ◽  
Next Generation Sequencing Ngs ◽  
Generation Sequencing

Abstract Summary For the analysis of high-throughput genomic data produced by next-generation sequencing (NGS) technologies, researchers need to identify linkage disequilibrium (LD) structure in the genome. In this work, we developed an R package gpart which provides clustering algorithms to define LD blocks or analysis units consisting of SNPs. The visualization tool in gpart can display the LD structure and gene positions for up to 20 000 SNPs in one image. The gpart functions facilitate construction of LD blocks and SNP partitions for vast amounts of genome sequencing data within reasonable time and memory limits in personal computing environments. Availability and implementation The R package is available at https://bioconductor.org/packages/gpart. Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals SCSsim: an integrated tool for simulating single-cell genome sequencing data

2019 ◽  
Author(s):  
Zhenhua Yu ◽  
Fang Du ◽  
Xuehong Sun ◽  
Ao Li
Keyword(s):  
Single Cell ◽  
High Efficiency ◽  
Detection Efficiency ◽  
Comprehensive Evaluation ◽  
Supplementary Information ◽  
Sequencing Data ◽  
Single Cell Sequencing ◽  
Technical Issues ◽  
Cell Genome ◽  
User Intervention

Abstract Motivation Allele dropout (ADO) and unbalanced amplification of alleles are main technical issues of single-cell sequencing (SCS), and effectively emulating these issues is necessary for reliably benchmarking SCS-based bioinformatics tools. Unfortunately, currently available sequencing simulators are free of whole-genome amplification involved in SCS technique and therefore not suited for generating SCS datasets. We develop a new software package (SCSsim) that can efficiently simulate SCS datasets in a parallel fashion with minimal user intervention. SCSsim first constructs the genome sequence of single cell by mimicking a complement of genomic variations under user-controlled manner, and then amplifies the genome according to MALBAC technique and finally yields sequencing reads from the amplified products based on inferred sequencing profiles. Comprehensive evaluation in simulating different ADO rates, variation detection efficiency and genome coverage demonstrates that SCSsim is a very useful tool in mimicking single-cell sequencing data with high efficiency. Availability and implementation SCSsim is freely available at https://github.com/qasimyu/scssim. Supplementary information Supplementary data are available at Bioinformatics online.

Sign in / Sign up

Export Citation Format

Share Document