scholarly journals Log-ratio analysis of microbiome data with many zeroes is library size dependent

2020 ◽  
Author(s):  
Dennis te Beest ◽  
Els Nijhuis ◽  
Tim Mohlmann ◽  
Caro Ter braak
Keyword(s):  
Compositional Data ◽  
Explanatory Variable ◽  
Principal Component ◽  
Real Data ◽  
Amplicon Sequencing ◽  
Sequencing Data ◽  
Library Size ◽  
Sequencing Platform ◽  
Microbiome Composition ◽  
Log Ratio

Microbiome composition data collected through amplicon sequencing are count data on taxa in which the total count per sample (the library size) is an artifact of the sequencing platform and as a result such data are compositional. To avoid library size dependency, one common way of analyzing multivariate compositional data is to perform a principal component analysis (PCA) on data transformed with the centered log-ratio, hereafter called a log-ratio PCA. Two aspects typical of amplicon sequencing data are the large differences in library size and the large number of zeroes. In this paper we show on real data and by simulation that, applied to data that combines these two aspects, log-ratio PCA is nevertheless heavily dependent on the library size. This leads to a reduction in power when testing against any explanatory variable in log-ratio redundancy analysis. If there is additionally a correlation between the library size and the explanatory variable, then the type 1 error becomes inflated. We explore putative solutions to this problem.

scholarly journals A field guide for the compositional analysis of any-omics data

2018 ◽  
Author(s):  
Thomas P. Quinn ◽  
Ionas Erb ◽  
Greg Gloor ◽  
Cedric Notredame ◽  
Mark F. Richardson ◽  
...  
Keyword(s):  
Data Analysis ◽  
General Solution ◽  
Compositional Data ◽  
Compositional Analysis ◽  
Compositional Data Analysis ◽  
Sequencing Data ◽  
Nucleotide Synthesis ◽  
Library Size ◽  
Concise Guide ◽  
Log Ratio

AbstractNext-generation sequencing (NGS) has made it possible to determine the sequence and relative abundance of all nucleotides in a biological or environmental sample. Today, NGS is routinely used to understand many important topics in biology from human disease to microorganism diversity. A cornerstone of NGS is the quantification of RNA or DNA presence as counts. However, these counts are not counts per se: the magnitude of the counts are determined arbitrarily by the sequencing depth, not by the input material. Consequently, counts must undergo normalization prior to use. Conventional normalization methods require a set of assumptions: they assume that the majority of features are unchanged, and that all environments under study have the same carrying capacity for nucleotide synthesis. These assumptions are often untestable and may not hold when comparing heterogeneous samples (e.g., samples collected across distinct cancers or tissues). Instead, methods developed within the field of compositional data analysis offer a general solution that is assumption-free and valid for all data. In this manuscript, we synthesize the extant literature to provide a concise guide on how to apply compositional data analysis to NGS count data. In doing so, we review zero replacement, differential abundance analysis, and within-group and between-group coordination analysis. We then discuss how this pipeline can accommodate complex study design, facilitate the analysis of vertically and horizontally integrated data, including multiomics data, and further extend to single-cell sequencing data. In highlighting the limitations of total library size, effective library size, and spike-in normalizations, we propose the log-ratio transformation as a general solution to answer the question, “Relative to some important activity of the cell, what is changing?”. Taken together, this manuscript establishes the first fully comprehensive analysis protocol that is suitable for any and all -omics data.

scholarly journals Differential expression analysis of log-ratio transformed counts: benchmarking methods for RNA-Seq data

2017 ◽  
Author(s):  
Thomas P. Quinn ◽  
Tamsyn M. Crowley ◽  
Mark F. Richardson
Keyword(s):  
16S Rrna ◽  
Differential Expression ◽  
High Precision ◽  
Differential Expression Analysis ◽  
Real Data ◽  
False Positives ◽  
Rna Seq ◽  
Sequencing Data ◽  
Library Size ◽  
Log Ratio

AbstractBackgroundCount data generated by next-generation sequencing assays do not measure absolute transcript abundances. Instead, the data are constrained to an arbitrary “library size” by the sequencing depth of the assay, and typically must be normalized prior to statistical analysis. The constrained nature of these data means one could alternatively use a log-ratio transformation in lieu of normalization, as often done when testing for differential abundance (DA) of operational taxonomic units (OTUs) in 16S rRNA data. Therefore, we benchmark how well the ALDEx2 package, a transformation-based DA tool, detects differential expression in high-throughput RNA-sequencing data (RNA-Seq), compared to conventional RNA-Seq differential expression methods.ResultsTo evaluate the performance of log-ratio transformation-based tools, we apply the ALDEx2 package to two simulated, and one real, RNA-Seq data sets. The latter was previously used to benchmark dozens of conventional RNA-Seq differential expression methods, enabling us to directly compare transformation-based approaches. We show that ALDEx2, widely used in meta-genomics research, identifies differentially expressed genes (and transcripts) from RNA-Seq data with high precision and, given sufficient sample sizes, high recall too (regardless of the alignment and quantification procedure used). Although we show that the choice in log-ratio transformation can affect performance, ALDEx2 has high precision (i.e., few false positives) across all transformations. Finally, we present a novel, iterative log-ratio transformation (now implemented in ALDEx2) that further improves performance in simulations.ConclusionsOur results suggest that log-ratio transformation-based methods can work to measure differential expression from RNA-Seq data, provided that certain assumptions are met. Moreover, these methods have high precision (i.e., few false positives) in simulations and perform as good as, or better than, than conventional methods on real data. With previously demonstrated applicability to 16S rRNA data, ALDEx2 can work as a single tool for data from multiple sequencing modalities.

scholarly journals High-dimensional Log-Error-in-Variable Regression with Applications to Microbial Compositional Data Analysis

Biometrika ◽  
2021 ◽  
Author(s):  
Pixu Shi ◽  
Yuchen Zhou ◽  
Anru R Zhang
Keyword(s):  
Data Analysis ◽  
Compositional Data ◽  
Estimation Error ◽  
Real Data ◽  
Upper And Lower Bounds ◽  
High Dimensional ◽  
Compositional Data Analysis ◽  
Sequencing Data ◽  
Contrast Model ◽  
Critical Issues

Abstract In microbiome and genomic studies, the regression of compositional data has been a crucial tool for identifying microbial taxa or genes that are associated with clinical phenotypes. To account for the variation in sequencing depth, the classic log-contrast model is often used where read counts are normalized into compositions. However, zero read counts and the randomness in covariates remain critical issues. In this article, we introduce a surprisingly simple, interpretable, and efficient method for the estimation of compositional data regression through the lens of a novel high-dimensional log-error-in-variable regression model. The proposed method provides both corrections on sequencing data with possible overdispersion and simultaneously avoids any subjective imputation of zero read counts. We provide theoretical justifications with matching upper and lower bounds for the estimation error. The merit of the procedure is illustrated through real data analysis and simulation studies.

scholarly journals Ensuring that fundamentals of quantitative microbiology are reflected in microbial diversity analyses based on next-generation sequencing

2021 ◽  
Author(s):  
Philip J Schmidt ◽  
Ellen S Cameron ◽  
Kirsten M Müller ◽  
Monica B Emelko
Keyword(s):  
Single Point ◽  
Amplicon Sequencing ◽  
Shannon Index ◽  
Most Probable Number ◽  
Diversity Analysis ◽  
Sequencing Data ◽  
Sample Collection ◽  
Library Size ◽  
Probable Number ◽  
Microbiological Methods

Diversity analysis of amplicon sequencing data is mainly limited to plug-in estimates calculated using normalized data to obtain a single value of an alpha diversity metric or a single point on a beta diversity ordination plot for each sample. As recognized for count data generated using classical microbiological methods, read counts obtained from a sample are random data linked to source properties by a probabilistic process. Thus, diversity analysis has focused on diversity of (normalized) samples rather than probabilistic inference about source diversity. This study applies fundamentals of statistical analysis for quantitative microbiology (e.g., microscopy, plating, most probable number methods) to sample collection and processing procedures of amplicon sequencing methods to facilitate inference reflecting the probabilistic nature of such data and evaluation of uncertainty in diversity metrics. Types of random error are described and clustering of microorganisms in the source, differential analytical recovery during sample processing, and amplification are found to invalidate a multinomial relative abundance model. The zeros often abounding in amplicon sequencing data and their implications are addressed, and Bayesian analysis is applied to estimate the source Shannon index given unnormalized data (both simulated and real). Inference about source diversity is found to require knowledge of the exact number of unique variants in the source, which is practically unknowable due to library size limitations and the inability to differentiate zeros corresponding to variants that are actually absent in the source from zeros corresponding to variants that were merely not detected. Given these problems with estimation of diversity in the source even when the basic multinomial model is valid, sample-level diversity analysis approaches are discussed.

scholarly journals To rarefy or not to rarefy: Enhancing microbial community analysis through next-generation sequencing

2020 ◽  
Author(s):  
Ellen S. Cameron ◽  
Philip J. Schmidt ◽  
Benjamin J.-M. Tremblay ◽  
Monica B. Emelko ◽  
Kirsten M. Müller
Keyword(s):  
Microbial Community ◽  
Community Analysis ◽  
Amplicon Sequencing ◽  
Microbial Community Analysis ◽  
List Type ◽  
Sequencing Data ◽  
Water And Wastewater Treatment ◽  
Library Size ◽  
Size Selection ◽  
The Impact

AbstractThe application of amplicon sequencing in water research provides a rapid and sensitive technique for microbial community analysis in a variety of environments ranging from freshwater lakes to water and wastewater treatment plants. It has revolutionized our ability to study DNA collected from environmental samples by eliminating the challenges associated with lab cultivation and taxonomic identification. DNA sequencing data consist of discrete counts of sequence reads, the total number of which is the library size. Samples may have different library sizes and thus, a normalization technique is required to meaningfully compare them. The process of randomly subsampling sequences to a selected normalized library size from the sample library—rarefying—is one such normalization technique. However, rarefying has been criticized as a normalization technique because data can be omitted through the exclusion of either excess sequences or entire samples, depending on the rarefied library size selected. Although it has been suggested that rarefying should be avoided altogether, we propose that repeatedly rarefying enables (i) characterization of the variation introduced to diversity analyses by this random subsampling and (ii) selection of smaller library sizes where necessary to incorporate all samples in the analysis. Rarefying may be a statistically valid normalization technique, but researchers should evaluate their data to make appropriate decisions regarding library size selection and subsampling type. The impact of normalized library size selection and rarefying with or without replacement in diversity analyses were evaluated herein.Highlights▪ Amplicon sequencing technology for environmental water samples is reviewed▪ Sequencing data must be normalized to allow comparison in diversity analyses▪ Rarefying normalizes library sizes by subsampling from observed sequences▪ Criticisms of data loss through rarefying can be resolved by rarefying repeatedly▪ Rarefying repeatedly characterizes errors introduced by subsampling sequences

scholarly journals A field guide for the compositional analysis of any-omics data

GigaScience ◽  
2019 ◽  
Vol 8 (9) ◽  
Author(s):  
Thomas P Quinn ◽  
Ionas Erb ◽  
Greg Gloor ◽  
Cedric Notredame ◽  
Mark F Richardson ◽  
...  
Keyword(s):  
Data Analysis ◽  
General Solution ◽  
Compositional Data ◽  
Compositional Analysis ◽  
Compositional Data Analysis ◽  
Nucleotide Synthesis ◽  
Library Size ◽  
Next Generation Sequencing Ngs ◽  
Concise Guide ◽  
Log Ratio

Abstract Background Next-generation sequencing (NGS) has made it possible to determine the sequence and relative abundance of all nucleotides in a biological or environmental sample. A cornerstone of NGS is the quantification of RNA or DNA presence as counts. However, these counts are not counts per se: their magnitude is determined arbitrarily by the sequencing depth, not by the input material. Consequently, counts must undergo normalization prior to use. Conventional normalization methods require a set of assumptions: they assume that the majority of features are unchanged and that all environments under study have the same carrying capacity for nucleotide synthesis. These assumptions are often untestable and may not hold when heterogeneous samples are compared. Results Methods developed within the field of compositional data analysis offer a general solution that is assumption-free and valid for all data. Herein, we synthesize the extant literature to provide a concise guide on how to apply compositional data analysis to NGS count data. Conclusions In highlighting the limitations of total library size, effective library size, and spike-in normalizations, we propose the log-ratio transformation as a general solution to answer the question, “Relative to some important activity of the cell, what is changing?”

scholarly journals On establishing ceramic chemical groups: exploring the influence of data analysis methods and the role of the elements chosen in analysis

2013 ◽  
Vol 1 (1) ◽  
pp. 1 ◽  
Author(s):  
Kostalena Michelaki ◽  
Michael J. Hughes ◽  
Ronald G.V. Hancock
Keyword(s):  
Principal Component Analysis ◽  
Data Analysis ◽  
Compositional Data ◽  
Principal Component ◽  
Component Analysis ◽  
Data Exploration ◽  
Short Paper ◽  
Bivariate Data ◽  
Chemical Groups ◽  
Log Ratio

Since the 1970s, archaeologists have increasingly depended on archaeometric rather than strictly stylistic data to explore questions of ceramic provenance and technol- ogy, and, by extension, trade, exchange, social networks and even identity. It is accepted as obvious by some archaeometrists and statisti- cians that the results of the analyses of compo- sitional data may be dependent on the format of the data used, on the data exploration method employed and, in the case of multivari- ate analyses, even on the number of elements considered. However, this is rarely articulated clearly in publications, making it less obvious to archaeologists. In this short paper, we re- examine compositional data from a collection of bricks, tiles and ceramics from Hill Hall, near Epping in Essex, England, as a case study to show how the method of data exploration used and the number of elements considered in multivariate analyses of compositional data can affect the sorting of ceramic samples into chemical groups. We compare bivariate data splitting (BDS) with principal component analysis (PCA) and centered log ratio-principal component analysis (CLR-PCA) of different unstandardized data formats [original concen- tration data and logarithmically transformed (i.e. log10 data)], using different numbers of elements. We confirm that PCA, in its various forms, is quite sensitive to the numbers and types of elements used in data analysis.

Workflow for analysis of compositional data in sedimentary petrology: provenance changes in sedimentary basins from spatio-temporal variation in heavy-mineral assemblages

2018 ◽  
Vol 156 (07) ◽  
pp. 1111-1130 ◽  
Author(s):  
J. VERHAEGEN ◽  
G.J. WELTJE ◽  
D. MUNSTERMAN
Keyword(s):  
Compositional Data ◽  
Principal Component ◽  
Sedimentary Basins ◽  
Heavy Mineral ◽  
Provenance Analysis ◽  
Sedimentary Petrology ◽  
Mineral Data ◽  
Spatio Temporal ◽  
Data Tables ◽  
Log Ratio

AbstractThe field of provenance analysis has seen a revival in the last decade as quantitative data-acquisition techniques continue to develop. In the 20th century, many heavy-mineral data were collected. These data were mostly used as qualitative indications for stratigraphy and provenance, and not incorporated in a quantitative provenance methodology. Even today, such data are mostly only used in classic data tables or cumulative heavy-mineral plots as a qualitative indication of variation. The main obstacle to rigorous statistical analysis is the compositional nature of these data which makes them unfit for standard multivariate statistics. To gain more information from legacy data, a straightforward workflow for quantitative analysis of compositional datasets is provided. First (1) a centred log-ratio transformation of the data is carried out to fix the constant-sum constraint and non-negativity of the compositional data. Next, (2) cluster analysis is followed by (3) principal component analysis and (4) bivariate log-ratio plots. Several (5) proxies for the effects of sorting and weathering are included to check the provenance significance of observed variations and finally a (6) spatial interpolation of a provenance proxy extracted from the dataset can be carried out. To test this methodology, available heavy-mineral data from the southern edge of the Miocene North Sea Basin are analysed. The results are compared with available information from literature and are used to gain improved insight into Miocene sediment input variations in the study area.

scholarly journals Amalgams: data-driven amalgamation for the reference-free dimensionality reduction of zero-laden compositional data

2020 ◽  
Author(s):  
Thomas P. Quinn ◽  
Ionas Erb
Keyword(s):  
Dimension Reduction ◽  
Domain Knowledge ◽  
Compositional Data ◽  
Real Data ◽  
R Package ◽  
Data Driven ◽  
Alternative Methods ◽  
Compositional Data Analysis ◽  
Data Sets ◽  
Log Ratio

AbstractIn the health sciences, many data sets produced by next-generation sequencing (NGS) only contain relative information because of biological and technical factors that limit the total number of nucleotides observed for a given sample. As mutually dependent elements, it is not possible to interpret any component in isolation, at least without normalization. The field of compositional data analysis (CoDA) has emerged with alternative methods for relative data based on log-ratio transforms. However, NGS data often contain many more features than samples, and thus require creative new ways to reduce the dimensionality of the data without sacrificing interpretability. The summation of parts, called amalgamation, is a practical way of reducing dimensionality, but can introduce a non-linear distortion to the data. We exploit this non-linearity to propose a powerful yet interpretable dimension reduction method. In this report, we present data-driven amalgamation as a new method and conceptual framework for reducing the dimensionality of compositional data. Unlike expert-driven amalgamation which requires prior domain knowledge, our data-driven amalgamation method uses a genetic algorithm to answer the question, “What is the best way to amalgamate the data to achieve the user-defined objective?”. We present a user-friendly R package, called amalgam, that can quickly find the optimal amalgamation to (a) preserve the distance between samples, or (b) classify samples as diseased or not. Our benchmark on 13 real data sets confirm that these amalgamations compete with the state-of-the-art unsupervised and supervised dimension reduction methods in terms of performance, but result in new variables that are much easier to understand: they are groups of features added together.

scholarly journals Understanding sequencing data as compositions: an outlook and review

2017 ◽  
Author(s):  
Thomas P. Quinn ◽  
Ionas Erb ◽  
Mark F. Richardson ◽  
Tamsyn M. Crowley
Keyword(s):  
Statistical Models ◽  
Compositional Data ◽  
Correlation Coefficients ◽  
Distance Measures ◽  
Compositional Data Analysis ◽  
Sequencing Data ◽  
Multivariate Statistical ◽  
Library Size ◽  
Future Directions ◽  
Sequencing Platforms

AbstractMotivationAlthough seldom acknowledged explicitly, count data generated by sequencing platforms exist as compositions for which the abundance of each component (e.g., gene or transcript) is only coherently interpretable relative to other components within that sample. This property arises from the assay technology itself, whereby the number of counts recorded for each sample is constrained by an arbitrary total sum (i.e., library size). Consequently, sequencing data, as compositional data, exist in a non-Euclidean space that renders invalid many conventional analyses, including distance measures, correlation coefficients, and multivariate statistical models.ResultsThe purpose of this review is to summarize the principles of compositional data analysis (CoDA), provide evidence for why sequencing data are compositional, discuss compositionally valid methods available for analyzing sequencing data, and highlight future directions with regard to this field of study.

Sign in / Sign up

Export Citation Format

Share Document