Combined Beta Metric for Unsupervised Clustering of Microbiome Data

2020 ◽  
Author(s):  
Yushu Shi ◽  
Liangliang Zhang ◽  
Christine Peterson ◽  
Kim-Anh Do ◽  
Robert Jenq
Keyword(s):  
Beta Diversity ◽  
High Performance ◽  
Unsupervised Clustering ◽  
Clustering Methods ◽  
Naturally Occurring ◽  
High Prevalence ◽  
Unweighted Unifrac ◽  
Single Method ◽  
Microbiome Data ◽  
Microbiome Data Analysis

Abstract Background In Microbiome data analysis, unsupervised clustering is often used to identify naturally occurring clusters, which can then be assessed for associations with characteristics of interest. In this work, we systematically compared beta diversity and clustering methods commonly used in microbiome analyses. We applied these to four published datasets where highly distinct microbiome profiles could be seen between sample groups. Results Although no single method outperformed the others consistently, we did identify key scenarios where certain methods can underperform. Specifically, the Bray Curtis metric resulted in poor clustering in a dataset where high-abundance OTUs were relatively rare. In contrast, the unweighted UniFrac metric clustered poorly when used on a dataset with a high prevalence of low-abundance OTUs. To test our proposition, we systematically modified properties of the poorly performing datasets and found that this approach resulted in improved Bray Curtis and unweighted UniFrac performance. Conclusions Based on these observations, we rationally combined the Bray Curtis metric and the unweighted UniFrac metrics and found that this new beta diversity metric showed high performance across all datasets.

scholarly journals Performance Determinants of Unsupervised Clustering Methods for Microbiome Data

2021 ◽  
Author(s):  
Yushu Shi ◽  
Liangliang Zhang ◽  
Christine Peterson ◽  
Kim-Anh Do ◽  
Robert Jenq
Keyword(s):  
Beta Diversity ◽  
High Performance ◽  
Unsupervised Clustering ◽  
Systematic Evaluation ◽  
Clustering Methods ◽  
High Prevalence ◽  
Unweighted Unifrac ◽  
Single Method ◽  
Performance Determinants ◽  
Microbiome Data

Abstract Background: In microbiome data analysis, unsupervised clustering is often used to identify naturally occurring clusters, which can then be assessed for associations with characteristics of interest. In this work, we systematically compared beta diversity and clustering methods commonly used in microbiome analyses. We applied these to four published datasets where highly distinct microbiome profiles could be seen between sample groups. Results: Although no single method outperformed the others consistently, we did identify key scenarios where certain methods can underperform. Specifically, the Bray Curtis metric resulted in poor clustering in a dataset where high-abundance OTUs were relatively rare. In contrast, the unweighted UniFrac metric clustered poorly when used on a dataset with a high prevalence of low-abundance OTUs. To test our proposition, we systematically modified properties of the poorly performing datasets and found that this approach resulted in improved Bray Curtis and unweighted UniFrac performance. Based on these observations, we rationally combined the Bray Curtis metric and the unweighted UniFrac metrics and found that this new beta diversity metric showed high performance across all datasets. We also evaluated our findings by examining a clinical dataset where clusters are less separated. Conclusions: Our systematic evaluation of clustering performance in these five datasets demonstrates that there is no existing clustering method that universally performs best across all datasets. We propose a combined metric of Bray Curtis and unweighted UniFrac that capitalizes on the complementary strengths of the two metrics.

scholarly journals Performance Determinants of Unsupervised Clustering Methods for Microbiome Data

2021 ◽  
Author(s):  
Yushu Shi ◽  
Liangliang Zhang ◽  
Christine Peterson ◽  
Kim-Anh Do ◽  
Robert Jenq
Keyword(s):  
Beta Diversity ◽  
High Performance ◽  
Unsupervised Clustering ◽  
Systematic Evaluation ◽  
Clustering Methods ◽  
High Prevalence ◽  
Unweighted Unifrac ◽  
Single Method ◽  
Performance Determinants ◽  
Microbiome Data

In microbiome data analysis, unsupervised clustering is often used to identify naturally occurring clusters, which can then be assessed for associations with characteristics of interest. In this work, we systematically compared beta diversity and clustering methods commonly used in microbiome analyses. We applied these to four published datasets where highly distinct microbiome profiles could be seen between sample groups. Although no single method outperformed the others consistently, we did identify key scenarios where certain methods can underperform. Specifically, the Bray Curtis metric resulted in poor clustering in a dataset where high-abundance OTUs were relatively rare. In contrast, the unweighted UniFrac metric clustered poorly when used on a dataset with a high prevalence of low-abundance OTUs. To test our proposition, we systematically modified properties of the poorly performing datasets and found that this approach resulted in improved Bray Curtis and unweighted UniFrac performance. Based on these observations, we rationally combined the Bray Curtis metric and the unweighted UniFrac metrics and found that this new beta diversity metric showed high performance across all datasets. We also evaluated our findings by examining a clinical dataset where clusters are less separated. Our systematic evaluation of clustering performance in these five datasets demonstrates that there is no existing clustering method that universally performs best across all datasets. We propose a combined metric of Bray Curtis and unweighted UniFrac that capitalizes on the complementary strengths of the two metrics.

scholarly journals Performance Determinants of Unsupervised Clustering Methods for Microbiome Data

2021 ◽  
Author(s):  
Yushu Shi ◽  
Liangliang Zhang ◽  
Christine Peterson ◽  
Kim-Anh Do ◽  
Robert Jenq
Keyword(s):  
High Performance ◽  
Unsupervised Clustering ◽  
Systematic Evaluation ◽  
Clustering Methods ◽  
Unifrac Distance ◽  
High Prevalence ◽  
Unweighted Unifrac ◽  
Single Method ◽  
Performance Determinants ◽  
Microbiome Data

Abstract Background: In microbiome data analysis, unsupervised clustering is often used to identify naturally occurring clusters, which can then be assessed for associations with characteristics of interest. In this work, we systematically compared beta diversity and clustering methods commonly used in microbiome analyses. We applied these to four published datasets where highly distinct microbiome profiles could be seen between sample groups, as well a clinical dataset with less clear separation between groups. Results: Although no single method outperformed the others consistently, we did identify key scenarios where certain methods can underperform. Specifically, the Bray Curtis (BC) metric resulted in poor clustering in a dataset where high-abundance OTUs were relatively rare. In contrast, the unweighted UniFrac (UU) metric clustered poorly on dataset with a high prevalence of low-abundance OTUs. To explore these hypotheses about BC and UU, we systematically modified properties of the poorly performing datasets and found that this approach resulted in improved BC and UU performance. Based on these observations, we rationally combined BC and UU to generate a novel metric. We tested its performance while varying the relative contributions of each metric and also compared it with another combined metric, the generalized UniFrac distance. The proposed metric showed high performance across all datasets. Conclusions Our systematic evaluation of clustering performance in these five datasets demonstrates that there is no existing clustering method that universally performs best across all datasets. We propose a combined metric of BC and UU that capitalizes on the complementary strengths of the two metrics.

scholarly journals Effects of microbiome rare taxa filtering on statistical analysis

2020 ◽  
Author(s):  
Quy Xuan Cao ◽  
Xinxin Sun ◽  
Karun Rajesh ◽  
Naga Chalasani ◽  
Kayla Gelow ◽  
...  
Keyword(s):  
Quality Control ◽  
Data Analysis ◽  
Beta Diversity ◽  
Marker Gene ◽  
Microbial Composition ◽  
Linear Discriminant ◽  
Technical Variability ◽  
Disease Study ◽  
Microbiome Data ◽  
Microbiome Data Analysis

Abstract Background: Accuracy of microbial community detection in 16S rRNA marker-gene and metagenomic studies suffers from contamination and sequencing errors that lead to either falsely identifying microbial taxa that were not in the sample or misclassifying the taxa of DNA fragment reads. Filtering is defined as removing taxa that are present in a small number of samples and have small counts in the samples where they are observed. This approach reduces extreme sparsity of microbiome data and has been shown to correctly remove contaminant taxa in cultured "mock" datasets, where the true taxa compositions are known. Although filtering is frequently used, careful evaluation of its effect on the data analysis and scientific conclusions remains unreported. Here, we assess the effect of filtering on the alpha and beta diversity estimation, as well as its impact on identifying taxa that discriminate between disease states. Results: The effect of filtering on microbiome data analysis is illustrated on four datasets: two mock quality control datasets where same cultured samples with known microbial composition are processed at different labs and two disease study datasets. Results show that in microbiome quality control datasets, filtering reduces the magnitude of differences in alpha diversity and alleviates technical variability between labs, while preserving between samples similarity (beta diversity). In the disease study datasets, DESeq2 and linear discriminant analysis Effect Size (LEfSe) methods were used to identify taxa that are differentially expressed across groups of samples, and random forest models to rank features with largest contribution towards disease classiffcation. Results reveal that filtering retains significant taxa and preserves the model classification ability measured by the area under the receiver operating characteristic curve (AUC). The comparison between filtering and contaminant removal method shows that they have complementary effects and are advised to be used in conjunction. Conclusions: Filtering reduces the complexity of microbiome data, while preserving their integrity in downstream analysis. This leads to mitigation of the classification methods' sensitivity and reduction of technical variability, allowing researchers to generate more reproducible and comparable results in microbiome data analysis.

scholarly journals Effects of Rare Microbiome Taxa Filtering on Statistical Analysis

2021 ◽  
Vol 11 ◽  
Author(s):  
Quy Cao ◽  
Xinxin Sun ◽  
Karun Rajesh ◽  
Naga Chalasani ◽  
Kayla Gelow ◽  
...  
Keyword(s):  
Quality Control ◽  
Data Analysis ◽  
Beta Diversity ◽  
Marker Gene ◽  
Microbial Composition ◽  
Linear Discriminant ◽  
Technical Variability ◽  
Disease Study ◽  
Microbiome Data ◽  
Microbiome Data Analysis

Background: The accuracy of microbial community detection in 16S rRNA marker-gene and metagenomic studies suffers from contamination and sequencing errors that lead to either falsely identifying microbial taxa that were not in the sample or misclassifying the taxa of DNA fragment reads. Removing contaminants and filtering rare features are two common approaches to deal with this problem. While contaminant detection methods use auxiliary sequencing process information to identify known contaminants, filtering methods remove taxa that are present in a small number of samples and have small counts in the samples where they are observed. The latter approach reduces the extreme sparsity of microbiome data and has been shown to correctly remove contaminant taxa in cultured “mock” datasets, where the true taxa compositions are known. Although filtering is frequently used, careful evaluation of its effect on the data analysis and scientific conclusions remains unreported. Here, we assess the effect of filtering on the alpha and beta diversity estimation as well as its impact on identifying taxa that discriminate between disease states.Results: The effect of filtering on microbiome data analysis is illustrated on four datasets: two mock quality control datasets where the same cultured samples with known microbial composition are processed at different labs and two disease study datasets. Results show that in microbiome quality control datasets, filtering reduces the magnitude of differences in alpha diversity and alleviates technical variability between labs while preserving the between samples similarity (beta diversity). In the disease study datasets, DESeq2 and linear discriminant analysis Effect Size (LEfSe) methods were used to identify taxa that are differentially abundant across groups of samples, and random forest models were used to rank features with the largest contribution toward disease classification. Results reveal that filtering retains significant taxa and preserves the model classification ability measured by the area under the receiver operating characteristic curve (AUC). The comparison between the filtering and the contaminant removal method shows that they have complementary effects and are advised to be used in conjunction.Conclusions: Filtering reduces the complexity of microbiome data while preserving their integrity in downstream analysis. This leads to mitigation of the classification methods' sensitivity and reduction of technical variability, allowing researchers to generate more reproducible and comparable results in microbiome data analysis.

Detection and characterisation of hepatitis E virus in naturally infected swine in Croatia

2013 ◽  
Vol 61 (4) ◽  
pp. 517-528 ◽  
Author(s):  
Zoran Lipej ◽  
Dinko Novosel ◽  
Lea Vojta ◽  
Besi Roić ◽  
Miljenko Šimpraga ◽  
...  
Keyword(s):  
Hepatitis E Virus ◽  
Age Groups ◽  
Hepatitis E ◽  
Health Concern ◽  
Rt Pcr ◽  
Animal Reservoir ◽  
Naturally Occurring ◽  
High Prevalence ◽  
Swine Herds ◽  
Bile Samples

Hepatitis E is a viral zoonotic disease infecting swine worldwide. Since pigs represent a likely animal reservoir for the hepatitis E virus, the epidemiology of naturally occurring hepatitis E was investigated in Croatian swine herds. Nearly all tested animals were seropositive for antibodies against the hepatitis E virus (55/60, 91.7%). Active infection was detected in all age groups by RT-PCR of viral RNA in serum (8/60, 13.3%) and bile samples (3/37, 8.1%), which was further confirmed by histopathological findings of characteristic lesions in the livers of the infected animals. Three new strains of hepatitis E virus were isolated from Croatian pig herds. Phylogenetic analysis using median-joining networks clustered those Croatian strains with isolates from various parts of the world, indicating their likely origin in international trade. Similarity to human isolates implies a zoonotic potential of Croatian strains, which raises a public health concern, especially in the light of the high prevalence of hepatitis E in the herds studied.

scholarly journals Bioconductor workflow for microbiome data analysis: from raw reads to community analyses

F1000Research ◽  
2016 ◽  
Vol 5 ◽  
pp. 1492 ◽  
Author(s):  
Ben J. Callahan ◽  
Kris Sankaran ◽  
Julia A. Fukuyama ◽  
Paul J. McMurdie ◽  
Susan P. Holmes
Keyword(s):  
High Throughput Sequencing ◽  
Linear Models ◽  
Reference Database ◽  
Rrna Gene ◽  
Microbial Composition ◽  
Nonparametric Testing ◽  
Abundance Estimates ◽  
Taxonomic Markers ◽  
Microbiome Data ◽  
Microbiome Data Analysis

High-throughput sequencing of PCR-amplified taxonomic markers (like the 16S rRNA gene) has enabled a new level of analysis of complex bacterial communities known as microbiomes. Many tools exist to quantify and compare abundance levels or microbial composition of communities in different conditions. The sequencing reads have to be denoised and assigned to the closest taxa from a reference database. Common approaches use a notion of 97% similarity and normalize the data by subsampling to equalize library sizes. In this paper, we show that statistical models allow more accurate abundance estimates. By providing a complete workflow in R, we enable the user to do sophisticated downstream statistical analyses, including both parameteric and nonparametric methods. We provide examples of using the R packages dada2, phyloseq, DESeq2, ggplot2 and vegan to filter, visualize and test microbiome data. We also provide examples of supervised analyses using random forests, partial least squares and linear models as well as nonparametric testing using community networks and the ggnetwork package.

scholarly journals Battery Sizing for Different Loads and RES Production Scenarios through Unsupervised Clustering Methods

Forecasting ◽  
2021 ◽  
Vol 3 (4) ◽  
pp. 663-681
Author(s):  
Alfredo Nespoli ◽  
Andrea Matteri ◽  
Silvia Pretto ◽  
Luca De De Ciechi ◽  
Emanuele Ogliari
Keyword(s):  
Power Generation ◽  
Renewable Energy Sources ◽  
Storage System ◽  
Energy Storage System ◽  
Hybrid Plant ◽  
Unsupervised Clustering ◽  
Clustering Methods ◽  
Pv System ◽  
Feasible Solutions ◽  
Battery Energy

The increasing penetration of Renewable Energy Sources (RESs) in the energy mix is determining an energy scenario characterized by decentralized power production. Between RESs power generation technologies, solar PhotoVoltaic (PV) systems constitute a very promising option, but their production is not programmable due to the intermittent nature of solar energy. The coupling between a PV facility and a Battery Energy Storage System (BESS) allows to achieve a greater flexibility in power generation. However, the design phase of a PV+BESS hybrid plant is challenging due to the large number of possible configurations. The present paper proposes a preliminary procedure aimed at predicting a family of batteries which is suitable to be coupled with a given PV plant configuration. The proposed procedure is applied to new hypothetical plants built to fulfill the energy requirements of a commercial and an industrial load. The energy produced by the PV system is estimated on the basis of a performance analysis carried out on similar real plants. The battery operations are established through two decision-tree-like structures regulating charge and discharge respectively. Finally, an unsupervised clustering is applied to all the possible PV+BESS configurations in order to identify the family of feasible solutions.

Sign in / Sign up

Export Citation Format

Share Document