scholarly journals reComBat: Batch effect removal in large-scale, multi-source omics data integration

2021 ◽  
Author(s):  
Michael F. Adamer ◽  
Sarah C. Brueningk ◽  
Alejandro Tejada-Arranz ◽  
Fabienne Estermann ◽  
Marek Basler ◽  
...  
Keyword(s):  
Gene Expression ◽  
Data Integration ◽  
Empirical Bayes ◽  
Large Scale ◽  
Batch Effect ◽  
Data Driven ◽  
Design Matrix ◽  
Omics Data ◽  
Experimental Conditions ◽  
Batch Correction

With the steadily increasing abundance of omics data produced all over the world, sometimes decades apart and under vastly different experimental conditions residing in public databases, a crucial step in many data-driven bioinformatics applications is that of data integration. The challenge of batch effect removal for entire databases lies in the large number and coincide of both batches and desired, biological variation resulting in design matrix singularity. This problem currently cannot be solved by any common batch correction algorithm. In this study, we present reComBat, a regularised version of the empirical Bayes method to overcome this limitation. We demonstrate our approach for the harmonisation of public gene expression data of the human opportunistic pathogen Pseudomonas aeruginosa and study a several metrics to empirically demonstrate that batch effects are successfully mitigated while biologically meaningful gene expression variation is retained. reComBat fills the gap in batch correction approaches applicable to large scale, public omics databases and opens up new avenues for data driven analysis of complex biological processes beyond the scope of a single study.

scholarly journals Challenges in the Integration of Omics and Non-Omics Data

Genes ◽  
2019 ◽  
Vol 10 (3) ◽  
pp. 238 ◽  
Author(s):  
Evangelina López de Maturana ◽  
Lola Alonso ◽  
Pablo Alarcón ◽  
Isabel Adoración Martín-Antoniano ◽  
Silvia Pineda ◽  
...  
Keyword(s):  
Data Integration ◽  
Large Scale ◽  
Predictive Ability ◽  
Epidemiological Data ◽  
Joint Modeling ◽  
Epidemiological Studies ◽  
Omics Data ◽  
Data Set ◽  
Health Domains ◽  
Analytical Strategies

Omics data integration is already a reality. However, few omics-based algorithms show enough predictive ability to be implemented into clinics or public health domains. Clinical/epidemiological data tend to explain most of the variation of health-related traits, and its joint modeling with omics data is crucial to increase the algorithm’s predictive ability. Only a small number of published studies performed a “real” integration of omics and non-omics (OnO) data, mainly to predict cancer outcomes. Challenges in OnO data integration regard the nature and heterogeneity of non-omics data, the possibility of integrating large-scale non-omics data with high-throughput omics data, the relationship between OnO data (i.e., ascertainment bias), the presence of interactions, the fairness of the models, and the presence of subphenotypes. These challenges demand the development and application of new analysis strategies to integrate OnO data. In this contribution we discuss different attempts of OnO data integration in clinical and epidemiological studies. Most of the reviewed papers considered only one type of omics data set, mainly RNA expression data. All selected papers incorporated non-omics data in a low-dimensionality fashion. The integrative strategies used in the identified papers adopted three modeling methods: Independent, conditional, and joint modeling. This review presents, discusses, and proposes integrative analytical strategies towards OnO data integration.

scholarly journals A Joint Deep Learning Model for Simultaneous Batch Effect Correction, Denoising and Clustering in Single-Cell Transcriptomics

2020 ◽  
Author(s):  
Justin Lakkis ◽  
David Wang ◽  
Yuanchao Zhang ◽  
Gang Hu ◽  
Kui Wang ◽  
...  
Keyword(s):  
Gene Expression ◽  
Deep Learning ◽  
Single Cell ◽  
Large Scale ◽  
Nearest Neighbor ◽  
Learning Model ◽  
Batch Effect ◽  
Marker Genes ◽  
Deep Learning Model ◽  
Variable Genes

AbstractRecent development of single-cell RNA-seq (scRNA-seq) technologies has led to enormous biological discoveries. As the scale of scRNA-seq studies increases, a major challenge in analysis is batch effect, which is inevitable in studies involving human tissues. Most existing methods remove batch effect in a low-dimensional embedding space. Although useful for clustering, batch effect is still present in the gene expression space, leaving downstream gene-level analysis susceptible to batch effect. Recent studies have shown that batch effect correction in the gene expression space is much harder than in the embedding space. Popular methods such as Seurat3.0 rely on the mutual nearest neighbor (MNN) approach to remove batch effect in the gene expression space, but MNN can only analyze two batches at a time and it becomes computationally infeasible when the number of batches is large. Here we present CarDEC, a joint deep learning model that simultaneously clusters and denoises scRNA-seq data, while correcting batch effect both in the embedding and the gene expression space. Comprehensive evaluations spanning different species and tissues showed that CarDEC consistently outperforms scVI, DCA, and MNN. With CarDEC denoising, those non-highly variable genes offer as much signal for clustering as the highly variable genes, suggesting that CarDEC substantially boosted information content in scRNA-seq. We also showed that trajectory analysis using CarDEC’s denoised and batch corrected expression as input revealed marker genes and transcription factors that are otherwise obscured in the presence of batch effect. CarDEC is computationally fast, making it a desirable tool for large-scale scRNA-seq studies.

scholarly journals GEDI: an R package for integration of transcriptomic data from multiple high-throughput platforms

2021 ◽  
Author(s):  
Mathias N Stokholm ◽  
Maria B Rabaglino ◽  
Haja N Kadarmideen
Keyword(s):  
Gene Expression ◽  
Data Integration ◽  
Principal Component ◽  
R Package ◽  
Gene Expression Omnibus ◽  
Batch Effect ◽  
Sequencing Data ◽  
Transcriptomic Data ◽  
Ncbi Gene Expression Omnibus ◽  
Forward Stepwise

Transcriptomic data is often expensive and difficult to generate in large cohorts in comparison to genomic data and therefore is often important to integrate multiple transcriptomic datasets from both microarray and next generation sequencing (NGS) based transcriptomic data across similar experiments or clinical trials to improve analytical power and discovery of novel transcripts and genes. However, transcriptomic data integration presents a few challenges including re-annotation and batch effect removal. We developed the Gene Expression Data Integration (GEDI) R package to enable transcriptomic data integration by combining already existing R packages. With just four functions, the GEDI R package makes constructing a transcriptomic data integration pipeline straightforward. Together, the functions overcome the complications in transcriptomic data integration by automatically re-annotating the data and removing the batch effect. The removal of the batch effect is verified with Principal Component Analysis and the data integration is verified using a logistic regression model with forward stepwise feature selection. To demonstrate the functionalities of the GEDI package, we integrated five bovine endometrial transcriptomic datasets from the NCBI Gene Expression Omnibus. The datasets included Affymetrix, Agilent and RNA-sequencing data. Furthermore, we compared the GEDI package to already existing tools and found that GEDI is the only tool that provides a full transcriptomic data integration pipeline including verification of both batch effect removal and data integration.

scholarly journals multiomics: A user-friendly multi-omics data harmonisation R pipeline

F1000Research ◽  
2021 ◽  
Vol 10 ◽  
pp. 538
Author(s):  
Tyrone Chen ◽  
Al J Abadi ◽  
Kim-Anh Lê Cao ◽  
Sonika Tyagi
Keyword(s):  
Data Integration ◽  
Case Studies ◽  
R Package ◽  
Heterogeneous Data ◽  
General Purpose ◽  
Omics Data ◽  
Experimental Conditions ◽  
Seamless Integration ◽  
Data Object ◽  
Omics Data Integration

Data from multiple omics layers of a biological system is growing in quantity, heterogeneity and dimensionality. Simultaneous multi-omics data integration is a growing field of research as it has strong potential to unlock information on previously hidden biological relationships leading to early diagnosis, prognosis and expedited treatments. Many tools for multi-omics data integration are being developed. However, these tools are often restricted to highly specific experimental designs, and types of omics data. While some general methods do exist, they require specific data formats and experimental conditions. A major limitation in the field is a lack of a single or multi-omics pipeline which can accept data in an unrefined, information-rich form pre-integration and subsequently generate output for further investigation. There is an increasing demand for a generic multi-omics pipeline to facilitate general-purpose data exploration and analysis of heterogeneous data. Therefore, we present our R multiomics pipeline as an easy to use and flexible pipeline that takes unrefined multi-omics data as input, sample information and user-specified parameters to generate a list of output plots and data tables for quality control and downstream analysis. We have demonstrated application of the pipeline on two separate COVID-19 case studies. We enabled limited checkpointing where intermediate output is staged to allow continuation after errors or interruptions in the pipeline and generate a script for reproducing the analysis to improve reproducibility. A seamless integration with the mixOmics R package is achieved, as the R data object can be loaded and manipulated with mixOmics functions. Our pipeline can be installed as an R package or from the git repository, and is accompanied by detailed documentation with walkthroughs on two case studies. The pipeline is also available as Docker and Singularity containers.

scholarly journals Gene regulatory effects of a large chromosomal inversion in highland maize

2019 ◽  
Author(s):  
Taylor Crow ◽  
James Ta ◽  
Saghi Nojoomi ◽  
M. Rocío Aguilar-Rangel ◽  
Jorge Vladimir Torres Rodríguez ◽  
...  
Keyword(s):  
Gene Expression ◽  
Zea Mays ◽  
Large Scale ◽  
Chromosomal Inversion ◽  
Experimental Conditions ◽  
Chromosomal Inversions ◽  
Functional Variants ◽  
Original Manuscript ◽  
Locally Adaptive ◽  
Highland Maize

AbstractChromosomal inversions play an important role in local adaptation. Inversions can capture multiple locally adaptive functional variants in a linked block by repressing recombination. However, this recombination suppression makes it difficult to identify the genetic mechanisms that underlie an inversion’s role in adaption. In this study, we explore how large-scale transcriptomic data can be used to dissect the functional importance of a 13 Mb inversion locus (Inv4m) found almost exclusively in highland populations of maize (Zea mays ssp. mays). Inv4m introgressed into highland maize from the wild relative Zea mays ssp. mexicana, also present in the highlands of Mexico, and is thought to be important for the adaptation of these populations to cultivation in highland environments. First, using a large publicly available association mapping panel, we confirmed that Inv4m is associated with locally adaptive agronomic phenotypes, but only in highland fields. Second, we created two families segregating for standard and inverted haplotypess of Inv4m in a isogenic B73 background, and measured gene expression variation association with Inv4m across 9 tissues in two experimental conditions. With these data, we quantified both the global transcriptomic effects of the highland Inv4m haplotype, and the local cis-regulatory variation present within the locus. We found diverse physiological effects of Inv4m, and speculate that the genetic basis of its effects on adaptive traits is distributed across many separate functional variants.Author SummaryChromosomal inversions are an important type of genomic structural variant. However, mapping causal alleles within their boundaries is difficult because inversions suppress recombination between homologous chromosomes. This means that inversions, regardless of their size, are inherited as a unit. We leveraged the high-dimensional phenotype of gene expression as a tool to study the genetics of a large chromosomal inversion found in highland maize populations in Mexico - Inv4m. We grew plants carrying multiple versions of Inv4m in a common genetic background, and quantified the transcriptional reprogramming induced by alternative alleles at the locus. Inv4m has been shown in previous studies to have a large effect on flowering, but we show that the functional variation within Inv4m affects many developmental and physiological processes.Author ContributionsT. Crow, R. Rellan-Alvarez, R. Sawers and D. Runcie conceived and designed the experiment. M. Aguilar-Rangel, J. Rodrǵuez, R. Rellan-Alvarez and R. Sawers generated the segregating families. T. Crow, J. Ta, S. Nojoomi, M. Aguilar-Rangel, J. Rodrǵuez D. Gates, D. Runcie performed the experiment. T. Crow, D. Gates, D. Runcie analyzed the data. T. Crow, D. Runcie wrote the original manuscript, and R. Rellan-Alvarez and R. Sawers provided review and editing.

scholarly journals deepMNN: Deep Learning-Based Single-Cell RNA Sequencing Data Batch Correction Using Mutual Nearest Neighbors

2021 ◽  
Vol 12 ◽  
Author(s):  
Bin Zou ◽  
Tongda Zhang ◽  
Ruilong Zhou ◽  
Xiaosen Jiang ◽  
Huanming Yang ◽  
...  
Keyword(s):  
Deep Learning ◽  
Single Cell ◽  
Rna Sequencing ◽  
Large Scale ◽  
Cell Types ◽  
Batch Effect ◽  
Batch Effects ◽  
Batch Correction ◽  
Single Cell Rna Sequencing ◽  
Identical Cell

It is well recognized that batch effect in single-cell RNA sequencing (scRNA-seq) data remains a big challenge when integrating different datasets. Here, we proposed deepMNN, a novel deep learning-based method to correct batch effect in scRNA-seq data. We first searched mutual nearest neighbor (MNN) pairs across different batches in a principal component analysis (PCA) subspace. Subsequently, a batch correction network was constructed by stacking two residual blocks and further applied for the removal of batch effects. The loss function of deepMNN was defined as the sum of a batch loss and a weighted regularization loss. The batch loss was used to compute the distance between cells in MNN pairs in the PCA subspace, while the regularization loss was to make the output of the network similar to the input. The experiment results showed that deepMNN can successfully remove batch effects across datasets with identical cell types, datasets with non-identical cell types, datasets with multiple batches, and large-scale datasets as well. We compared the performance of deepMNN with state-of-the-art batch correction methods, including the widely used methods of Harmony, Scanorama, and Seurat V4 as well as the recently developed deep learning-based methods of MMD-ResNet and scGen. The results demonstrated that deepMNN achieved a better or comparable performance in terms of both qualitative analysis using uniform manifold approximation and projection (UMAP) plots and quantitative metrics such as batch and cell entropies, ARI F1 score, and ASW F1 score under various scenarios. Additionally, deepMNN allowed for integrating scRNA-seq datasets with multiple batches in one step. Furthermore, deepMNN ran much faster than the other methods for large-scale datasets. These characteristics of deepMNN made it have the potential to be a new choice for large-scale single-cell gene expression data analysis.

scholarly journals Implementing privacy-preserving record linkage: welcome to the real world

2017 ◽  
Vol 1 (1) ◽  
Author(s):  
James Boyd ◽  
Anna Ferrante ◽  
Adrian Brown ◽  
Sean Randall ◽  
James Semmens
Keyword(s):  
Data Integration ◽  
Real World ◽  
Error Detection ◽  
Record Linkage ◽  
Large Scale ◽  
Privacy Preserving ◽  
Estimation Methods ◽  
Experimental Conditions ◽  
Encrypted Data ◽  
Personally Identifying Information

ABSTRACT ObjectivesWhile record linkage has become a strategic research priority within Australia and internationally, legal and administrative issues prevent data linkage in some situations due to privacy concerns. Even current best practices in record linkage carry some privacy risk as they require the release of personally identifying information to trusted third parties. Application of record linkage systems that do not require the release of personal information can overcome legal and privacy issues surrounding data integration. Current conceptual and experimental privacy-preserving record linkage (PPRL) models show promise in addressing data integration challenges but do not yet address all of the requirements for real-world operations. This paper aims to identify and address some of the challenges of operationalising PPRL frameworks. ApproachTraditional linkage processes involve comparing personally identifying information (name, address, date of birth) on pairs of records to determine whether the records belong to the same person. Designing appropriate linkage strategies is an important part of the process. These are typically based on the analysis of data attributes (metadata) such as data completeness, consistency, constancy and field discriminating power. Under a PPRL model, however, these factors cannot be discerned from the encrypted data, so an alternative approach is required. This paper explores methods for data profiling, blocking, weight/threshold estimation and error detection within a PPRL framework. ResultsProbabilistic record linkage typically involves the estimation of weights and thresholds to optimise the linkage and ensure highly accurate results. The paper outlines the metadata requirements and automated methods necessary to collect data without compromising privacy. We present work undertaken to develop parameter estimation methods which can help optimise a linkage strategy without the release of personally identifiable information. These are required in all parts of the privacy preserving record linkage process (pre-processing, standardising activities, linkage, grouping and extracting). ConclusionsPPRL techniques that operate on encrypted data have the potential for large-scale record linkage, performing both accurately and efficiently under experimental conditions. Our research has advanced the current state of PPRL with a framework for secure record linkage that can be implemented to improve and expand linkage service delivery while protecting an individual’s privacy. However, more research is required to supplement this technique with additional elements to ensure the end-to-end method is practical and can be incorporated into real-world models.

scholarly journals Data Matrix Normalization and Merging Strategies Minimize Batch-specific Systemic Variation in scRNA-Seq Data

2021 ◽  
Author(s):  
Benjamin R Babcock ◽  
Astrid Kosters ◽  
Junkai Yang ◽  
Mackenzie L White ◽  
Eliver Ghosn
Keyword(s):  
Data Integration ◽  
Large Scale ◽  
Human Peripheral Blood ◽  
Data Interpretation ◽  
Batch Effect ◽  
Data Matrix ◽  
Gene Expression Matrix ◽  
Downstream Effects ◽  
Experimental Approaches ◽  
Multiple Samples

Single-cell RNA sequencing (scRNA-seq) can reveal accurate and sensitive RNA abundance in a single sample, but robust integration of multiple samples remains challenging. Large-scale scRNA-seq data generated by different workflows or laboratories can contain batch-specific systemic variation. Such variation challenges data integration by confounding sample-specific biology with undesirable batch-specific systemic effects. Therefore, there is a need for guidance in selecting computational and experimental approaches to minimize batch-specific impacts on data interpretation and a need to empirically evaluate the sources of systemic variation in a given dataset. To uncover the contributions of experimental variables to systemic variation, we intentionally perturb four potential sources of batch-effect in five human peripheral blood samples. We investigate sequencing replicate, sequencing depth, sample replicate, and the effects of pooling libraries for concurrent sequencing. To quantify the downstream effects of these variables on data interpretation, we introduced a new scoring metric, the Cell Misclassification Statistic (CMS), which identifies losses to cell type fidelity that occur when merging datasets of different batches. CMS reveals an undesirable overcorrection by popular batch-effect correction and data integration methods. We show that optimizing gene expression matrix normalization and merging can reduce the need for batch-effect correction and minimize the risk of overcorrecting true biological differences between samples.

Integrating –omics data into genome-scale metabolic network models: principles and challenges

2018 ◽  
Vol 62 (4) ◽  
pp. 563-574 ◽  
Author(s):  
Charlotte Ramon ◽  
Mattia G. Gollub ◽  
Jörg Stelling
Keyword(s):  
Data Integration ◽  
Large Scale ◽  
Network Models ◽  
Omics Data ◽  
Scale Models ◽  
Common Framework ◽  
Genome Scale ◽  
Constraint Based Models ◽  
Omics Data Integration

At genome scale, it is not yet possible to devise detailed kinetic models for metabolism because data on the in vivo biochemistry are too sparse. Predictive large-scale models for metabolism most commonly use the constraint-based framework, in which network structures constrain possible metabolic phenotypes at steady state. However, these models commonly leave many possibilities open, making them less predictive than desired. With increasingly available –omics data, it is appealing to increase the predictive power of constraint-based models (CBMs) through data integration. Many corresponding methods have been developed, but data integration is still a challenge and existing methods perform less well than expected. Here, we review main approaches for the integration of different types of –omics data into CBMs focussing on the methods’ assumptions and limitations. We argue that key assumptions – often derived from single-enzyme kinetics – do not generally apply in the context of networks, thereby explaining current limitations. Emerging methods bridging CBMs and biochemical kinetics may allow for –omics data integration in a common framework to provide more accurate predictions.

Sign in / Sign up

Export Citation Format

Share Document