scholarly journals Lossless Indexing with Counting de Bruijn Graphs

2021 ◽  
Author(s):  
Mikhail Karasikov ◽  
Harun Mustafa ◽  
Gunnar Rätsch ◽  
André Kahles
Keyword(s):  
High Throughput Sequencing ◽  
Sparse Matrices ◽  
Rna Seq ◽  
Sequencing Data ◽  
De Bruijn Graphs ◽  
High Throughput Sequencing Data ◽  
Alignment Algorithms ◽  
Compressed Data Structures ◽  
De Bruijn ◽  
Public Repositories

High-throughput sequencing data is rapidly accumulating in public repositories. Making this resource accessible for interactive analysis at scale requires efficient approaches for its storage and indexing. There have recently been remarkable advances in solving the experiment discovery problem and building compressed representations of annotated de Bruijn graphs where k-mer sets can be efficiently indexed and interactively queried. However, approaches for representing and retrieving other quantitative attributes such as gene expression or genome positions in a general manner have yet to be developed. In this work, we propose the concept of Counting de Bruijn graphs generalizing the notion of annotated (or colored) de Bruijn graphs. Counting de Bruijn graphs supplement each node-label relation with one or many attributes (e.g., a k-mer count or its positions in genome). To represent them, we first observe that many schemes for the representation of compressed binary matrices already support the rank operation on the columns or rows, which can be used to define an inherent indexing of any additional quantitative attributes. Based on this property, we generalize these schemes and introduce a new approach for representing non-binary sparse matrices in compressed data structures. Finally, we notice that relation attributes are often easily predictable from a node's local neighborhood in the graph. Notable examples are genome positions shifting by 1 for neighboring nodes in the graph, or expression levels that are often shared across neighbors. We exploit this regularity of graph annotations and apply an invertible delta-like coding to achieve better compression. We show that Counting de Bruijn graphs index k-mer counts from 2,652 human RNA-Seq read sets in representations over 8-fold smaller and yet faster to query compared to state-of-the-art bioinformatics tools. Furthermore, Counting de Bruijn graphs with positional annotations losslessly represent entire reads in indexes on average 27% smaller than the input compressed with gzip -9 for human Illumina RNA-Seq and 57% smaller for PacBio HiFi sequencing of viral samples. A complete joint searchable index of all viral PacBio SMRT reads from NCBI's SRA (152,884 read sets, 875 Gbp) comprises only 178 GB. Finally, on the full RefSeq collection, they generate a lossless and fully queryable index that is 4.4-fold smaller compared to the MegaBLAST index. The techniques proposed in this work naturally complement existing methods and tools employing de Bruijn graphs and significantly broaden their applicability: from indexing k-mer counts and genome positions to implementing novel sequence alignment algorithms on top of highly compressed and fully searchable graph-based sequence indexes.

Inference of viral quasispecies with a paired de Bruijn graph

2020 ◽  
Author(s):  
Borja Freire ◽  
Susana Ladra ◽  
Jose R Paramá ◽  
Leena Salmela
Keyword(s):  
High Throughput Sequencing ◽  
De Novo ◽  
Supplementary Information ◽  
De Bruijn Graph ◽  
Viral Quasispecies ◽  
Sequencing Data ◽  
De Bruijn Graphs ◽  
Sequencing Errors ◽  
High Throughput Sequencing Data ◽  
De Bruijn

Abstract Motivation RNA viruses exhibit a high mutation rate and thus they exist in infected cells as a population of closely related strains called viral quasispecies. The viral quasispecies assembly problem asks to characterize the quasispecies present in a sample from high-throughput sequencing data. We study the de novo version of the problem, where reference sequences of the quasispecies are not available. Current methods for assembling viral quasispecies are either based on overlap graphs or on de Bruijn graphs. Overlap graph-based methods tend to be accurate but slow, whereas de Bruijn graph-based methods are fast but less accurate. Results We present viaDBG, which is a fast and accurate de Bruijn graph-based tool for de novo assembly of viral quasispecies. We first iteratively correct sequencing errors in the reads, which allows us to use large k-mers in the de Bruijn graph. To incorporate the paired-end information in the graph, we also adapt the paired de Bruijn graph for viral quasispecies assembly. These features enable the use of long-range information in contig construction without compromising the speed of de Bruijn graph-based approaches. Our experimental results show that viaDBG is both accurate and fast, whereas previous methods are either fast or accurate but not both. In particular, viaDBG has comparable or better accuracy than SAVAGE, while being at least nine times faster. Furthermore, the speed of viaDBG is comparable to PEHaplo but viaDBG is able to retrieve also low abundance quasispecies, which are often missed by PEHaplo. Availability and implementation viaDBG is implemented in C++ and it is publicly available at https://bitbucket.org/bfreirec1/viadbg. All datasets used in this article are publicly available at https://bitbucket.org/bfreirec1/data-viadbg/. Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals Succinct Dynamic de Bruijn Graphs

2020 ◽  
Author(s):  
Bahar Alipanahi ◽  
Alan Kuhnle ◽  
Simon J. Puglisi ◽  
Leena Salmela ◽  
Christina Boucher
Keyword(s):  
Data Structures ◽  
Large Scale ◽  
High Throughput Sequencing ◽  
De Bruijn Graph ◽  
Sequencing Data ◽  
Efficient Manner ◽  
De Bruijn Graphs ◽  
High Throughput Sequencing Data ◽  
Efficient Data ◽  
De Bruijn

AbstractMotivationThe de Bruijn graph is one of the fundamental data structures for analysis of high throughput sequencing data. In order to be applicable to population-scale studies, it is essential to build and store the graph in a space- and time-efficient manner. In addition, due to the ever-changing nature of population studies, it has become essential to update the graph after construction e.g. add and remove nodes and edges. Although there has been substantial effort on making the construction and storage of the graph efficient, there is a limited amount of work in building the graph in an efficient and mutable manner. Hence, most space efficient data structures require complete reconstruction of the graph in order to add or remove edges or nodes.ResultsIn this paper we present DynamicBOSS, a succinct representation of the de Bruijn graph that allows for an unlimited number of additions and deletions of nodes and edges. We compare our method with other competing methods and demonstrate that DynamicBOSS is the only method that supports both addition and deletion and is applicable to very large samples (e.g. greater than 15 billion k-mers). Competing dynamic methods e.g., FDBG (Crawford et al., 2018) cannot be constructed on large scale datasets, or cannot support both addition and deletion e.g., BiFrost (Holley and Melsted, 2019).AvailabilityDynamicBOSS is publicly available at https://github.com/baharpan/[email protected]

scholarly journals Succinct Dynamic de Bruijn Graphs

2020 ◽  
Author(s):  
Bahar Alipanahi ◽  
Alan Kuhnle ◽  
Simon J Puglisi ◽  
Leena Salmela ◽  
Christina Boucher
Keyword(s):  
Data Structures ◽  
Large Scale ◽  
High Throughput Sequencing ◽  
Supplementary Information ◽  
De Bruijn Graph ◽  
Sequencing Data ◽  
Efficient Manner ◽  
De Bruijn Graphs ◽  
High Throughput Sequencing Data ◽  
De Bruijn

Abstract Motivation The de Bruijn graph is one of the fundamental data structures for analysis of high throughput sequencing data. In order to be applicable to population-scale studies, it is essential to build and store the graph in a space- and time- efficient manner. In addition, due to the ever-changing nature of population studies, it has become essential to update the graph after construction e.g. add and remove nodes and edges. Although there has been substantial effort on making the construction and storage of the graph efficient, there is a limited amount of work in building the graph in an efficient and mutable manner. Hence, most space efficient data structures require complete reconstruction of the graph in order to add or remove edges or nodes. Results In this paper we present DynamicBOSS, a succinct representation of the de Bruijn graph that allows for an unlimited number of additions and deletions of nodes and edges. We compare our method with other competing methods and demonstrate that DynamicBOSS is the only method that supports both addition and deletion and is applicable to very large samples (e.g. greater than 15 billion k-mers). Competing dynamic methods e.g., FDBG (Crawford et al., 2018) cannot be constructed on large scale datasets, or cannot support both addition and deletion e.g., BiFrost (Holley and Melsted, 2019). Availability DynamicBOSS is publicly available at https://github.com/baharpan/dynboss. Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals HTSeq - A Python framework to work with high-throughput sequencing data

2014 ◽  
Author(s):  
Simon Anders ◽  
Paul Theodor Pyl ◽  
Wolfgang Huber
Keyword(s):  
High Throughput ◽  
High Throughput Sequencing ◽  
Rapid Development ◽  
Differential Expression Analysis ◽  
Rna Seq ◽  
Sequencing Data ◽  
Standard Work ◽  
Data Formats ◽  
High Throughput Sequencing Data ◽  
Python Package

Motivation: A large choice of tools exists for many standard tasks in the analysis of high-throughput sequencing (HTS) data. However, once a project deviates from standard work flows, custom scripts are needed. Results: We present HTSeq, a Python library to facilitate the rapid development of such scripts. HTSeq offers parsers for many common data formats in HTS projects, as well as classes to represent data such as genomic coordinates, sequences, sequencing reads, alignments, gene model information, variant calls, and provides data structures that allow for querying via genomic coordinates. We also present htseq-count, a tool developed with HTSeq that preprocesses RNA-Seq data for differential expression analysis by counting the overlap of reads with genes. Availability: HTSeq is released as open-source software under the GNU General Public Licence and available from http://www-huber.embl.de/HTSeq or from the Python Package Index, https://pypi.python.org/pypi/HTSeq

scholarly journals Building Genomic Analysis Pipelines in a Hackathon Setting with Bioinformatician Teams: DNA-seq, Epigenomics, Metagenomics and RNA-seq

2015 ◽  
Author(s):  
Ben Busby ◽  
Allissa Dillman ◽  
Claire L. Simpson ◽  
Ian Fingerman ◽  
Sijung Yun ◽  
...  
Keyword(s):  
Web Service ◽  
High Throughput ◽  
High Throughput Sequencing ◽  
Genomic Analysis ◽  
National Institutes Of Health ◽  
Rna Seq ◽  
Sequencing Data ◽  
Collaborative Software ◽  
High Throughput Sequencing Data ◽  
Collaborative Software Development

We assembled teams of genomics professionals to assess whether we could rapidly develop pipelines to answer biological questions commonly asked by biologists and others new to bioinformatics by facilitating analysis of high-throughput sequencing data. In January 2015, teams were assembled on the National Institutes of Health (NIH) campus to address questions in the DNA-seq, epigenomics, metagenomics and RNA-seq subfields of genomics. The only two rules for this hackathon were that either the data used were housed at the National Center for Biotechnology Information (NCBI) or would be submitted there by a participant in the next six months, and that all software going into the pipeline was open-source or open-use. Questions proposed by organizers, as well as suggested tools and approaches, were distributed to participants a few days before the event and were refined during the event. Pipelines were published on GitHub, a web service providing publicly available, free-usage tiers for collaborative software development (https://github.com/features/). The code was published at https://github.com/DCGenomics/ with separate repositories for each team, starting with hackathon_v001.

scholarly journals Reference-free compression of high throughput sequencing data with a probabilistic de Bruijn graph

2015 ◽  
Vol 16 (1) ◽  
Author(s):  
Gaëtan Benoit ◽  
Claire Lemaitre ◽  
Dominique Lavenier ◽  
Erwan Drezen ◽  
Thibault Dayris ◽  
...  
Keyword(s):  
High Throughput ◽  
High Throughput Sequencing ◽  
De Bruijn Graph ◽  
Sequencing Data ◽  
High Throughput Sequencing Data ◽  
De Bruijn

scholarly journals Reference-free comparison of microbial communities via de Bruijn graphs

2016 ◽  
Author(s):  
Serghei Mangul ◽  
David Koslicki
Keyword(s):  
Microbial Communities ◽  
High Throughput Sequencing ◽  
Microbial Community Composition ◽  
De Bruijn Graph ◽  
Data Sets ◽  
Sequencing Data ◽  
De Bruijn Graphs ◽  
De Bruijn ◽  
The Individual ◽  
And Control

ABSTRACTMicrobial communities inhabiting the human body exhibit significant variability across different individuals and tissues, and are suggested to play an important role in health and disease. High-throughput sequencing offers unprecedented possibilities to profile microbial community composition, but limitations of existing taxonomic classification methods (including incompleteness of existing microbial reference databases) limits the ability to accurately compare microbial communities across different samples. In this paper, we present a method able to overcome these limitations by circumventing the classification step and directly using the sequencing data to compare microbial communities. The proposed method provides a powerful reference-free way to assess differences in microbial abundances across samples. This method, called EMDeBruijn, condenses the sequencing data into a de Bruijn graph. The Earth Mover's Distance (EMD) is then used to measure similarities and differences of the microbial communities associated with the individual graphs. We apply this method to RNA-Seq data sets from a coronary artery calcification (CAC) study and shown that EMDeBruijn is able to differentiate between case and control CAC samples while utilizing all the candidate microbial reads. We compare these results to current reference-based methods, which are shown to have a limited capacity to discriminate between case and control samples. We conclude that this reference-free approach is a viable choice in comparative metatranscriptomic studies.

The simple fool's guide to population genomics via RNA ‐Seq: an introduction to high‐throughput sequencing data analysis

2012 ◽  
Vol 12 (6) ◽  
pp. 1058-1067 ◽  
Author(s):  
Pierre Wit ◽  
Melissa H. Pespeni ◽  
Jason T. Ladner ◽  
Daniel J. Barshis ◽  
François Seneca ◽  
...  
Keyword(s):  
Data Analysis ◽  
High Throughput ◽  
High Throughput Sequencing ◽  
Population Genomics ◽  
Rna Seq ◽  
Sequencing Data ◽  
High Throughput Sequencing Data ◽  
Sequencing Data Analysis

scholarly journals AlnC: An extensive database of long non-coding RNAs in angiosperms

PLoS ONE ◽  
2021 ◽  
Vol 16 (4) ◽  
pp. e0247215
Author(s):  
Ajeet Singh ◽  
A. T. Vivek ◽  
Shailesh Kumar
Keyword(s):  
Plant Species ◽  
Large Scale ◽  
High Throughput Sequencing ◽  
Significant Rise ◽  
Open Reading Frames ◽  
Sequencing Data ◽  
High Throughput Sequencing Data ◽  
Non Coding Rnas ◽  
Public Repositories ◽  
User Friendly

Long non-coding RNAs (lncRNAs) are defined as transcripts of greater than 200 nucleotides that play a crucial role in various cellular processes such as the development, differentiation and gene regulation across all eukaryotes, including plant cells. Since the last decade, there has been a significant rise in our understanding of lncRNA molecular functions in plants, resulting in an exponential increase in lncRNA transcripts, while these went unannounced from the major Angiosperm plant species despite the availability of large-scale high throughput sequencing data in public repositories. We, therefore, developed a user-friendly, open-access web interface, AlnC (Angiosperm lncRNA Catalogue) for the exploration of lncRNAs in diverse Angiosperm plant species using recent 1000 plant (1KP) trancriptomes data. The current version of AlnC offers 10,855,598 annotated lncRNA transcripts across 682 Angiosperm plant species encompassing 809 tissues. To improve the user interface, we added features for browsing, searching, and downloading lncRNA data, interactive graphs, and an online BLAST service. Additionally, each lncRNA record is annotated with possible small open reading frames (sORFs) to facilitate the study of peptides encoded within lncRNAs. With this user-friendly interface, we anticipate that AlnC will provide a rich source of lncRNAs for small-and large-scale studies in a variety of flowering plants, as well as aid in the improvement of key characteristics in relevance to their economic importance. Database URL: http://www.nipgr.ac.in/AlnC

Sign in / Sign up

Export Citation Format

Share Document