scholarly journals A space and time-efficient index for the compacted colored de Bruijn graph

2017 ◽  
Author(s):  
Fatemeh Almodaresi ◽  
Hirak Sarkar ◽  
Rob Patro
Keyword(s):  
Data Structure ◽  
Pattern Search ◽  
De Bruijn Graph ◽  
Existing Structures ◽  
Link Type ◽  
Reference Information ◽  
De Bruijn ◽  
Colored De Bruijn Graph ◽  
Asymptotically Efficient ◽  
Fast Access

AbstractWe present a novel data structure for representing and indexing the compacted colored de Bruijn graph, which allows for efficient pattern matching and retrieval of the reference information associated with each k-mer. As the popularity of the de Bruijn graph as an index has increased over the past few years, so have the number of proposed representations of this structure. Existing structures typically fall into two categories; those that are hashing-based and provide very fast access to the underlying k-mer information, and those that are space-frugal and provide asymptotically efficient but practically slower pattern search.Our representation achieves a compromise between these two extremes. By building upon minimum perfect hashing, carefully organizing our data structure, and making use of succinct representations where applicable, our data structure provides practically fast k-mer lookup while greatly reducing the space compared to traditional hashing-based implementations. Further, we describe a sampling scheme built on the same underlying representation, which provides the ability to trade off k-mer query speed for a reduction in the de Bruijn graph index size. We believe this representation strikes a desirable balance between speed and space usage, and it will allow for fast search on large reference sequences.Pufferfish is developed in C++11, is open source (GPL v3), and is available at https://github.com/COMBINE-lab/Pufferfish. The scripts used to generate the results in this manuscript are available at https://github.com/COMBINE-lab/pufferfish_experiments.

scholarly journals Succinct Colored de Bruijn Graphs

2016 ◽  
Author(s):  
Keith Belk ◽  
Christina Boucher ◽  
Alexander Bowe ◽  
Travis Gagie ◽  
Paul Morley ◽  
...  
Keyword(s):  
Data Structure ◽  
Genetic Variants ◽  
Population Level ◽  
De Bruijn Graph ◽  
De Bruijn Graphs ◽  
Graph Representations ◽  
Additional Information ◽  
Level Data ◽  
De Bruijn ◽  
Colored De Bruijn Graph

Iqbal et al. (Nature Genetics, 2012) introduced the colored de Bruijn graph, a variant of the classic de Bruijn graph, which is aimed at "detecting and genotyping simple and complex genetic variants in an individual or population". Because they are intended to be applied to massive population level data, it is essential that the graphs be represented efficiently. Unfortunately, current succinct de Bruijn graph representations are not directly applicable to the colored de Bruijn graph, which require additional information to be succinctly encoded as well as support for non-standard traversal operations. Our data structure dramatically reduces the amount of memory required to store and use the colored de Bruijn graph, with some penalty to runtime, allowing it to be applied in much larger and more ambitious sequence projects than was previously possible.

scholarly journals Somatic variant analysis of linked-reads sequencing data with Lancet

2020 ◽  
Author(s):  
Rajeeva Musunuri ◽  
Kanika Arora ◽  
André Corvelo ◽  
Minita Shah ◽  
Jennifer Shelton ◽  
...  
Keyword(s):  
Supplementary Information ◽  
De Bruijn Graph ◽  
Haplotype Structure ◽  
Sequencing Data ◽  
Somatic Variant ◽  
Local Assembly ◽  
De Bruijn ◽  
Variant Analysis ◽  
Colored De Bruijn Graph ◽  
Commercial Research

Abstract Summary We present a new version of the popular somatic variant caller, Lancet, that supports the analysis of linked-reads sequencing data. By seamlessly integrating barcodes and haplotype read assignments within the colored De Bruijn graph local-assembly framework, Lancet computes a barcode-aware coverage and identifies variants that disagree with the local haplotype structure. Availability and implementation Lancet is implemented in C++ and available for academic and non-commercial research purposes as an open-source package at https://github.com/nygenome/lancet. Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals REINDEER: efficient indexing of k-mer presence and abundance in sequencing datasets

2020 ◽  
Author(s):  
Camille Marchet ◽  
Zamin Iqbal ◽  
Daniel Gautheret ◽  
Mikael Salson ◽  
Rayan Chikhi
Keyword(s):  
Large Datasets ◽  
Computational Method ◽  
De Bruijn Graph ◽  
Rna Seq ◽  
Indexing Methods ◽  
Link Type ◽  
De Bruijn

AbstractMotivationIn this work we present REINDEER, a novel computational method that performs indexing of sequences and records their abundances across a collection of datasets. To the best of our knowledge, other indexing methods have so far been unable to record abundances efficiently across large datasets.ResultsWe used REINDEER to index the abundances of sequences within 2,585 human RNA-seq experiments in 45 hours using only 56 GB of RAM. This makes REINDEER the first method able to record abundances at the scale of 4 billion distinct k-mers across 2,585 datasets. REINDEER also supports exact presence/absence queries of k-mers. Briefly, REINDEER constructs the compacted de Bruijn graph (DBG) of each dataset, then conceptually merges those DBGs into a single global one. Then, REINDEER constructs and indexes monotigs, which in a nutshell are groups of k-mers of similar abundances.Availabilityhttps://github.com/kamimrcht/[email protected]

scholarly journals A space and time-efficient index for the compacted colored de Bruijn graph

2018 ◽  
Vol 34 (13) ◽  
pp. i169-i177 ◽  
Author(s):  
Fatemeh Almodaresi ◽  
Hirak Sarkar ◽  
Avi Srivastava ◽  
Rob Patro
Keyword(s):  
De Bruijn Graph ◽  
Space And Time ◽  
De Bruijn ◽  
Colored De Bruijn Graph

scholarly journals Somatic variant analysis of linked-reads sequencing data with Lancet

2020 ◽  
Author(s):  
Rajeeva Musunuri ◽  
Kanika Arora ◽  
André Corvelo ◽  
Minita Shah ◽  
Jennifer Shelton ◽  
...  
Keyword(s):  
Open Source ◽  
De Bruijn Graph ◽  
Haplotype Structure ◽  
Sequencing Data ◽  
Somatic Variant ◽  
Local Assembly ◽  
De Bruijn ◽  
Variant Analysis ◽  
Colored De Bruijn Graph ◽  
Commercial Research

AbstractSummaryWe present a new version of the popular somatic variant caller, Lancet, that supports the analysis of linked-reads sequencing data. By seamlessly integrating barcodes and haplotype read assignments within the colored De Bruijn graph local-assembly framework, Lancet computes a barcode-aware coverage and identifies variants that disagree with the local haplotype structure.Availability and ImplementationLancet is implemented in C++ and is available for academic and non-commercial research purposes as an open-source package at https://github.com/nygenome/[email protected]

scholarly journals Cuttlefish: Fast, parallel, and low-memory compaction of de Bruijn graphs from large-scale genome collections

2020 ◽  
Author(s):  
Jamshed Khan ◽  
Rob Patro
Keyword(s):  
Large Scale ◽  
De Bruijn Graph ◽  
Comparative Genomic ◽  
De Bruijn Graphs ◽  
Long Reads ◽  
Genomic Analyses ◽  
Finite State ◽  
De Bruijn ◽  
Colored De Bruijn Graph ◽  
Memory Compaction

AbstractMotivationThe construction of the compacted de Bruijn graph from a large collection of reference genomes is a task of increasing interest in genomic analyses. For example, compacted colored reference de Bruijn graphs are increasingly used as sequence indices for the purposes of alignment of short and long reads. Also, as we sequence and assemble a greater diversity of individual genomes, the compacted colored de Bruijn graph can be used as the basis for methods aiming to perform comparative genomic analyses on these genomes. While algorithms have been developed to construct the compacted colored de Bruijn graph from reference sequences, there is still room for improvement, especially in the memory and the runtime performance as the number and the scale of the genomes over which the de Bruijn graph is built grow.ResultsWe introduce a new algorithm, implemented in the tool Cuttlefish, to construct the colored compacted de Bruijn graph from a collection of one or more genome references. Cuttlefish introduces a novel modeling scheme of the de Bruijn graph vertices as finite-state automata, and constrains the state-space for the automata to enable tracking of their transitioning states with very low memory usage. Cuttlefish is also fast and highly parallelizable. Experimental results demonstrate that the algorithm scales much better than existing approaches, especially as the number and scale of the input references grow. For example, on a typical shared-memory machine, Cuttlefish constructed the compacted graph for 100 human genomes in less than 7 hours, using ~29 GB of memory; no other tested tool successfully completed this task on the testing hardware. We also applied Cuttlefish on 11 diverse conifer plant genomes, and the compacted graph was constructed in under 11 hours, using ~84 GB of memory, while the only other tested tool able to complete this compaction on our hardware took more than 16 hours and ~289 GB of memory.AvailabilityCuttlefish is written in C++14, and is available under an open source license at https://github.com/COMBINE-lab/[email protected]

scholarly journals Metagenome SNP calling via read-colored de Bruijn graphs

2020 ◽  
Author(s):  
Bahar Alipanahi ◽  
Martin D Muggli ◽  
Musa Jundi ◽  
Noelle R Noyes ◽  
Christina Boucher
Keyword(s):  
Pathogenic Bacteria ◽  
Read Length ◽  
Supplementary Information ◽  
De Bruijn Graph ◽  
Nucleotide Polymorphisms ◽  
Chromosomal Dna ◽  
Shotgun Metagenomics ◽  
De Bruijn Graphs ◽  
De Bruijn ◽  
Colored De Bruijn Graph

Abstract Motivation Metagenomics refers to the study of complex samples containing of genetic contents of multiple individual organisms and, thus, has been used to elucidate the microbiome and resistome of a complex sample. The microbiome refers to all microbial organisms in a sample, and the resistome refers to all of the antimicrobial resistance (AMR) genes in pathogenic and non-pathogenic bacteria. Single-nucleotide polymorphisms (SNPs) can be effectively used to ‘fingerprint’ specific organisms and genes within the microbiome and resistome and trace their movement across various samples. However, to effectively use these SNPs for this traceability, a scalable and accurate metagenomics SNP caller is needed. Moreover, such an SNP caller should not be reliant on reference genomes since 95% of microbial species is unculturable, making the determination of a reference genome extremely challenging. In this article, we address this need. Results We present LueVari, a reference-free SNP caller based on the read-colored de Bruijn graph, an extension of the traditional de Bruijn graph that allows repeated regions longer than the k-mer length and shorter than the read length to be identified unambiguously. LueVari is able to identify SNPs in both AMR genes and chromosomal DNA from shotgun metagenomics data with reliable sensitivity (between 91% and 99%) and precision (between 71% and 99%) as the performance of competing methods varies widely. Furthermore, we show that LueVari constructs sequences containing the variation, which span up to 97.8% of genes in datasets, which can be helpful in detecting distinct AMR genes in large metagenomic datasets. Availability and implementation Code and datasets are publicly available at https://github.com/baharpan/cosmo/tree/LueVari. Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals Rainbowfish: A Succinct Colored de Bruijn Graph Representation

2017 ◽  
Author(s):  
Fatemeh Almodaresi ◽  
Prashant Pandey ◽  
Rob Patro
Keyword(s):  
Relevant Information ◽  
Large Datasets ◽  
Graph Representation ◽  
Combinatorial Structure ◽  
De Bruijn Graph ◽  
Color Information ◽  
Efficient Representation ◽  
De Bruijn ◽  
Colored De Bruijn Graph ◽  
Memory Efficient

AbstractThe colored de Bruijn graph— a variant of the de Bruijn graph which associates each edge (i.e., k-mer) with some set of colors — is an increasingly important combinatorial structure in computational biology. Iqbal et al. demonstrated the utility of this structure for representing and assembling a collection (pop-ulation) of genomes, and showed how it can be used to accurately detect genetic variants. Muggli et al. introduced VARI, a representation of the colored de Bruijn graph that adopts the BOSS representation for the de Bruijn graph topology and achieves considerable savings in space over Cortex, albeit with some sacrifice in speed. The memory-efficient representation of VARI allows the colored de Bruijn graph to be constructed and analyzed for large datasets, beyond what is possible with Cortex.In this paper, we introduce Rainbowfish, a succinct representation of the color information of the colored de Bruijn graph that reduces the space usage even further. Our representation also uses BOSS to represent the de Bruijn graph, but decomposes the color sets based on an equivalence relation and exploits the inherent skewness in the distribution of these color sets. The Rainbowfish representation is compressed based on the 0th-order entropy of the color sets, which can lead to a significant reduction in the space required to store the relevant information for each edge. In practice, Rainbowfish achieves up to a 20 × improvement in space over VARI. Rainbowfish is written in C++11 and is available at https://github.com/COMBINE-lab/rainbowfish.

scholarly journals Integrating long-range connectivity information into de Bruijn graphs

2017 ◽  
Author(s):  
Isaac Turner ◽  
Kiran V Garimella ◽  
Zamin Iqbal ◽  
Gil McVean
Keyword(s):  
Data Structure ◽  
Long Range ◽  
De Bruijn Graph ◽  
Sequencing Data ◽  
Genomic Context ◽  
De Bruijn Graphs ◽  
Connectivity Information ◽  
String Graph ◽  
Free Data ◽  
De Bruijn

AbstractMotivationThe de Bruijn graph is a simple and efficient data structure that is used in many areas of sequence analysis including genome assembly, read error correction and variant calling. The data structure has a single parameter k, is straightforward to implement and is tractable for large genomes with high sequencing depth. It also enables representation of multiple samples simultaneously to facilitate comparison. However, unlike the string graph, a de Bruijn graph does not retain long range information that is inherent in the read data. For this reason, applications that rely on de Bruijn graphs can produce sub-optimal results given their input.ResultsWe present a novel assembly graph data structure: the Linked de Bruijn Graph (LdBG). Constructed by adding annotations on top of a de Bruijn graph, it stores long range connectivity information through the graph. We show that with error-free data it is possible to losslessly store and recover sequence from a Linked de Bruijn graph. With assembly simulations we demonstrate that the LdBG data structure outperforms both the de Bruijn graph and the String Graph Assembler (SGA). Finally we apply the LdBG to Klebsiella pneumoniae short read data to make large (12 kbp) variant calls, which we validate using PacBio sequencing data, and to characterise the genomic context of drug-resistance genes.AvailabilityLinked de Bruijn Graphs and associated algorithms are implemented as part of McCortex, available under the MIT license at https://github.com/mcvean/[email protected].

Sign in / Sign up

Export Citation Format

Share Document