scholarly journals metaXplor: an interactive viral and microbial metagenomic data manager

GigaScience ◽  
2021 ◽  
Vol 10 (2) ◽  
Author(s):  
Guilhem Sempéré ◽  
Adrien Pétel ◽  
Magsen Abbé ◽  
Pierre Lefeuvre ◽  
Philippe Roumagnac ◽  
...  
Keyword(s):  
Heterogeneous Data ◽  
Metagenomic Data ◽  
Online Data ◽  
Data Repositories ◽  
Ongoing Research ◽  
Efficient Management ◽  
Public Data ◽  
Reference Databases ◽  
Interactive Data ◽  
User Friendly

Abstract Background Efficiently managing large, heterogeneous data in a structured yet flexible way is a challenge to research laboratories working with genomic data. Specifically regarding both shotgun- and metabarcoding-based metagenomics, while online reference databases and user-friendly tools exist for running various types of analyses (e.g., Qiime, Mothur, Megan, IMG/VR, Anvi'o, Qiita, MetaVir), scientists lack comprehensive software for easily building scalable, searchable, online data repositories on which they can rely during their ongoing research. Results metaXplor is a scalable, distributable, fully web-interfaced application for managing, sharing, and exploring metagenomic data. Being based on a flexible NoSQL data model, it has few constraints regarding dataset contents and thus proves useful for handling outputs from both shotgun and metabarcoding techniques. By supporting incremental data feeding and providing means to combine filters on all imported fields, it allows for exhaustive content browsing, as well as rapid narrowing to find specific records. The application also features various interactive data visualization tools, ways to query contents by BLASTing external sequences, and an integrated pipeline to enrich assignments with phylogenetic placements. The project home page provides the URL of a live instance allowing users to test the system on public data. Conclusion metaXplor allows efficient management and exploration of metagenomic data. Its availability as a set of Docker containers, making it easy to deploy on academic servers, on the cloud, or even on personal computers, will facilitate its adoption.

scholarly journals SimpleITK Image-Analysis Notebooks: a Collaborative Environment for Education and Reproducible Research

2017 ◽  
Vol 31 (3) ◽  
pp. 290-303 ◽  
Author(s):  
Ziv Yaniv ◽  
Bradley C. Lowekamp ◽  
Hans J. Johnson ◽  
Richard Beare
Keyword(s):  
Image Analysis ◽  
Software Development ◽  
Web Application ◽  
Reproducible Research ◽  
Online Data ◽  
Development Environment ◽  
Data Repositories ◽  
Testing Framework ◽  
Collaborative Environment ◽  
User Friendly

Abstract Modern scientific endeavors increasingly require team collaborations to construct and interpret complex computational workflows. This work describes an image-analysis environment that supports the use of computational tools that facilitate reproducible research and support scientists with varying levels of software development skills. The Jupyter notebook web application is the basis of an environment that enables flexible, well-documented, and reproducible workflows via literate programming. Image-analysis software development is made accessible to scientists with varying levels of programming experience via the use of the SimpleITK toolkit, a simplified interface to the Insight Segmentation and Registration Toolkit. Additional features of the development environment include user friendly data sharing using online data repositories and a testing framework that facilitates code maintenance. SimpleITK provides a large number of examples illustrating educational and research-oriented image analysis workflows for free download from GitHub under an Apache 2.0 license: github.com/InsightSoftwareConsortium/SimpleITK-Notebooks.

scholarly journals virMine: automated detection of viral sequences from complex metagenomic samples

PeerJ ◽  
2019 ◽  
Vol 7 ◽  
pp. e6695 ◽  
Author(s):  
Andrea Garretto ◽  
Thomas Hatzopoulos ◽  
Catherine Putonti
Keyword(s):  
Software Tool ◽  
Metagenomic Data ◽  
Data Sets ◽  
Data Repositories ◽  
Viral Genomes ◽  
Manual Curation ◽  
Public Data ◽  
Control Assembly ◽  
Signature Sequences ◽  
Viral Sequences

Metagenomics has enabled sequencing of viral communities from a myriad of different environments. Viral metagenomic studies routinely uncover sequences with no recognizable homology to known coding regions or genomes. Nevertheless, complete viral genomes have been constructed directly from complex community metagenomes, often through tedious manual curation. To address this, we developed the software tool virMine to identify viral genomes from raw reads representative of viral or mixed (viral and bacterial) communities. virMine automates sequence read quality control, assembly, and annotation. Researchers can easily refine their search for a specific study system and/or feature(s) of interest. In contrast to other viral genome detection tools that often rely on the recognition of viral signature sequences, virMine is not restricted by the insufficient representation of viral diversity in public data repositories. Rather, viral genomes are identified through an iterative approach, first omitting non-viral sequences. Thus, both relatives of previously characterized viruses and novel species can be detected, including both eukaryotic viruses and bacteriophages. Here we present virMine and its analysis of synthetic communities as well as metagenomic data sets from three distinctly different environments: the gut microbiota, the urinary microbiota, and freshwater viromes. Several new viral genomes were identified and annotated, thus contributing to our understanding of viral genetic diversity in these three environments.

scholarly journals Model, Integrate, Search... Repeat: A Sound Approach to Building Integrated Repositories of Genomic Data

2022 ◽  
pp. 89-99
Author(s):  
Anna Bernasconi
Keyword(s):  
Genomic Data ◽  
Transformation Process ◽  
Data Sources ◽  
Data Repositories ◽  
Genomic Data Integration ◽  
Data Formats ◽  
Public Data ◽  
The Common ◽  
Human Genomic ◽  
User Friendly

AbstractA wealth of public data repositories is available to drive genomics and clinical research. However, there is no agreement among the various data formats and models; in the common practice, data sources are accessed one by one, learning their specific descriptions with tedious efforts. In this context, the integration of genomic data and of their describing metadata becomes—at the same time—an important, difficult, and well-recognized challenge. In this chapter, after overviewing the most important human genomic data players, we propose a conceptual model of metadata and an extended architecture for integrating datasets, retrieved from a variety of data sources, based upon a structured transformation process; we then describe a user-friendly search system providing access to the resulting consolidated repository, enriched by a multi-ontology knowledge base. Inspired by our work on genomic data integration, during the COVID-19 pandemic outbreak we successfully re-applied the previously proposed model-build-search paradigm, building on the analogies among the human and viral genomics domains. The availability of conceptual models, related databases, and search systems for both humans and viruses will provide important opportunities for research, especially if virus data will be connected to its host, provider of genomic and phenotype information.

Clin-mNGS: Automated Pipeline for Pathogen Detection from Clinical Metagenomic Data

2020 ◽  
Vol 15 ◽  
Author(s):  
Akshatha Prasanna ◽  
Vidya Niranjan
Keyword(s):  
Antimicrobial Resistance ◽  
High Performance ◽  
Pathogen Detection ◽  
Bacterial Species ◽  
Workflow Management ◽  
Metagenomic Data ◽  
Antimicrobial Resistance Genes ◽  
Culture Independent ◽  
Automated Pipeline ◽  
User Friendly

Background: Since bacteria are the earliest known organisms, there has been significant interest in their variety and biology, most certainly concerning human health. Recent advances in Metagenomics sequencing (mNGS), a culture-independent sequencing technology have facilitated an accelerated development in clinical microbiology and our understanding of pathogens. Objective: For the implementation of mNGS in routine clinical practice to become feasible, a practical and scalable strategy for the study of mNGS data is essential. This study presents a robust automated pipeline to analyze clinical metagenomic data for pathogen identification and classification. Method: The proposed Clin-mNGS pipeline is an integrated, open-source, scalable, reproducible, and user-friendly framework scripted using the Snakemake workflow management software. The implementation avoids the hassle of manual installation and configuration of the multiple command-line tools and dependencies. The approach directly screens pathogens from clinical raw reads and generates consolidated reports for each sample. Results: The pipeline is demonstrated using publicly available data and is tested on a desktop Linux system and a High-performance cluster. The study compares variability in results from different tools and versions. The versions of the tools are made user modifiable. The pipeline results in quality check, filtered reads, host subtraction, assembled contigs, assembly metrics, relative abundances of bacterial species, antimicrobial resistance genes, plasmid finding, and virulence factors identification. The results obtained from the pipeline are evaluated based on sensitivity and positive predictive value. Conclusion: Clin-mNGS is an automated Snakemake pipeline validated for the analysis of microbial clinical metagenomics reads to perform taxonomic classification and antimicrobial resistance prediction.

Improvements for research data repositories: The case of text spam

2021 ◽  
pp. 016555152199863
Author(s):  
Ismael Vázquez ◽  
María Novo-Lourés ◽  
Reyes Pavón ◽  
Rosalía Laza ◽  
José Ramón Méndez ◽  
...  
Keyword(s):  
Web Application ◽  
Research Data ◽  
Data Sets ◽  
Data Repositories ◽  
Software Applications ◽  
Public Data ◽  
Protection Mechanisms ◽  
Experimental Protocols ◽  
Learning Research ◽  
Processing Steps

Current research has evolved in such a way scientists must not only adequately describe the algorithms they introduce and the results of their application, but also ensure the possibility of reproducing the results and comparing them with those obtained through other approximations. In this context, public data sets (sometimes shared through repositories) are one of the most important elements for the development of experimental protocols and test benches. This study has analysed a significant number of CS/ML ( Computer Science/ Machine Learning) research data repositories and data sets and detected some limitations that hamper their utility. Particularly, we identify and discuss the following demanding functionalities for repositories: (1) building customised data sets for specific research tasks, (2) facilitating the comparison of different techniques using dissimilar pre-processing methods, (3) ensuring the availability of software applications to reproduce the pre-processing steps without using the repository functionalities and (4) providing protection mechanisms for licencing issues and user rights. To show the introduced functionality, we created STRep (Spam Text Repository) web application which implements our recommendations adapted to the field of spam text repositories. In addition, we launched an instance of STRep in the URL https://rdata.4spam.group to facilitate understanding of this study.

The prediction of online time series with concept drift based on dynamic intuitionistic fuzzy cognitive map

2021 ◽  
Vol 25 (4) ◽  
pp. 949-972
Author(s):  
Nannan Zhang ◽  
Xixi Yao ◽  
Chao Luo
Keyword(s):  
Fuzzy Set ◽  
Sequence Data ◽  
Concept Drift ◽  
Intuitionistic Fuzzy Set ◽  
Cognitive Map ◽  
Fuzzy Cognitive Maps ◽  
Fuzzy Cognitive Map ◽  
Online Data ◽  
Intuitionistic Fuzzy ◽  
Public Data

Fuzzy cognitive maps (FCMs) have widely been applied for knowledge representation and reasoning. However, in real life, reasoning is always accompanied with hesitation, which is deriving from the uncertainty and fuzziness. Especially, when processing the online data, since the internal and external interference, the distribution and characteristics of sequence data would be considerably changed along with the passage of time, which further increase the difficulty of modeling. In this article, based on intuitionistic fuzzy set theory, a new dynamic intuitionistic fuzzy cognitive map (DIFCM) scheme is proposed for online data prediction. Combined with a novel detection algorithm of concept drift, the structure of DIFCM can be adaptively updated with the online learning scheme, which can effectively improve the representation of online information by capturing the real-time changes of sequence data. Moreover, in order to tackle with the possible hesitancy in the process of modeling, intuitionistic fuzzy set is applied in the construction of dynamic FCM, where hesitation degree as a quantitative index explicitly expresses the hesitancy. Finally, a series of experiments using public data sets verify the effectiveness of the proposed method.

scholarly journals SHI7 Is a Self-Learning Pipeline for Multipurpose Short-Read DNA Quality Control

mSystems ◽  
2018 ◽  
Vol 3 (3) ◽  
Author(s):  
Gabriel A. Al-Ghalith ◽  
Benjamin Hillmann ◽  
Kaiwei Ang ◽  
Robin Shields-Cutler ◽  
Dan Knights
Keyword(s):  
Quality Control ◽  
Dna Sequences ◽  
Sequence Data ◽  
Background Knowledge ◽  
Sequencing Technology ◽  
Data Set ◽  
Short Read ◽  
Dna Quality ◽  
Public Data ◽  
User Friendly

ABSTRACT Next-generation sequencing technology is of great importance for many biological disciplines; however, due to technical and biological limitations, the short DNA sequences produced by modern sequencers require numerous quality control (QC) measures to reduce errors, remove technical contaminants, or merge paired-end reads together into longer or higher-quality contigs. Many tools for each step exist, but choosing the appropriate methods and usage parameters can be challenging because the parameterization of each step depends on the particularities of the sequencing technology used, the type of samples being analyzed, and the stochasticity of the instrumentation and sample preparation. Furthermore, end users may not know all of the relevant information about how their data were generated, such as the expected overlap for paired-end sequences or type of adaptors used to make informed choices. This increasing complexity and nuance demand a pipeline that combines existing steps together in a user-friendly way and, when possible, learns reasonable quality parameters from the data automatically. We propose a user-friendly quality control pipeline called SHI7 (canonically pronounced “shizen”), which aims to simplify quality control of short-read data for the end user by predicting presence and/or type of common sequencing adaptors, what quality scores to trim, whether the data set is shotgun or amplicon sequencing, whether reads are paired end or single end, and whether pairs are stitchable, including the expected amount of pair overlap. We hope that SHI7 will make it easier for all researchers, expert and novice alike, to follow reasonable practices for short-read data quality control. IMPORTANCE Quality control of high-throughput DNA sequencing data is an important but sometimes laborious task requiring background knowledge of the sequencing protocol used (such as adaptor type, sequencing technology, insert size/stitchability, paired-endedness, etc.). Quality control protocols typically require applying this background knowledge to selecting and executing numerous quality control steps with the appropriate parameters, which is especially difficult when working with public data or data from collaborators who use different protocols. We have created a streamlined quality control pipeline intended to substantially simplify the process of DNA quality control from raw machine output files to actionable sequence data. In contrast to other methods, our proposed pipeline is easy to install and use and attempts to learn the necessary parameters from the data automatically with a single command.

Developing annotation solutions for online Data Driven Learning

ReCALL ◽  
2009 ◽  
Vol 21 (1) ◽  
pp. 55-75 ◽  
Author(s):  
Pascual Pérez-Paredes ◽  
Jose M. Alcaraz-Calero
Keyword(s):  
Corpus Linguistics ◽  
Language Teaching ◽  
Language Education ◽  
Data Driven ◽  
Online Data ◽  
Analysis And Design ◽  
Language Classroom ◽  
Youth Language ◽  
User Friendly

AbstractAlthough annotation is a widely-researched topic in Corpus Linguistics (CL), its potential role in Data Driven Learning (DDL) has not been addressed in depth by Foreign Language Teaching (FLT) practitioners. Furthermore, most of the research in the use of DDL methods pays little attention to annotation in the design and implementation of corpus-based/driven language teaching.In this paper, we set out to examine the process of development of SACODEYL Annotator, an application that seeks to assist SACODEYL system users in annotating XML multilingual corpora. First, we discuss the role of annotation in DDL and the dominating paradigm in general corpus applications. In the context of the language classroom, we argue that it is essential that corpora should be pedagogically motivated (Braun, 2005 and 2007a). Then, we move on to deal with the analysis and design stages of our annotation solution by illustrating its main features. Some of these include a user friendly hierarchical and extensible taxonomy tree to facilitate the learner-oriented annotation of the corpora; real-time graphics representation of the annotated corpus matching the XML TEI-compliant (Text Encoding Initiative) standard, as well as an intuitive management of the different data sections and associated metadata.SACODEYL (System Aided Compilation and Open Distribution of European Youth Language) is an EU funded MINERVA project which aims to develop an ICT-based system for the assisted compilation and open distribution of multimedia European teen talk in the context of language education. This research lays emphasis on the functionalities of the application within the SACODEYL context. However, our paper addresses similarly the needs of potential multimedia language corpus administrators in general on the lookout for powerful annotation assisting software. SACODEYL Annotator is free to use and can be downloaded from our website.

scholarly journals SHAMAN: a user-friendly website for metataxonomic analysis from raw reads to statistical analysis

2020 ◽  
Author(s):  
Stevenn Volant ◽  
Pierre Lechat ◽  
Perrine Woringer ◽  
Laurence Motreff ◽  
Christophe Malabat ◽  
...  
Keyword(s):  
Statistical Analysis ◽  
Data Processing ◽  
Web Application ◽  
Graphical Representation ◽  
Statistical Modelling ◽  
Metagenomic Data ◽  
Sequencing Data ◽  
Microbiome Research ◽  
Interactive Visualizations ◽  
User Friendly

Abstract BackgroundComparing the composition of microbial communities among groups of interest (e.g., patients vs healthy individuals) is a central aspect in microbiome research. It typically involves sequencing, data processing, statistical analysis and graphical representation of the detected signatures. Such an analysis is normally obtained by using a set of different applications that require specific expertise for installation, data processing and in some case, programming skills. ResultsHere, we present SHAMAN, an interactive web application we developed in order to facilitate the use of (i) a bioinformatic workflow for metataxonomic analysis, (ii) a reliable statistical modelling and (iii) to provide among the largest panels of interactive visualizations as compared to the other options that are currently available. SHAMAN is specifically designed for non-expert users who may benefit from using an integrated version of the different analytic steps underlying a proper metagenomic analysis. The application is freely accessible at http://shaman.pasteur.fr/, and may also work as a standalone application with a Docker container (aghozlane/shaman), conda and R. The source code is written in R and is available at https://github.com/aghozlane/shaman. Using two datasets (a mock community sequencing and published 16S rRNA metagenomic data), we illustrate the strengths of SHAMAN in quickly performing a complete metataxonomic analysis. ConclusionsWe aim with SHAMAN to provide the scientific community with a platform that simplifies reproducible quantitative analysis of metagenomic data.

scholarly journals Authentic and Reliable Data in the Service of National Public Data Asset

2021 ◽  
Vol 66 (Special edition 2021/1) ◽  
pp. 52-67
Author(s):  
Gergely Pálmai ◽  
Szabolcs Csernyák ◽  
Zoltán Erdélyi
Keyword(s):  
Artificial Intelligence ◽  
Public Sector ◽  
Public Information ◽  
Reliable Data ◽  
Financial Loss ◽  
National Data ◽  
The Public ◽  
Efficient Management ◽  
Public Data ◽  
Authentic Data

The analysis focused on how efficient management of the national data asset is supported by the Hungarian regulatory framework concerning the use of public information, and whether public data constituting part of the national data asset can be deemed authentic and reliable to support the efforts for the digitalisation and artificial intelligence-based developments of the public sector. The analysis shows why the availability of authentic and reliable data in terms of the national data asset has outstanding significance. In support of this assertion, it presents the different levels of data asset use, the role of using artificial intelligence in the public sector, and the significance, risks and challenges of the authenticity and reliability of public data, from both a data protection and a public finance aspect. Inaccuracy, unreliability of input data predestines the generation of incorrect result products (conclusion, decision), even if the appropriate algorithm is used, which could lead to direct financial loss, for both the citizens and the state. The authors of the analysis therefore suggest that a paradigm shift is necessary in the strategies targeting the efficient use of the public sector’s data, with the necessity to record the fundamental precondition that the national data asset must be based on reliable and authentic data.

Sign in / Sign up

Export Citation Format

Share Document