scholarly journals Learning, Visualizing and Exploring 16S rRNA Structure Using an Attention-based Deep Neural Network

2020 ◽  
Author(s):  
Zhengqiao Zhao ◽  
Stephen Woloszynek ◽  
Felix Agbavor ◽  
Joshua Chang Mell ◽  
Bahrad A. Sokhansanj ◽  
...  
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Microbial Community ◽  
Language Processing ◽  
Recurrent Neural Networks ◽  
Network Architecture ◽  
Community Sample ◽  
Biological Research ◽  
Marker Genes ◽  
Learning Approach

AbstractRecurrent neural networks (RNNs) with memory (e.g. LSTMs) and attention mechanisms are widely used in natural language processing because they can capture short and long term sequential information for diverse tasks. We propose an integrated deep learning model for microbial DNA sequence data, which exploits convolutional networks, recurrent neural networks, and attention mechanisms to perform sample-associated attribute prediction—phenotype prediction—and extract interesting features, such as informative taxa and predictive k-mer context. In this paper, we develop this novel deep learning approach and evaluate its application to amplicon sequences. We focus on typically short DNA reads of 16s ribosomal RNA (rRNA) marker genes, which identify the heterogeneity of a microbial community sample. Our deep learning approach enables sample-level attribute and taxonomic prediction, with the aim of aiding biological research and supporting medical diagnosis. We demonstrate that our implementation of a novel attention-based deep network architecture, Read2Pheno, achieves read-level phenotypic prediction and, in turn, that aggregating read-level information can robustly predict microbial community properties, host phenotype, and taxonomic classification, with performance comparable to conventional approaches. Most importantly, as a further result of the training process, the network architecture will encode sequences (reads) into dense, meaningful representations: learned embedded vectors output on the intermediate layer of the network model, which can provide biological insight when visualized. Finally, we demonstrate that a model with an attention layer can automatically identify informative regions in sequences/reads which are particularly informative for classification tasks. An implementation of the attention-based deep learning network is available at https://github.com/EESI/sequence_attention.

scholarly journals Learning, visualizing and exploring 16S rRNA structure using an attention-based deep neural network

2021 ◽  
Vol 17 (9) ◽  
pp. e1009345
Author(s):  
Zhengqiao Zhao ◽  
Stephen Woloszynek ◽  
Felix Agbavor ◽  
Joshua Chang Mell ◽  
Bahrad A. Sokhansanj ◽  
...  
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Microbial Community ◽  
Language Processing ◽  
Recurrent Neural Networks ◽  
Network Architecture ◽  
Sequence Data ◽  
Community Sample ◽  
Marker Genes ◽  
Link Type

Recurrent neural networks with memory and attention mechanisms are widely used in natural language processing because they can capture short and long term sequential information for diverse tasks. We propose an integrated deep learning model for microbial DNA sequence data, which exploits convolutional neural networks, recurrent neural networks, and attention mechanisms to predict taxonomic classifications and sample-associated attributes, such as the relationship between the microbiome and host phenotype, on the read/sequence level. In this paper, we develop this novel deep learning approach and evaluate its application to amplicon sequences. We apply our approach to short DNA reads and full sequences of 16S ribosomal RNA (rRNA) marker genes, which identify the heterogeneity of a microbial community sample. We demonstrate that our implementation of a novel attention-based deep network architecture, Read2Pheno, achieves read-level phenotypic prediction. Training Read2Pheno models will encode sequences (reads) into dense, meaningful representations: learned embedded vectors output from the intermediate layer of the network model, which can provide biological insight when visualized. The attention layer of Read2Pheno models can also automatically identify nucleotide regions in reads/sequences which are particularly informative for classification. As such, this novel approach can avoid pre/post-processing and manual interpretation required with conventional approaches to microbiome sequence classification. We further show, as proof-of-concept, that aggregating read-level information can robustly predict microbial community properties, host phenotype, and taxonomic classification, with performance at least comparable to conventional approaches. An implementation of the attention-based deep learning network is available at https://github.com/EESI/sequence_attention (a python package) and https://github.com/EESI/seq2att (a command line tool).

scholarly journals Ultrasound Medical Images Classification Based on Deep Learning Algorithms: A Review

2021 ◽  
pp. 29-42
Author(s):  
admin admin ◽  
◽  
◽  
Adnan Mohsin Abdulazeez
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Recurrent Neural Networks ◽  
Ultrasound Image ◽  
Smart Devices ◽  
Learning Approach ◽  
Ultrasound Images ◽  
Medical Field ◽  
Modern Methods

With the development of technology and smart devices in the medical field, the computer system has become an essential part of this development to learn devices in the medical field. One of the learning methods is deep learning (DL), which is a branch of machine learning (ML). The deep learning approach has been used in this field because it is one of the modern methods of obtaining accurate results through its algorithms, and among these algorithms that are used in this field are convolutional neural networks (CNN) and recurrent neural networks (RNN). In this paper we reviewed what have researchers have done in their researches to solve fetal problems, then summarize and carefully discuss the applications in different tasks identified for segmentation and classification of ultrasound images. Finally, this study discussed the potential challenges and directions for applying deep learning in ultrasound image analysis.

scholarly journals Deep learning in clinical natural language processing: a methodical review

2019 ◽  
Vol 27 (3) ◽  
pp. 457-470 ◽  
Author(s):  
Stephen Wu ◽  
Kirk Roberts ◽  
Surabhi Datta ◽  
Jingcheng Du ◽  
Zongcheng Ji ◽  
...  
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Natural Language Processing ◽  
Natural Language ◽  
Computational Linguistics ◽  
Language Processing ◽  
Recurrent Neural Networks ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Named Entity

Abstract Objective This article methodically reviews the literature on deep learning (DL) for natural language processing (NLP) in the clinical domain, providing quantitative analysis to answer 3 research questions concerning methods, scope, and context of current research. Materials and Methods We searched MEDLINE, EMBASE, Scopus, the Association for Computing Machinery Digital Library, and the Association for Computational Linguistics Anthology for articles using DL-based approaches to NLP problems in electronic health records. After screening 1,737 articles, we collected data on 25 variables across 212 papers. Results DL in clinical NLP publications more than doubled each year, through 2018. Recurrent neural networks (60.8%) and word2vec embeddings (74.1%) were the most popular methods; the information extraction tasks of text classification, named entity recognition, and relation extraction were dominant (89.2%). However, there was a “long tail” of other methods and specific tasks. Most contributions were methodological variants or applications, but 20.8% were new methods of some kind. The earliest adopters were in the NLP community, but the medical informatics community was the most prolific. Discussion Our analysis shows growing acceptance of deep learning as a baseline for NLP research, and of DL-based NLP in the medical community. A number of common associations were substantiated (eg, the preference of recurrent neural networks for sequence-labeling named entity recognition), while others were surprisingly nuanced (eg, the scarcity of French language clinical NLP with deep learning). Conclusion Deep learning has not yet fully penetrated clinical NLP and is growing rapidly. This review highlighted both the popular and unique trends in this active field.

scholarly journals Temporal Convolutional Neural Network for the Classification of Satellite Image Time Series

2019 ◽  
Vol 11 (5) ◽  
pp. 523 ◽  
Author(s):  
Charlotte Pelletier ◽  
Geoffrey Webb ◽  
François Petitjean
Keyword(s):  
Neural Networks ◽  
Time Series ◽  
Deep Learning ◽  
Land Cover ◽  
Network Architecture ◽  
Satellite Image ◽  
Classification Systems ◽  
Batch Size ◽  
Learning Approach ◽  
Land Cover Maps

Latest remote sensing sensors are capable of acquiring high spatial and spectral Satellite Image Time Series (SITS) of the world. These image series are a key component of classification systems that aim at obtaining up-to-date and accurate land cover maps of the Earth’s surfaces. More specifically, current SITS combine high temporal, spectral and spatial resolutions, which makes it possible to closely monitor vegetation dynamics. Although traditional classification algorithms, such as Random Forest (RF), have been successfully applied to create land cover maps from SITS, these algorithms do not make the most of the temporal domain. This paper proposes a comprehensive study of Temporal Convolutional Neural Networks (TempCNNs), a deep learning approach which applies convolutions in the temporal dimension in order to automatically learn temporal (and spectral) features. The goal of this paper is to quantitatively and qualitatively evaluate the contribution of TempCNNs for SITS classification, as compared to RF and Recurrent Neural Networks (RNNs) —a standard deep learning approach that is particularly suited to temporal data. We carry out experiments on Formosat-2 scene with 46 images and one million labelled time series. The experimental results show that TempCNNs are more accurate than the current state of the art for SITS classification. We provide some general guidelines on the network architecture, common regularization mechanisms, and hyper-parameter values such as batch size; we also draw out some differences with standard results in computer vision (e.g., about pooling layers). Finally, we assess the visual quality of the land cover maps produced by TempCNNs.

scholarly journals A Deep Learning Approach for Intrusion Detection Using Recurrent Neural Networks

IEEE Access ◽  
2017 ◽  
Vol 5 ◽  
pp. 21954-21961 ◽  
Author(s):  
Chuanlong Yin ◽  
Yuefei Zhu ◽  
Jinlong Fei ◽  
Xinzheng He
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Intrusion Detection ◽  
Recurrent Neural Networks ◽  
Learning Approach

Image Captioning Using Deep Learning

2021 ◽  
pp. 381-395
Author(s):  
Bhavana D. ◽  
K. Chaitanya Krishna ◽  
Tejaswini K. ◽  
N. Venkata Vikas ◽  
A. N. V. Sahithya
Keyword(s):  
Neural Networks ◽  
Computer Vision ◽  
Feature Extraction ◽  
Deep Learning ◽  
Natural Language Processing ◽  
Language Processing ◽  
Recurrent Neural Networks ◽  
Input Image ◽  
The Other ◽  
Image Caption

The task of image caption generator is mainly about extracting the features and ongoings of an image and generating human-readable captions that translate the features of the objects in the image. The contents of an image can be described by having knowledge about natural language processing and computer vision. The features can be extracted using convolution neural networks which makes use of transfer learning to implement the exception model. It stands for extreme inception, which has a feature extraction base with 36 convolution layers. This shows accurate results when compared with the other CNNs. Recurrent neural networks are used for describing the image and to generate accurate sentences. The feature vector that is extracted by using the CNN is fed to the LSTM. The Flicker 8k dataset is used to train the network in which the data is labeled properly. The model will be able to generate accurate captions that nearly describe the activities carried in the image when an input image is given to it. Further, the authors use the BLEU scores to validate the model.

Levenshtein Augmentation Improves Performance of SMILES Based Deep-Learning Synthesis Prediction

2020 ◽  
Author(s):  
Dean Sumner ◽  
Jiazhen He ◽  
Amol Thakkar ◽  
Ola Engkvist ◽  
Esben Jannik Bjerrum
Keyword(s):  
Neural Networks ◽  
Pattern Recognition ◽  
Deep Learning ◽  
Recurrent Neural Networks ◽  
Data Augmentation ◽  
State Of The Art ◽  
Sequence Similarity ◽  
Learning Models ◽  
Underlying Network

<p>SMILES randomization, a form of data augmentation, has previously been shown to increase the performance of deep learning models compared to non-augmented baselines. Here, we propose a novel data augmentation method we call “Levenshtein augmentation” which considers local SMILES sub-sequence similarity between reactants and their respective products when creating training pairs. The performance of Levenshtein augmentation was tested using two state of the art models - transformer and sequence-to-sequence based recurrent neural networks with attention. Levenshtein augmentation demonstrated an increase performance over non-augmented, and conventionally SMILES randomization augmented data when used for training of baseline models. Furthermore, Levenshtein augmentation seemingly results in what we define as <i>attentional gain </i>– an enhancement in the pattern recognition capabilities of the underlying network to molecular motifs.</p>

scholarly journals Classification of Skin Disease Using Deep Learning Neural Networks with MobileNet V2 and LSTM

Sensors ◽  
2021 ◽  
Vol 21 (8) ◽  
pp. 2852
Author(s):  
Parvathaneni Naga Srinivasu ◽  
Jalluri Gnana SivaSai ◽  
Muhammad Fazal Ijaz ◽  
Akash Kumar Bhoi ◽  
Wonjoon Kim ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Skin Disease ◽  
Network Architecture ◽  
Large Scale ◽  
Short Term Memory ◽  
Convolutional Networks ◽  
Occurrence Matrix

Deep learning models are efficient in learning the features that assist in understanding complex patterns precisely. This study proposed a computerized process of classifying skin disease through deep learning based MobileNet V2 and Long Short Term Memory (LSTM). The MobileNet V2 model proved to be efficient with a better accuracy that can work on lightweight computational devices. The proposed model is efficient in maintaining stateful information for precise predictions. A grey-level co-occurrence matrix is used for assessing the progress of diseased growth. The performance has been compared against other state-of-the-art models such as Fine-Tuned Neural Networks (FTNN), Convolutional Neural Network (CNN), Very Deep Convolutional Networks for Large-Scale Image Recognition developed by Visual Geometry Group (VGG), and convolutional neural network architecture that expanded with few changes. The HAM10000 dataset is used and the proposed method has outperformed other methods with more than 85% accuracy. Its robustness in recognizing the affected region much faster with almost 2× lesser computations than the conventional MobileNet model results in minimal computational efforts. Furthermore, a mobile application is designed for instant and proper action. It helps the patient and dermatologists identify the type of disease from the affected region’s image at the initial stage of the skin disease. These findings suggest that the proposed system can help general practitioners efficiently and effectively diagnose skin conditions, thereby reducing further complications and morbidity.

Sign in / Sign up

Export Citation Format

Share Document