Shedding Light on Microbial Dark Matter with A Universal Language of Life

AbstractThe majority of microbial genomes have yet to be cultured, and most proteins predicted from microbial genomes or sequenced from the environment cannot be functionally annotated. As a result, current computational approaches to describe microbial systems rely on incomplete reference databases that cannot adequately capture the full functional diversity of the microbial tree of life, limiting our ability to model high-level features of biological sequences. The scientific community needs a means to capture the functionally and evolutionarily relevant features underlying biology, independent of our incomplete reference databases. Such a model can form the basis for transfer learning tasks, enabling downstream applications in environmental microbiology, medicine, and bioengineering. Here we present LookingGlass, a deep learning model capturing a “universal language of life”. LookingGlass encodes contextually-aware, functionally and evolutionarily relevant representations of short DNA reads, distinguishing reads of disparate function, homology, and environmental origin. We demonstrate the ability of LookingGlass to be fine-tuned to perform a range of diverse tasks: to identify novel oxidoreductases, to predict enzyme optimal temperature, and to recognize the reading frames of DNA sequence fragments. LookingGlass is the first contextually-aware, general purpose pre-trained “biological language” representation model for short-read DNA sequences. LookingGlass enables functionally relevant representations of otherwise unknown and unannotated sequences, shedding light on the microbial dark matter that dominates life on Earth.AvailabilityThe pretrained LookingGlass model and the transfer learning-derived models demonstrated in this paper are available in the LookingGlass release v1.01. The open source fastBio Github repository and python package provides classes and functions for training and fine tuning deep learning models with biological data2. Code for reproducing analyses presented in this paper are available as an open source Github repository3.

Download Full-text

Evaluation of Scalability and Degree of Fine-Tuning of Deep Convolutional Neural Networks for COVID-19 Screening on Chest X-ray Images Using Explainable Deep-Learning Algorithm

Journal of Personalized Medicine ◽

10.3390/jpm10040213 ◽

2020 ◽

Vol 10 (4) ◽

pp. 213 ◽

Cited By ~ 1

Author(s):

Ki-Sun Lee ◽

Jae Young Kim ◽

Eun-tae Jeon ◽

Won Suk Choi ◽

Nan Hee Kim ◽

...

Keyword(s):

Deep Learning ◽

Learning Strategies ◽

Transfer Learning ◽

Learning Algorithm ◽

Fine Tuning ◽

Deep Convolutional Neural Networks ◽

X Ray ◽

Backbone Network ◽

Deep Learning Algorithm ◽

Chest X Ray

According to recent studies, patients with COVID-19 have different feature characteristics on chest X-ray (CXR) than those with other lung diseases. This study aimed at evaluating the layer depths and degree of fine-tuning on transfer learning with a deep convolutional neural network (CNN)-based COVID-19 screening in CXR to identify efficient transfer learning strategies. The CXR images used in this study were collected from publicly available repositories, and the collected images were classified into three classes: COVID-19, pneumonia, and normal. To evaluate the effect of layer depths of the same CNN architecture, CNNs called VGG-16 and VGG-19 were used as backbone networks. Then, each backbone network was trained with different degrees of fine-tuning and comparatively evaluated. The experimental results showed the highest AUC value to be 0.950 concerning COVID-19 classification in the experimental group of a fine-tuned with only 2/5 blocks of the VGG16 backbone network. In conclusion, in the classification of medical images with a limited number of data, a deeper layer depth may not guarantee better results. In addition, even if the same pre-trained CNN architecture is used, an appropriate degree of fine-tuning can help to build an efficient deep learning model.

Download Full-text

Coal-Rock Recognition in Top Coal Caving Using Bimodal Deep Learning and Hilbert-Huang Transform

Shock and Vibration ◽

10.1155/2017/3809525 ◽

2017 ◽

Vol 2017 ◽

pp. 1-13 ◽

Cited By ~ 8

Author(s):

Guoxin Zhang ◽

Zengcai Wang ◽

Lei Zhao ◽

Yazhou Qi ◽

Jinshan Wang

Keyword(s):

Deep Learning ◽

Transfer Learning ◽

Acoustic Waves ◽

Sound Pressure ◽

Mechanical Vibration ◽

Training Sample ◽

Fine Tuning ◽

Learning Method ◽

Hilbert Huang Transform ◽

Coal Rock

This study employs the mechanical vibration and acoustic waves of a hydraulic support tail beam for an accurate and fast coal-rock recognition. The study proposes a diagnosis method based on bimodal deep learning and Hilbert-Huang transform. The bimodal deep neural networks (DNN) adopt bimodal learning and transfer learning. The bimodal learning method attempts to learn joint representation by considering acceleration and sound pressure modalities, which both contribute to coal-rock recognition. The transfer learning method solves the problem regarding DNN, in which a large number of labeled training samples are necessary to optimize the parameters while the labeled training sample is limited. A suitable installation location for sensors is determined in recognizing coal-rock. The extraction features of acceleration and sound pressure signals are combined and effective combination features are selected. Bimodal DNN consists of two deep belief networks (DBN), each DBN model is trained with related samples, and the parameters of the pretrained DBNs are transferred to the final recognition model. Then the parameters of the proposed model are continuously optimized by pretraining and fine-tuning. Finally, the comparison of experimental results demonstrates the superiority of the proposed method in terms of recognition accuracy.

Download Full-text

Deep Learning Approaches to Automated Video Classification of Upper Limb Tension Test

Healthcare ◽

10.3390/healthcare9111579 ◽

2021 ◽

Vol 9 (11) ◽

pp. 1579

Author(s):

Wansuk Choi ◽

Seoyoon Heo

Keyword(s):

Deep Learning ◽

Transfer Learning ◽

High Performance ◽

Model Fit ◽

Fine Tuning ◽

Learning Approaches ◽

Video Classification ◽

Video Clips ◽

Performance Computing

The purpose of this study was to classify ULTT videos through transfer learning with pre-trained deep learning models and compare the performance of the models. We conducted transfer learning by combining a pre-trained convolution neural network (CNN) model into a Python-produced deep learning process. Videos were processed on YouTube and 103,116 frames converted from video clips were analyzed. In the modeling implementation, the process of importing the required modules, performing the necessary data preprocessing for training, defining the model, compiling, model creation, and model fit were applied in sequence. Comparative models were Xception, InceptionV3, DenseNet201, NASNetMobile, DenseNet121, VGG16, VGG19, and ResNet101, and fine tuning was performed. They were trained in a high-performance computing environment, and validation and loss were measured as comparative indicators of performance. Relatively low validation loss and high validation accuracy were obtained from Xception, InceptionV3, and DenseNet201 models, which is evaluated as an excellent model compared with other models. On the other hand, from VGG16, VGG19, and ResNet101, relatively high validation loss and low validation accuracy were obtained compared with other models. There was a narrow range of difference between the validation accuracy and the validation loss of the Xception, InceptionV3, and DensNet201 models. This study suggests that training applied with transfer learning can classify ULTT videos, and that there is a difference in performance between models.

Download Full-text

Deeplasmid: Deep learning accurately separates plasmids from bacterial chromosomes

10.1101/2021.03.11.434936 ◽

2021 ◽

Author(s):

William B Andreopoulos ◽

Alexander M Geller ◽

Miriam Lucke ◽

Jan Balewski ◽

Alicia Clum ◽

...

Keyword(s):

Deep Learning ◽

High Reliability ◽

Biological Data ◽

Yersinia Ruckeri ◽

Microbial Genomes ◽

Sequencing Platform ◽

Assembly Algorithm ◽

Antimicrobial Resistance Genes ◽

Bacterial Chromosomes ◽

Long Read

AbstractPlasmids are mobile genetic elements that play a key role in microbial ecology and evolution by mediating horizontal transfer of important genes, such as antimicrobial resistance genes. Many microbial genomes have been sequenced by short read sequencers and have resulted in a mix of contigs that derive from plasmids or chromosomes. New tools that accurately identify plasmids are needed to elucidate new plasmid-borne genes of high biological importance. We have developed Deeplasmid, a deep learning tool for distinguishing plasmids from bacterial chromosomes based on the DNA sequence and its encoded biological data. It requires as input only assembled sequences generated by any sequencing platform and assembly algorithm and its runtime scales linearly with the number of assembled sequences. Deeplasmid achieves an AUC-ROC of over 93%, and it was much more precise than the state-of-the-art methods. Finally, as a proof of concept, we used Deeplasmid to predict new plasmids in the fish pathogen Yersinia ruckeri ATCC 29473 that has no annotated plasmids. Deeplasmid predicted with high reliability that a long assembled contig is part of a plasmid. Using long read sequencing we indeed validated the existence of a 102 Kbp long plasmid, demonstrating Deeplasmid’s ability to detect novel plasmids.AvailabilityThe software is available with a BSD license: deeplasmid.sourceforge.io. A Docker container is available on DockerHub under: billandreo/[email protected]@mail.huji.ac.il

Download Full-text

Deep learning prediction of mild cognitive impairment conversion to Alzheimer’s disease at 3 years after diagnosis using longitudinal and whole-brain 3D MRI

PeerJ Computer Science ◽

10.7717/peerj-cs.560 ◽

2021 ◽

Vol 7 ◽

pp. e560

Author(s):

Ethan Ocasio ◽

Tim Q. Duong

Keyword(s):

Neural Network ◽

Deep Learning ◽

Cognitive Impairment ◽

Mild Cognitive Impairment ◽

Longitudinal Data ◽

Convolutional Neural Network ◽

Transfer Learning ◽

Fine Tuning ◽

3D Mri ◽

Whole Brain

Background While there is no cure for Alzheimer’s disease (AD), early diagnosis and accurate prognosis of AD may enable or encourage lifestyle changes, neurocognitive enrichment, and interventions to slow the rate of cognitive decline. The goal of our study was to develop and evaluate a novel deep learning algorithm to predict mild cognitive impairment (MCI) to AD conversion at three years after diagnosis using longitudinal and whole-brain 3D MRI. Methods This retrospective study consisted of 320 normal cognition (NC), 554 MCI, and 237 AD patients. Longitudinal data include T1-weighted 3D MRI obtained at initial presentation with diagnosis of MCI and at 12-month follow up. Whole-brain 3D MRI volumes were used without a priori segmentation of regional structural volumes or cortical thicknesses. MRIs of the AD and NC cohort were used to train a deep learning classification model to obtain weights to be applied via transfer learning for prediction of MCI patient conversion to AD at three years post-diagnosis. Two (zero-shot and fine tuning) transfer learning methods were evaluated. Three different convolutional neural network (CNN) architectures (sequential, residual bottleneck, and wide residual) were compared. Data were split into 75% and 25% for training and testing, respectively, with 4-fold cross validation. Prediction accuracy was evaluated using balanced accuracy. Heatmaps were generated. Results The sequential convolutional approach yielded slightly better performance than the residual-based architecture, the zero-shot transfer learning approach yielded better performance than fine tuning, and CNN using longitudinal data performed better than CNN using a single timepoint MRI in predicting MCI conversion to AD. The best CNN model for predicting MCI conversion to AD at three years after diagnosis yielded a balanced accuracy of 0.793. Heatmaps of the prediction model showed regions most relevant to the network including the lateral ventricles, periventricular white matter and cortical gray matter. Conclusions This is the first convolutional neural network model using longitudinal and whole-brain 3D MRIs without extracting regional brain volumes or cortical thicknesses to predict future MCI to AD conversion at 3 years after diagnosis. This approach could lead to early prediction of patients who are likely to progress to AD and thus may lead to better management of the disease.

Download Full-text

Transfer Learning for Inference of Metastatic Origin from Whole Slide Histology

10.1101/2021.04.21.440864 ◽

2021 ◽

Author(s):

Geoffrey F. Schau ◽

Hassan Ghani ◽

Erik A. Burlingame ◽

Guillaume Thibault ◽

Joe W. Gray ◽

...

Keyword(s):

Deep Learning ◽

Transfer Learning ◽

Primary Tumor ◽

Large Scale ◽

Metastatic Cancer ◽

Clinical Diagnostics ◽

Training Data ◽

Fine Tuning ◽

Learning Approach ◽

Whole Slide Images

AbstractAccurate diagnosis of metastatic cancer is essential for prescribing optimal control strategies to halt further spread of metastasizing disease. While pathological inspection aided by immunohistochemistry staining provides a valuable gold standard for clinical diagnostics, deep learning methods have emerged as powerful tools for identifying clinically relevant features of whole slide histology relevant to a tumor’s metastatic origin. Although deep learning models require significant training data to learn effectively, transfer learning paradigms provide mechanisms to circumvent limited training data by first training a model on related data prior to fine-tuning on smaller data sets of interest. In this work we propose a transfer learning approach that trains a convolutional neural network to infer the metastatic origin of tumor tissue from whole slide images of hematoxylin and eosin (H&E) stained tissue sections and illustrate the advantages of pre-training network on whole slide images of primary tumor morphology. We further characterize statistical dissimilarity between primary and metastatic tumors of various indications on patch-level images to highlight limitations of our indication-specific transfer learning approach. Using a primary-to-metastatic transfer learning approach, we achieved mean class-specific areas under receiver operator characteristics curve (AUROC) of 0.779, which outperformed comparable models trained on only images of primary tumor (mean AUROC of 0.691) or trained on only images of metastatic tumor (mean AUROC of 0.675), supporting the use of large scale primary tumor imaging data in developing computer vision models to characterize metastatic origin of tumor lesions.

Download Full-text

Biologically-relevant transfer learning improves transcription factor binding prediction

10.1101/2020.12.21.423873 ◽

2020 ◽

Author(s):

Gherman Novakovsky ◽

Manu Saraswat ◽

Oriol Fornes ◽

Sara Mostafavi ◽

Wyeth W. Wasserman

Keyword(s):

Transcription Factor ◽

Deep Learning ◽

Transfer Learning ◽

Model Performance ◽

Fine Tuning ◽

Binding Prediction ◽

Single Task ◽

Biologically Relevant ◽

Powerful Technique ◽

Task Model

AbstractBackgroundDeep learning has proven to be a powerful technique for transcription factor (TF) binding prediction, but requires large training datasets. Transfer learning can reduce the amount of data required for deep learning, while improving overall model performance, compared to training a separate model for each new task.ResultsWe assess a transfer learning strategy for TF binding prediction consisting of a pre-training step, wherein we train a multi-task model with multiple TFs, and a fine-tuning step, wherein we initialize single-task models for individual TFs with the weights learned by the multi-task model, after which the single-task models are trained at a lower learning rate. We corroborate that transfer learning improves model performance, especially if in the pre-training step the multi-task model is trained with biologically-relevant TFs. We show the effectiveness of transfer learning for TFs with ∼500 ChIP-seq peak regions. Using model interpretation techniques, we demonstrate that the features learned in the pre-training step are refined in the fine-tuning step to resemble the binding motif of the target TF (i.e. the recipient of transfer learning in the fine-tuning step). Moreover, pre-training with biologically-relevant TFs allows single-task models in the fine-tuning step to learn features other than the motif of the target TF.ConclusionsOur results confirm that transfer learning is a powerful technique for TF binding prediction.

Download Full-text

TISSUE CLASSIFICATION FOR COLORECTAL CANCER UTILIZING TECHNIQUES OF DEEP LEARNING AND MACHINE LEARNING

Biomedical Engineering Applications Basis and Communications ◽

10.4015/s1016237221500228 ◽

2021 ◽

pp. 2150022

Author(s):

Kasikrit Damkliang ◽

Thakerng Wongsirichot ◽

Paramee Thongsuksai

Keyword(s):

Colorectal Cancer ◽

Machine Learning ◽

Deep Learning ◽

Transfer Learning ◽

Texture Classification ◽

False Positive Rate ◽

Fine Tuning ◽

Machine Learning Techniques ◽

Cancer Tissues

Since the introduction of image pattern recognition and computer vision processing, the classification of cancer tissues has been a challenge at pixel-level, slide-level, and patient-level. Conventional machine learning techniques have given way to Deep Learning (DL), a contemporary, state-of-the-art approach to texture classification and localization of cancer tissues. Colorectal Cancer (CRC) is the third ranked cause of death from cancer worldwide. This paper proposes image-level texture classification of a CRC dataset by deep convolutional neural networks (CNN). Simple DL techniques consisting of transfer learning and fine-tuning were exploited. VGG-16, a Keras pre-trained model with initial weights by ImageNet, was applied. The transfer learning architecture and methods responding to VGG-16 are proposed. The training, validation, and testing sets included 5000 images of 150 × 150 pixels. The application set for detection and localization contained 10 large original images of 5000 × 5000 pixels. The model achieved F1-score and accuracy of 0.96 and 0.99, respectively, and produced a false positive rate of 0.01. AUC-based evaluation was also measured. The model classified ten large previously unseen images from the application set represented in false color maps. The reported results show the satisfactory performance of the model. The simplicity of the architecture, configuration, and implementation also contributes to the outcome this work.

Download Full-text

Small aircraft detection using deep learning

Aircraft Engineering and Aerospace Technology ◽

10.1108/aeat-11-2020-0259 ◽

2021 ◽

Vol ahead-of-print (ahead-of-print) ◽

Author(s):

Emre Kiyak ◽

Gulay Unal

Keyword(s):

Neural Network ◽

Neural Networks ◽

Deep Learning ◽

Convolutional Neural Network ◽

Transfer Learning ◽

Convolutional Neural Networks ◽

Fine Tuning ◽

Deep Convolutional Neural Networks ◽

Content Type ◽

Training Time

Purpose The paper aims to address the tracking algorithm based on deep learning and four deep learning tracking models developed. They compared with each other to prevent collision and to obtain target tracking in autonomous aircraft. Design/methodology/approach First, to follow the visual target, the detection methods were used and then the tracking methods were examined. Here, four models (deep convolutional neural networks (DCNN), deep convolutional neural networks with fine-tuning (DCNNFN), transfer learning with deep convolutional neural network (TLDCNN) and fine-tuning deep convolutional neural network with transfer learning (FNDCNNTL)) were developed. Findings The training time of DCNN took 9 min 33 s, while the accuracy percentage was calculated as 84%. In DCNNFN, the training time of the network was calculated as 4 min 26 s and the accuracy percentage was 91%. The training of TLDCNN) took 34 min and 49 s and the accuracy percentage was calculated as 95%. With FNDCNNTL, the training time of the network was calculated as 34 min 33 s and the accuracy percentage was nearly 100%. Originality/value Compared to the results in the literature ranging from 89.4% to 95.6%, using FNDCNNTL, better results were found in the paper.

Download Full-text

Internet of Medical Things: An Effective and Fully Automatic IoT Approach Using Deep Learning and Fine-Tuning to Lung CT Segmentation

Sensors ◽

10.3390/s20236711 ◽

2020 ◽

Vol 20 (23) ◽

pp. 6711

Author(s):

Luís Fabrício de Freitas Souza ◽

Iágson Carlos Lima Silva ◽

Adriell Gomes Marques ◽

Francisco Hércules dos S. Silva ◽

Virgínia Xavier Nunes ◽

...

Keyword(s):

Deep Learning ◽

Transfer Learning ◽

Application Programming Interface ◽

Fine Tuning ◽

Chronic Obstructive ◽

Obstructive Pulmonary Disease ◽

Lung Segmentation ◽

Fully Automatic ◽

Internet Of Medical Things ◽

Ct Segmentation

Several pathologies have a direct impact on society, causing public health problems. Pulmonary diseases such as Chronic obstructive pulmonary disease (COPD) are already the third leading cause of death in the world, leaving tuberculosis at ninth with 1.7 million deaths and over 10.4 million new occurrences. The detection of lung regions in images is a classic medical challenge. Studies show that computational methods contribute significantly to the medical diagnosis of lung pathologies by Computerized Tomography (CT), as well as through Internet of Things (IoT) methods based in the context on the health of things. The present work proposes a new model based on IoT for classification and segmentation of pulmonary CT images, applying the transfer learning technique in deep learning methods combined with Parzen’s probability density. The proposed model uses an Application Programming Interface (API) based on the Internet of Medical Things to classify lung images. The approach was very effective, with results above 98% accuracy for classification in pulmonary images. Then the model proceeds to the lung segmentation stage using the Mask R-CNN network to create a pulmonary map and use fine-tuning to find the pulmonary borders on the CT image. The experiment was a success, the proposed method performed better than other works in the literature, reaching high segmentation metrics values such as accuracy of 98.34%. Besides reaching 5.43 s in segmentation time and overcoming other transfer learning models, our methodology stands out among the others because it is fully automatic. The proposed approach has simplified the segmentation process using transfer learning. It has introduced a faster and more effective method for better-performing lung segmentation, making our model fully automatic and robust.

Download Full-text