scholarly journals Epistatic Net allows the sparse spectral regularization of deep neural networks for inferring fitness functions

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Amirali Aghazadeh ◽  
Hunter Nisonoff ◽  
Orhan Ocal ◽  
David H. Brookes ◽  
Yijie Huang ◽  
...  
Keyword(s):  
Neural Networks ◽  
Coding Theory ◽  
Prediction Accuracy ◽  
Deep Neural Networks ◽  
Epistatic Interactions ◽  
Fitness Functions ◽  
Spectral Regularization ◽  
Competing Models ◽  
Combinatorial Mutagenesis ◽  
Recovery Algorithms

AbstractDespite recent advances in high-throughput combinatorial mutagenesis assays, the number of labeled sequences available to predict molecular functions has remained small for the vastness of the sequence space combined with the ruggedness of many fitness functions. While deep neural networks (DNNs) can capture high-order epistatic interactions among the mutational sites, they tend to overfit to the small number of labeled sequences available for training. Here, we developed Epistatic Net (EN), a method for spectral regularization of DNNs that exploits evidence that epistatic interactions in many fitness functions are sparse. We built a scalable extension of EN, usable for larger sequences, which enables spectral regularization using fast sparse recovery algorithms informed by coding theory. Results on several biological landscapes show that EN consistently improves the prediction accuracy of DNNs and enables them to outperform competing models which assume other priors. EN estimates the higher-order epistatic interactions of DNNs trained on massive sequence spaces-a computational problem that otherwise takes years to solve.

scholarly journals Sparse Epistatic Regularization of Deep Neural Networks for Inferring Fitness Functions

2020 ◽  
Author(s):  
Amirali Aghazadeh ◽  
Hunter Nisonoff ◽  
Orhan Ocal ◽  
Yijie Huang ◽  
O. Ozan Koyluoglu ◽  
...  
Keyword(s):  
Neural Networks ◽  
Sequence Space ◽  
Prediction Accuracy ◽  
Deep Neural Networks ◽  
Spectral Representation ◽  
Epistatic Interactions ◽  
Inductive Bias ◽  
Fitness Functions ◽  
Code Structure ◽  
Spectral Transform

AbstractDespite recent advances in high-throughput combinatorial mutagenesis assays, the number of labeled sequences available to predict molecular functions has remained small for the vastness of the sequence space combined with the ruggedness of many fitness functions. Expressive models in machine learning (ML), such as deep neural networks (DNNs), can model the nonlinearities in rugged fitness functions, which manifest as high-order epistatic interactions among the mutational sites. However, in the absence of an inductive bias, DNNs overfit to the small number of labeled sequences available for training. Herein, we exploit the recent biological evidence that epistatic interactions in many fitness functions are sparse; this knowledge can be used as an inductive bias to regularize DNNs. We have developed a method for sparse epistatic regularization of DNNs, called the epistatic net (EN), which constrains the number of non-zero coefficients in the spectral representation of DNNs. For larger sequences, where finding the spectral transform becomes computationally intractable, we have developed a scalable extension of EN, which subsamples the combinatorial sequence space uniformly inducing a sparse-graph-code structure, and regularizes DNNs using the resulting greedy optimization method. Results on several biological landscapes, from bacterial to protein fitness functions, showed that EN consistently improves the prediction accuracy of DNNs and enables them to outperform competing models which assume other forms of inductive biases. EN estimates all the higher-order epistatic interactions of DNNs trained on massive sequence spaces—a computational problem that takes years to solve without leveraging the epistatic sparsity in the fitness functions.Significance StatementPredicting the properties of small molecules (such as proteins) from their sequence is an important problem in computational biology. The main challenge is in developing a model that can capture the non-linearities in the function mapping the sequence to the property of interest (e.g., fluorescence) using the limited number of available labeled sequences from biological assays. In this paper, we identify a biologically-plausible sparsity prior and develop a method to infuse this prior into the structure of deep neural networks (DNNs) by regularizing their spectral representation. We demonstrate that our method significantly improves the prediction accuracy of DNNs and enables an interpretable explanation of DNNs—a task that is computationally intractable without leveraging the hidden structure in biological functions.

scholarly journals MOLI: multi-omics late integration with deep neural networks for drug response prediction

2019 ◽  
Vol 35 (14) ◽  
pp. i501-i509 ◽  
Author(s):  
Hossein Sharifi-Noghabi ◽  
Olga Zolotareva ◽  
Colin C Collins ◽  
Martin Ester
Keyword(s):  
Gene Expression ◽  
Neural Networks ◽  
Prediction Accuracy ◽  
Drug Response ◽  
Deep Neural Networks ◽  
Response Prediction ◽  
Supplementary Information ◽  
Precision Oncology

Abstract Motivation Historically, gene expression has been shown to be the most informative data for drug response prediction. Recent evidence suggests that integrating additional omics can improve the prediction accuracy which raises the question of how to integrate the additional omics. Regardless of the integration strategy, clinical utility and translatability are crucial. Thus, we reasoned a multi-omics approach combined with clinical datasets would improve drug response prediction and clinical relevance. Results We propose MOLI, a multi-omics late integration method based on deep neural networks. MOLI takes somatic mutation, copy number aberration and gene expression data as input, and integrates them for drug response prediction. MOLI uses type-specific encoding sub-networks to learn features for each omics type, concatenates them into one representation and optimizes this representation via a combined cost function consisting of a triplet loss and a binary cross-entropy loss. The former makes the representations of responder samples more similar to each other and different from the non-responders, and the latter makes this representation predictive of the response values. We validate MOLI on in vitro and in vivo datasets for five chemotherapy agents and two targeted therapeutics. Compared to state-of-the-art single-omics and early integration multi-omics methods, MOLI achieves higher prediction accuracy in external validations. Moreover, a significant improvement in MOLI’s performance is observed for targeted drugs when training on a pan-drug input, i.e. using all the drugs with the same target compared to training only on drug-specific inputs. MOLI’s high predictive power suggests it may have utility in precision oncology. Availability and implementation https://github.com/hosseinshn/MOLI. Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals Maximizing the Prediction Accuracy in Tweet Sentiment Extraction using Tensor Flow based Deep Neural Networks

2021 ◽  
Vol 3 (2) ◽  
pp. 61-79
Author(s):  
S Thivaharan ◽  
G Srivatsun
Keyword(s):  
Neural Networks ◽  
Social Media ◽  
Prediction Accuracy ◽  
Deep Neural Networks ◽  
Modern Technology ◽  
The Other ◽  
Data Generation ◽  
Class Prediction ◽  
The Social ◽  
Communication Devices

The amount of data generated by modern communication devices is enormous, reaching petabytes. The rate of data generation is also increasing at an unprecedented rate. Though modern technology supports storage in massive amounts, the industry is reluctant in retaining the data, which includes the following characteristics: redundancy in data, unformatted records with outdated information, data that misleads the prediction and data with no impact on the class prediction. Out of all of this data, social media plays a significant role in data generation. As compared to other data generators, the ratio at which the social media generates the data is comparatively higher. Industry and governments are both worried about the circulation of mischievous or malcontents, as they are extremely susceptible and are used by criminals. So it is high time to develop a model to classify the social media contents as fair and unfair. The developed model should have higher accuracy in predicting the class of contents. In this article, tensor flow based deep neural networks are deployed with a fixed Epoch count of 15, in order to attain 25% more accuracy over the other existing models. Activation methods like “Relu” and “Sigmoid”, which are specific for Tensor flow platforms support to attain the improved prediction accuracy.

Cascaded Deeply Supervised Convolutional Networks for Liver Lesion Segmentation

2021 ◽  
pp. 2152014
Author(s):  
Kaiyi Peng ◽  
Bin Fang ◽  
Mingliang Zhou
Keyword(s):  
Computed Tomography ◽  
Neural Networks ◽  
Prediction Accuracy ◽  
Deep Neural Networks ◽  
Small Volume ◽  
State Of The Art ◽  
Liver Lesion ◽  
Segmentation Method ◽  
Lesion Segmentation ◽  
Convolutional Networks

Liver lesion segmentation from abdomen computed tomography (CT) with deep neural networks remains challenging due to the small volume and the unclear boundary. To effectively tackle these problems, in this paper, we propose a cascaded deeply supervised convolutional networks (CDS-Net). The cascaded deep supervision (CDS) mechanism uses auxiliary losses to construct a cascaded segmentation method in a single network, focusing the network attention on pixels that are more difficult to classify, so that the network can segment the lesion more effectively. CDS mechanism can be easily integrated into standard CNN models and it helps to increase the model sensitivity and prediction accuracy. Based on CDS mechanism, we propose a cascaded deep supervised ResUNet, which is an end-to-end liver lesion segmentation network. We conduct experiments on LiTS and 3DIRCADb dataset. Our method has achieved competitive results compared with other state-of-the-art ones.

scholarly journals MOLI: Multi-Omics Late Integration with deep neural networks for drug response prediction

2019 ◽  
Author(s):  
Hossein Sharifi-Noghabi ◽  
Olga Zolotareva ◽  
Colin C. Collins ◽  
Martin Ester
Keyword(s):  
Gene Expression ◽  
Neural Networks ◽  
Prediction Accuracy ◽  
Drug Response ◽  
Deep Neural Networks ◽  
Response Prediction ◽  
Precision Oncology ◽  
Early Integration

AbstractMotivationHistorically, gene expression has been shown to be the most informative data for drug response prediction. Recent evidence suggests that integrating additional omics can improve the prediction accuracy which raises the question of how to integrate the additional omics. Regardless of the integration strategy, clinical utility and translatability are crucial. Thus, we reasoned a multi-omics approach combined with clinical datasets would improve drug response prediction and clinical relevance.ResultsWe propose MOLI, a Multi-Omics Late Integration method based on deep neural networks. MOLI takes somatic mutation, copy number aberration, and gene expression data as input, and integrates them for drug response prediction. MOLI uses type-specific encoding subnetworks to learn features for each omics type, concatenates them into one representation and optimizes this representation via a combined cost function consisting of a triplet loss and a binary cross-entropy loss. The former makes the representations of responder samples more similar to each and different from the non-responders, and the latter makes this representation predictive of the response values. We validate MOLI on in vitro and in vivo datasets for five chemotherapy agents and two targeted therapeutics. Compared to state-of-the-art single-omics and early integration multi-omics methods, MOLI achieves higher prediction accuracy in external validations. Moreover, a significant improvement in MOLI’s performance is observed for targeted drugs when training on a pan-drug input, i.e. using all the drugs with the same target compared to training only on drug-specific inputs. MOLI’s high predictive power suggests it may have utility in precision oncology.Availability of the implemented codeshttps://github.com/hosseinshn/[email protected] and [email protected]

scholarly journals Early Detection of COVID-19 Patient’s Survavibility Based On The Image Of Lung X-Ray Image Using Deep Neural Networks

2021 ◽  
Author(s):  
Hilmy Bahy Hakim ◽  
Fitri Utaminingrum ◽  
Agung Setia Budi
Keyword(s):  
Neural Networks ◽  
Early Detection ◽  
Prediction Accuracy ◽  
Deep Neural Networks ◽  
Minority Class ◽  
X Ray ◽  
Testing Dataset ◽  
Prediction Time ◽  
Class Weight ◽  
Future Condition

SARS-CoV-2 causes an infection called COVID-19, which is caused by a new coronavirus. One of the symptomps that dangerous to the patients is developing pneumonia in their lungs. To detect pneumonia symptoms, one of the newest methods is using CNN (Convolution Neural Networks). The problem is when able to detect pneumonia, the patient's survivability, which knowing this will be helpful to decide the priority for each patient, is still in question. The CNN used in this research to classify the patient’s future condition, but met some major problems that the dataset is very few and unbalance. The image augmentation was used to multiply the dataset, and class weight was applied to prevent miscalculation on minority class. 6 CNN architectures used to find the best model. The result VGG19 architecture has the best overall accuracy, in training, it has 80% accuracy, 89% accuracy invalidation, and 82% f1 score accuracy on classifying the testing dataset means the best model if looking for accuracy on prediction, but this cost a prediction time that longest compared to other CNN architectures. MobileNet is the fastest, but it cost much worse on prediction accuracy, only 55%. The ResNet50 model has balanced prediction accuracy/time, it got 77% f1 accuracy, and also 8.49 seconds of prediction time, 9 seconds less than VGG19.

Comparing Deep Neural Networks and Gradient Boosting for Pneumonia Detection Using Chest X-Rays

2022 ◽  
pp. 58-79
Author(s):  
Son Nguyen ◽  
Matthew Quinn ◽  
Alan Olinsky ◽  
John Quinn
Keyword(s):  
Neural Networks ◽  
Prediction Accuracy ◽  
Deep Neural Networks ◽  
Classical Model ◽  
Gradient Boosting ◽  
X Rays ◽  
Computational Power ◽  
Computing Services ◽  
Cloud Computing Services ◽  
Machine Learning Models

In recent years, with the development of computational power and the explosion of data available for analysis, deep neural networks, particularly convolutional neural networks, have emerged as one of the default models for image classification, outperforming most of the classical machine learning models in this task. On the other hand, gradient boosting, a classical model, has been widely used for tabular structure data and leading data competitions, such as those from Kaggle. In this study, the authors compare the performance of deep neural networks with gradient boosting models for detecting pneumonia using chest x-rays. The authors implement several popular architectures of deep neural networks, such as Resnet50, InceptionV3, Xception, and MobileNetV3, and variants of a gradient boosting model. The authors then evaluate these two classes of models in terms of prediction accuracy. The computation in this study is done using cloud computing services offered by Google Colab Pro.

scholarly journals Modeling positional effects of regulatory sequences with spline transformations increases prediction accuracy of deep neural networks

2017 ◽  
Author(s):  
Žiga Avsec ◽  
Mohammadamin Barekatain ◽  
Jun Cheng ◽  
Julien Gagneur
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Prediction Accuracy ◽  
Deep Neural Networks ◽  
Rna Binding ◽  
Piecewise Linear ◽  
Regulatory Sequences ◽  
Link Type

AbstractMotivationRegulatory sequences are not solely defined by their nucleic acid sequence but also by their relative distances to genomic landmarks such as transcription start site, exon boundaries, or polyadenylation site. Deep learning has become the approach of choice for modeling regulatory sequences because of its strength to learn complex sequence features. However, modeling relative distances to genomic landmarks in deep neural networks has not been addressed.ResultsHere we developed spline transformation, a neural network module based on splines to flexibly and robustly model distances. Modeling distances to various genomic landmarks with spline transformations significantly increased state-of-the-art prediction accuracy of in vivo RNA-binding protein binding sites for 114 out of 123 proteins. We also developed a deep neural network for human splice branchpoint based on spline transformations that outperformed the current best, already distance-based, machine learning model. Compared to piecewise linear transformation, as obtained by composition of rectified linear units, spline transformation yields higher prediction accuracy as well as faster and more robust training. As spline transformation can be applied to further quantities beyond distances, such as methylation or conservation, we foresee it as a versatile component in the genomics deep learning toolbox.AvailabilitySpline transformation is implemented as a Keras layer in the CONCISE python package: https://github.com/gagneurlab/concise. Analysis code is available at goo.gl/[email protected]; [email protected]

Sign in / Sign up

Export Citation Format

Share Document