scholarly journals The whole is greater than its parts: ensembling improves protein contact prediction

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Wendy M. Billings ◽  
Connor J. Morris ◽  
Dennis Della Corte
Keyword(s):  
Neural Networks ◽  
Structure Prediction ◽  
Deep Neural Networks ◽  
Protein Structures ◽  
Network Models ◽  
Critical Assessment ◽  
Neural Network Models ◽  
Contact Prediction ◽  
Homologous Sequences ◽  
Contact Predictions

AbstractThe prediction of amino acid contacts from protein sequence is an important problem, as protein contacts are a vital step towards the prediction of folded protein structures. We propose that a powerful concept from deep learning, called ensembling, can increase the accuracy of protein contact predictions by combining the outputs of different neural network models. We show that ensembling the predictions made by different groups at the recent Critical Assessment of Protein Structure Prediction (CASP13) outperforms all individual groups. Further, we show that contacts derived from the distance predictions of three additional deep neural networks—AlphaFold, trRosetta, and ProSPr—can be substantially improved by ensembling all three networks. We also show that ensembling these recent deep neural networks with the best CASP13 group creates a superior contact prediction tool. Finally, we demonstrate that two ensembled networks can successfully differentiate between the folds of two highly homologous sequences. In order to build further on these findings, we propose the creation of a better protein contact benchmark set and additional open-source contact prediction methods.

Explainable Deep Learning Reproduces a ‘Professional Eye’ on the Diagnosis of Internal Disorders in Persimmon Fruit

2020 ◽  
Vol 61 (11) ◽  
pp. 1967-1973
Author(s):  
Takashi Akagi ◽  
Masanori Onishi ◽  
Kanae Masuda ◽  
Ryohei Kuroki ◽  
Kohei Baba ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Binary Classification ◽  
Network Models ◽  
Plant Biology ◽  
Neural Network Models ◽  
Persimmon Fruit

Abstract Recent rapid progress in deep neural network techniques has allowed recognition and classification of various objects, often exceeding the performance of the human eye. In plant biology and crop sciences, some deep neural network frameworks have been applied mainly for effective and rapid phenotyping. In this study, beyond simple optimizations of phenotyping, we propose an application of deep neural networks to make an image-based internal disorder diagnosis that is hard even for experts, and to visualize the reasons behind each diagnosis to provide biological interpretations. Here, we exemplified classification of calyx-end cracking in persimmon fruit by using five convolutional neural network models with various layer structures and examined potential analytical options involved in the diagnostic qualities. With 3,173 visible RGB images from the fruit apex side, the neural networks successfully made the binary classification of each degree of disorder, with up to 90% accuracy. Furthermore, feature visualizations, such as Grad-CAM and LRP, visualize the regions of the image that contribute to the diagnosis. They suggest that specific patterns of color unevenness, such as in the fruit peripheral area, can be indexes of calyx-end cracking. These results not only provided novel insights into indexes of fruit internal disorders but also proposed the potential applicability of deep neural networks in plant biology.

scholarly journals Integrating ECG Monitoring and Classification via IoT and Deep Neural Networks

Biosensors ◽  
2021 ◽  
Vol 11 (6) ◽  
pp. 188
Author(s):  
Li-Ren Yeh ◽  
Wei-Chin Chen ◽  
Hua-Yan Chan ◽  
Nan-Han Lu ◽  
Chi-Yuan Wang ◽  
...  
Keyword(s):  
Neural Networks ◽  
Real Time ◽  
Deep Neural Networks ◽  
Network Models ◽  
Kappa Statistics ◽  
Neural Network Models ◽  
Ecg Signals ◽  
St Depression ◽  
Deep Model ◽  
St Elevation

Anesthesia assessment is most important during surgery. Anesthesiologists use electrocardiogram (ECG) signals to assess the patient’s condition and give appropriate medications. However, it is not easy to interpret the ECG signals. Even physicians with more than 10 years of clinical experience may still misjudge. Therefore, this study uses convolutional neural networks to classify ECG image types to assist in anesthesia assessment. The research uses Internet of Things (IoT) technology to develop ECG signal measurement prototypes. At the same time, it classifies signal types through deep neural networks, divided into QRS widening, sinus rhythm, ST depression, and ST elevation. Three models, ResNet, AlexNet, and SqueezeNet, are developed with 50% of the training set and test set. Finally, the accuracy and kappa statistics of ResNet, AlexNet, and SqueezeNet in ECG waveform classification were (0.97, 0.96), (0.96, 0.95), and (0.75, 0.67), respectively. This research shows that it is feasible to measure ECG in real time through IoT and then distinguish four types through deep neural network models. In the future, more types of ECG images will be added, which can improve the real-time classification practicality of the deep model.

scholarly journals A Novel Low-Bit Quantization Strategy for Compressing Deep Neural Networks

2020 ◽  
Vol 2020 ◽  
pp. 1-7
Author(s):  
Xin Long ◽  
XiangRong Zeng ◽  
Zongcheng Ben ◽  
Dianle Zhou ◽  
Maojun Zhang
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Neural Networks ◽  
Computational Cost ◽  
Network Models ◽  
Memory Consumption ◽  
Neural Network Models ◽  
The Neural Network ◽  
The Neural Networks ◽  
Novel Strategy

The increase in sophistication of neural network models in recent years has exponentially expanded memory consumption and computational cost, thereby hindering their applications on ASIC, FPGA, and other mobile devices. Therefore, compressing and accelerating the neural networks are necessary. In this study, we introduce a novel strategy to train low-bit networks with weights and activations quantized by several bits and address two corresponding fundamental issues. One is to approximate activations through low-bit discretization for decreasing network computational cost and dot-product memory. The other is to specify weight quantization and update mechanism for discrete weights to avoid gradient mismatch. With quantized low-bit weights and activations, the costly full-precision operation will be replaced by shift operation. We evaluate the proposed method on common datasets, and results show that this method can dramatically compress the neural network with slight accuracy loss.

scholarly journals Surrogate Object Detection Explainer (SODEx) with YOLOv4 and LIME

2021 ◽  
Vol 3 (3) ◽  
pp. 662-671
Author(s):  
Jonas Herskind Sejr ◽  
Peter Schneider-Kamp ◽  
Naeem Ayoub
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Object Detection ◽  
Deep Neural Networks ◽  
Empirical Evaluation ◽  
Network Models ◽  
Detection Algorithm ◽  
Neural Network Models ◽  
The Neural Network

Due to impressive performance, deep neural networks for object detection in images have become a prevalent choice. Given the complexity of the neural network models used, users of these algorithms are typically given no hint as to how the objects were found. It remains, for example, unclear whether an object is detected based on what it looks like or based on the context in which it is located. We have developed an algorithm, Surrogate Object Detection Explainer (SODEx), that can explain any object detection algorithm using any classification explainer. We evaluate SODEx qualitatively and quantitatively by detecting objects in the COCO dataset with YOLOv4 and explaining these detections with LIME. This empirical evaluation does not only demonstrate the value of explainable object detection, it also provides valuable insights into how YOLOv4 detects objects.

scholarly journals Detecting failure modes in image reconstructions with interval neural network uncertainty

2021 ◽  
Author(s):  
Luis Oala ◽  
Cosmas Heiß ◽  
Jan Macdonald ◽  
Maximilian März ◽  
Gitta Kutyniok ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Image Reconstruction ◽  
Deep Neural Networks ◽  
Failure Modes ◽  
Network Models ◽  
Quantitative Detection ◽  
Neural Network Models ◽  
Post Hoc ◽  
Interval Neural Network

Abstract Purpose The quantitative detection of failure modes is important for making deep neural networks reliable and usable at scale. We consider three examples for common failure modes in image reconstruction and demonstrate the potential of uncertainty quantification as a fine-grained alarm system. Methods We propose a deterministic, modular and lightweight approach called Interval Neural Network (INN) that produces fast and easy to interpret uncertainty scores for deep neural networks. Importantly, INNs can be constructed post hoc for already trained prediction networks. We compare it against state-of-the-art baseline methods (MCDrop, ProbOut). Results We demonstrate on controlled, synthetic inverse problems the capacity of INNs to capture uncertainty due to noise as well as directional error information. On a real-world inverse problem with human CT scans, we can show that INNs produce uncertainty scores which improve the detection of all considered failure modes compared to the baseline methods. Conclusion Interval Neural Networks offer a promising tool to expose weaknesses of deep image reconstruction models and ultimately make them more reliable. The fact that they can be applied post hoc to equip already trained deep neural network models with uncertainty scores makes them particularly interesting for deployment.

Building on prior knowledge without building it in

2017 ◽  
Vol 40 ◽  
Author(s):  
Steven S. Hansen ◽  
Andrew K. Lampinen ◽  
Gaurav Suri ◽  
James L. McClelland
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Prior Knowledge ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Network Models ◽  
Causal Models ◽  
Neural Network Models ◽  
Start Up ◽  
The Many

AbstractLake et al. propose that people rely on “start-up software,” “causal models,” and “intuitive theories” built using compositional representations to learn new tasks more efficiently than some deep neural network models. We highlight the many drawbacks of a commitment to compositional representations and describe our continuing effort to explore how the ability to build on prior knowledge and to learn new tasks efficiently could arise through learning in deep neural networks.

scholarly journals Determining the Age of the Author of the Text Based on Deep Neural Network Models

Information ◽  
2020 ◽  
Vol 11 (12) ◽  
pp. 589 ◽  
Author(s):  
Aleksandr Sergeevich Romanov ◽  
Anna Vladimirovna Kurtukova ◽  
Artem Alexandrovich Sobolev ◽  
Alexander Alexandrovich Shelupanov ◽  
Anastasia Mikhailovna Fedotova
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Network Models ◽  
Russian Language ◽  
Neural Network Models ◽  
Inaccurate Data ◽  
Language Text

This paper is devoted to solving the problem of determining the age of the author of the text based on models of deep neural networks. The article presents an analysis of methods for determining the age of the author of a text and approaches to determining the age of a user by a photo. This could be a solution to the problem of inaccurate data for training by filtering out incorrect user-specified age data. A detailed description of the author’s technique based on deep neural network models and the interpretation of the results is also presented. The study found that the proposed technique achieved 82% accuracy in determining the age of the author from Russian-language text, which makes it competitive in comparison with approaches for other languages.

scholarly journals Acoustic-Based UAV Detection Using Late Fusion of Deep Neural Networks

Drones ◽  
2021 ◽  
Vol 5 (3) ◽  
pp. 54
Author(s):  
Pietro Casabianca ◽  
Yu Zhang
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Deep Neural Networks ◽  
Network Models ◽  
Visual Methods ◽  
Late Fusion ◽  
Neural Network Models ◽  
Average Accuracy ◽  
Fusion Methods ◽  
Future Work

Multirotor UAVs have become ubiquitous in commercial and public use. As they become more affordable and more available, the associated security risks further increase, especially in relation to airspace breaches and the danger of drone-to-aircraft collisions. Thus, robust systems must be set in place to detect and deal with hostile drones. This paper investigates the use of deep learning methods to detect UAVs using acoustic signals. Deep neural network models are trained with mel-spectrograms as inputs. In this case, Convolutional Neural Networks (CNNs) are shown to be the better performing network, compared with Recurrent Neural Networks (RNNs) and Convolutional Recurrent Neural Networks (CRNNs). Furthermore, late fusion methods have been evaluated using an ensemble of deep neural networks, where the weighted soft voting mechanism has achieved the highest average accuracy of 94.7%, which has outperformed the solo models. In future work, the developed late fusion technique could be utilized with radar and visual methods to further improve the UAV detection performance.

scholarly journals ACOUSTIC EXTINGUISHING OF FLAMES DETECTED BY DEEP NEURAL NETWORKS IN EMBEDDED SYSTEMS

2021 ◽  
Vol XLVI-4/W5-2021 ◽  
pp. 307-312
Author(s):  
S. Ivanov ◽  
S. Stankov
Keyword(s):  
Neural Networks ◽  
Embedded Systems ◽  
Deep Neural Networks ◽  
Research Work ◽  
Network Models ◽  
Fire Detection ◽  
Artificial Vision ◽  
New Combination ◽  
Smoke Detection ◽  
Neural Network Models

Abstract. This paper presents the application of an intelligent flame and smoke detection platform based on artificial vision using Deep Neural Networks (DNN). An acoustic fire extinguisher can be connected to such a platform. In light of recent research, the use of acoustic technology is an environmentally friendly way of extinguishing flames. One of the advantages is then extinguishing immediately after positive detection of flames or smoke (without unnecessary time delay), which is a new combination in the field of fire protection. The authors contribution is the presentation of the conception of the intelligent acoustic extinguisher and the research work made to accomplish that including algorithms and tests. The main objective of the paper is to present the possibilities of acoustic extinguishing and fire detection using selected deep neural network models in embedded systems.

Sign in / Sign up

Export Citation Format

Share Document