On Generalized Residual Network for Deep Learning of Unknown Dynamical Systems

The lack of research on the metering characteristics of electricity power meters under complex conditions is a major obstacle to the on-site verification of electrical energy metering equipment. Establishing a predictive model for electricity power meter errors offers an effective way of dealing with this issue. Deep learning has been proven to have the capacity to reduce end-to-end dimensionality and improve recognition. Through the analysis of the back propagation process in residual networks, an improved residual network is set out in this paper. While preserving the advantages of residual network gradient propagation, it adds an adjustable shortcut and designs a convex [Formula: see text]-parameter strategy that can be improved according to different processing objects. Experimental results show that the predicted errors produced by the proposed technique are significantly lower than in a comparable model. At the same time, the improved residual network does not increase the network’s complexity.

Download Full-text

Hyperparameters optimization for ResNet and Xception in the purpose of diagnosing COVID-19

Journal of Intelligent & Fuzzy Systems ◽

10.3233/jifs-210925 ◽

2021 ◽

pp. 1-17

Author(s):

Hania H. Farag ◽

Lamiaa A. A. Said ◽

Mohamed R. M. Rizk ◽

Magdy Abd ElAzim Ahmed

Keyword(s):

Neural Network ◽

Neural Networks ◽

Deep Learning ◽

Convolutional Neural Network ◽

Random Search ◽

Learning Networks ◽

Residual Network ◽

Global Pandemic ◽

Search Optimization

COVID-19 has been considered as a global pandemic. Recently, researchers are using deep learning networks for medical diseases’ diagnosis. Some of these researches focuses on optimizing deep learning neural networks for enhancing the network accuracy. Optimizing the Convolutional Neural Network includes testing various networks which are obtained through manually configuring their hyperparameters, then the configuration with the highest accuracy is implemented. Each time a different database is used, a different combination of the hyperparameters is required. This paper introduces two COVID-19 diagnosing systems using both Residual Network and Xception Network optimized by random search in the purpose of finding optimal models that give better diagnosis rates for COVID-19. The proposed systems showed that hyperparameters tuning for the ResNet and the Xception Net using random search optimization give more accurate results than other techniques with accuracies 99.27536% and 100 % respectively. We can conclude that hyperparameters tuning using random search optimization for either the tuned Residual Network or the tuned Xception Network gives better accuracies than other techniques diagnosing COVID-19.

Download Full-text

Deep Learning Assisted Covid-19 Detection using full CT-scans

10.36227/techrxiv.13162049.v1 ◽

2020 ◽

Author(s):

varan singhrohila ◽

Nitin Gupta ◽

Amit Kaul ◽

Deepak Sharma

Keyword(s):

Deep Learning ◽

Ct Scans ◽

Detection Accuracy ◽

Residual Network ◽

Excellent Performance ◽

Examination Procedure ◽

Medical Institutions ◽

Proposed Model ◽

Learning Technique ◽

Intensive Approach

<div>The ongoing pandemic of COVID-19 has shown</div><div>the limitations of our current medical institutions. There</div><div>is a need for research in the field of automated diagnosis</div><div>for speeding up the process while maintaining accuracy</div><div>and reducing computational requirements. In this work, an</div><div>automatic diagnosis of COVID-19 infection from CT scans</div><div>of the patients using Deep Learning technique is proposed.</div><div>The proposed model, ReCOV-101 uses full chest CT scans to</div><div>detect varying degrees of COVID-19 infection, and requires</div><div>less computational power. Moreover, in order to improve</div><div>the detection accuracy the CT-scans were preprocessed by</div><div>employing segmentation and interpolation. The proposed</div><div>scheme is based on the residual network, taking advantage</div><div>of skip connection, allowing the model to go deeper.</div><div>Moreover, the model was trained on a single enterpriselevel</div><div>GPU such that it can easily be provided on the edge of</div><div>the network, reducing communication with the cloud often</div><div>required for processing the data. The objective of this work</div><div>is to demonstrate a less hardware-intensive approach for COVID-19 detection with excellent performance that can</div><div>be combined with medical equipment and help ease the</div><div>examination procedure. Moreover, with the proposed model</div><div>an accuracy of 94.9% was achieved.</div>

Download Full-text

Is the Skip Connection Provable to Reform the Neural Network Loss Landscape?

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2020/387 ◽

2020 ◽

Author(s):

Lifu Wang ◽

Bo Shen ◽

Ning Zhao ◽

Zhiyuan Zhang

Keyword(s):

Neural Network ◽

Deep Learning ◽

Level Sets ◽

Theoretical Explanation ◽

Learning Ability ◽

Local Minima ◽

Global Minima ◽

Residual Network ◽

The Neural Network ◽

Hidden Nodes

The residual network is now one of the most effective structures in deep learning, which utilizes the skip connections to “guarantee" the performance will not get worse. However, the non-convexity of the neural network makes it unclear whether the skip connections do provably improve the learning ability since the nonlinearity may create many local minima. In some previous works [Freeman and Bruna, 2016], it is shown that despite the non-convexity, the loss landscape of the two-layer ReLU network has good properties when the number m of hidden nodes is very large. In this paper, we follow this line to study the topology (sub-level sets) of the loss landscape of deep ReLU neural networks with a skip connection and theoretically prove that the skip connection network inherits the good properties of the two-layer network and skip connections can help to control the connectedness of the sub-level sets, such that any local minima worse than the global minima of some two-layer ReLU network will be very “shallow". The “depth" of these local minima are at most O(m^(η-1)/n), where n is the input dimension, η<1. This provides a theoretical explanation for the effectiveness of the skip connection in deep learning.

Download Full-text

A deep learning model for bearing fault diagnosis based on convolution neural network with multi-channel and residual network

10.1109/ccdc52312.2021.9601592 ◽

2021 ◽

Author(s):

Jianyong Tuo ◽

Yu Hu ◽

Xin Ma ◽

Youqing Wang

Keyword(s):

Neural Network ◽

Deep Learning ◽

Fault Diagnosis ◽

Learning Model ◽

Convolution Neural Network ◽

Residual Network ◽

Bearing Fault ◽

Bearing Fault Diagnosis ◽

Deep Learning Model

Download Full-text

Deep Residual Neural Networks Resolve Quartet Molecular Phylogenies

Molecular Biology and Evolution ◽

10.1093/molbev/msz307 ◽

2019 ◽

Vol 37 (5) ◽

pp. 1495-1507 ◽

Cited By ~ 1

Author(s):

Zhengting Zou ◽

Hongjiu Zhang ◽

Yuanfang Guan ◽

Jianzhi Zhang

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Sequence Data ◽

Model Misspecification ◽

Phylogenetic Reconstruction ◽

Likelihood Method ◽

Primary Data ◽

Sequence Evolution ◽

Residual Network ◽

Inference Problems

Abstract Phylogenetic inference is of fundamental importance to evolutionary as well as other fields of biology, and molecular sequences have emerged as the primary data for this task. Although many phylogenetic methods have been developed to explicitly take into account substitution models of sequence evolution, such methods could fail due to model misspecification or insufficiency, especially in the face of heterogeneities in substitution processes across sites and among lineages. In this study, we propose to infer topologies of four-taxon trees using deep residual neural networks, a machine learning approach needing no explicit modeling of the subject system and having a record of success in solving complex nonlinear inference problems. We train residual networks on simulated protein sequence data with extensive amino acid substitution heterogeneities. We show that the well-trained residual network predictors can outperform existing state-of-the-art inference methods such as the maximum likelihood method on diverse simulated test data, especially under extensive substitution heterogeneities. Reassuringly, residual network predictors generally agree with existing methods in the trees inferred from real phylogenetic data with known or widely believed topologies. Furthermore, when combined with the quartet puzzling algorithm, residual network predictors can be used to reconstruct trees with more than four taxa. We conclude that deep learning represents a powerful new approach to phylogenetic reconstruction, especially when sequences evolve via heterogeneous substitution processes. We present our best trained predictor in a freely available program named Phylogenetics by Deep Learning (PhyDL, https://gitlab.com/ztzou/phydl; last accessed January 3, 2020).

Download Full-text

Enhanced deep residual network with multilevel correlation information for fault diagnosis of rotating machinery

Journal of Vibration and Control ◽

10.1177/1077546320949719 ◽

2020 ◽

pp. 107754632094971 ◽

Cited By ~ 1

Author(s):

Shoucong Xiong ◽

Shuai He ◽

Jianping Xuan ◽

Qi Xia ◽

Tielin Shi

Keyword(s):

Deep Learning ◽

Fault Diagnosis ◽

Wavelet Packet ◽

Learning Algorithms ◽

Vibration Signal ◽

Rolling Bearing ◽

Classification Performance ◽

Residual Network ◽

Time Frequency ◽

Correlation Information

Modern machinery becomes more precious with the advance of science, and fault diagnosis is vital for avoiding economical losses or casualties. Among massive diagnosis methods, deep learning algorithms stand out to open an era of intelligent fault diagnosis. Deep residual networks are the state-of-the-art deep learning models which can continuously improve performance by deepening the network structures. However, in vibration-based fault diagnosis, the transient property instability of vibration signal usually calls for time–frequency analysis methods, and the characters of time–frequency matrices are distinct from standard images, which brings some natural limitations for the diagnosis performance of deep learning algorithms. To handle this issue, an enhanced deep residual network named the multilevel correlation stack-deep residual network is proposed in this article. Wavelet packet transform is used to preprocess the sensor signal, and then the proposed multilevel correlation stack-deep residual network uses kernels with different shapes to fully dig various kinds of useful information from any local regions of the processed input. Experiments on two rolling bearing datasets are carried out. Test results show that the multilevel correlation stack-deep residual network exhibits a more satisfactory classification performance than original deep residual networks and other similar methods, revealing significant potentials for realistic fault diagnosis applications.

Download Full-text

Deep Learning-Based Target Tracking and Classification for Low Quality Videos Using Coded Aperture Cameras

Sensors ◽

10.3390/s19173702 ◽

2019 ◽

Vol 19 (17) ◽

pp. 3702 ◽

Cited By ~ 7

Author(s):

Chiman Kwan ◽

Bryan Chou ◽

Jonathan Yang ◽

Akshay Rangamani ◽

Trac Tran ◽

...

Keyword(s):

Deep Learning ◽

Power Consumption ◽

Compressive Sensing ◽

Target Tracking ◽

Low Power Consumption ◽

Coded Aperture ◽

Learning Approach ◽

Residual Network ◽

Practical Applications ◽

Measurement Domain

Compressive sensing has seen many applications in recent years. One type of compressive sensing device is the Pixel-wise Code Exposure (PCE) camera, which has low power consumption and individual control of pixel exposure time. In order to use PCE cameras for practical applications, a time consuming and lossy process is needed to reconstruct the original frames. In this paper, we present a deep learning approach that directly performs target tracking and classification in the compressive measurement domain without any frame reconstruction. In particular, we propose to apply You Only Look Once (YOLO) to detect and track targets in the frames and we propose to apply Residual Network (ResNet) for classification. Extensive simulations using low quality optical and mid-wave infrared (MWIR) videos in the SENSIAC database demonstrated the efficacy of our proposed approach.

Download Full-text