scholarly journals Deep neural networks using a single neuron: folded-in-time architecture using feedback-modulated delay loops

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Florian Stelzer ◽  
André Röhm ◽  
Raul Vicente ◽  
Ingo Fischer ◽  
Serhiy Yanchuk
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Neural Network ◽  
Single Neuron ◽  
Deep Neural Networks ◽  
Back Propagation ◽  
Local Network ◽  
Multiple Time ◽  
Learning Tools ◽  
Back Propagation Algorithm

AbstractDeep neural networks are among the most widely applied machine learning tools showing outstanding performance in a broad range of tasks. We present a method for folding a deep neural network of arbitrary size into a single neuron with multiple time-delayed feedback loops. This single-neuron deep neural network comprises only a single nonlinearity and appropriately adjusted modulations of the feedback signals. The network states emerge in time as a temporal unfolding of the neuron’s dynamics. By adjusting the feedback-modulation within the loops, we adapt the network’s connection weights. These connection weights are determined via a back-propagation algorithm, where both the delay-induced and local network connections must be taken into account. Our approach can fully represent standard Deep Neural Networks (DNN), encompasses sparse DNNs, and extends the DNN concept toward dynamical systems implementations. The new method, which we call Folded-in-time DNN (Fit-DNN), exhibits promising performance in a set of benchmark tasks.

Inclusion of Multiple Cycling of the Potential into Deep Neural Network Classification of Voltammetric Reaction Mechanisms

2021 ◽  
Author(s):  
Luke Gundry ◽  
Gareth Kennedy ◽  
Alan Bond ◽  
Jie Zhang
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Reaction Mechanisms ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Neural Network Classification ◽  
Initial Cycle

The use of Deep Neural Networks (DNNs) for the classification of electrochemical mechanisms based on training with simulations of the initial cycle of potential have been reported. In this paper,...

Automated vessel segmentation in lung CT and CTA images via deep neural networks

2021 ◽  
pp. 1-15
Author(s):  
Wenjun Tan ◽  
Luyu Zhou ◽  
Xiaoshuo Li ◽  
Xiaoyu Yang ◽  
Yufei Chen ◽  
...  
Keyword(s):  
Neural Network ◽  
Computed Tomography ◽  
Neural Networks ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Vessel Segmentation ◽  
Multi Scale ◽  
Vascular Segmentation ◽  
Study Results ◽  
Lung Ct

BACKGROUND: The distribution of pulmonary vessels in computed tomography (CT) and computed tomography angiography (CTA) images of lung is important for diagnosing disease, formulating surgical plans and pulmonary research. PURPOSE: Based on the pulmonary vascular segmentation task of International Symposium on Image Computing and Digital Medicine 2020 challenge, this paper reviews 12 different pulmonary vascular segmentation algorithms of lung CT and CTA images and then objectively evaluates and compares their performances. METHODS: First, we present the annotated reference dataset of lung CT and CTA images. A subset of the dataset consisting 7,307 slices for training and 3,888 slices for testing was made available for participants. Second, by analyzing the performance comparison of different convolutional neural networks from 12 different institutions for pulmonary vascular segmentation, the reasons for some defects and improvements are summarized. The models are mainly based on U-Net, Attention, GAN, and multi-scale fusion network. The performance is measured in terms of Dice coefficient, over segmentation ratio and under segmentation rate. Finally, we discuss several proposed methods to improve the pulmonary vessel segmentation results using deep neural networks. RESULTS: By comparing with the annotated ground truth from both lung CT and CTA images, most of 12 deep neural network algorithms do an admirable job in pulmonary vascular extraction and segmentation with the dice coefficients ranging from 0.70 to 0.85. The dice coefficients for the top three algorithms are about 0.80. CONCLUSIONS: Study results show that integrating methods that consider spatial information, fuse multi-scale feature map, or have an excellent post-processing to deep neural network training and optimization process are significant for further improving the accuracy of pulmonary vascular segmentation.

scholarly journals Extensive deep neural networks for transferring small scale learning to large scale systems

2019 ◽  
Vol 10 (15) ◽  
pp. 4129-4140 ◽  
Author(s):  
Kyle Mills ◽  
Kevin Ryczko ◽  
Iryna Luchak ◽  
Adam Domurad ◽  
Chris Beeler ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Large Scale ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Small Scale ◽  
Large Scale Systems ◽  
Energy Entropy ◽  
Large Systems ◽  
Number Of Particles

We present a physically-motivated topology of a deep neural network that can efficiently infer extensive parameters (such as energy, entropy, or number of particles) of arbitrarily large systems, doing so with scaling.

scholarly journals Development and Validation of a Deep Neural Network Model for Prediction of Postoperative In-hospital Mortality

2018 ◽  
Vol 129 (4) ◽  
pp. 649-662 ◽  
Author(s):  
Christine K. Lee ◽  
Ira Hofer ◽  
Eilon Gabel ◽  
Pierre Baldi ◽  
Maxime Cannesson
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Hospital Mortality ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Operating Characteristics ◽  
Physical Status ◽  
Physical Status Classification ◽  
Receiver Operating Characteristics Curve ◽  
Status Classification

Abstract Editor’s Perspective What We Already Know about This Topic What This Article Tells Us That Is New Background The authors tested the hypothesis that deep neural networks trained on intraoperative features can predict postoperative in-hospital mortality. Methods The data used to train and validate the algorithm consists of 59,985 patients with 87 features extracted at the end of surgery. Feed-forward networks with a logistic output were trained using stochastic gradient descent with momentum. The deep neural networks were trained on 80% of the data, with 20% reserved for testing. The authors assessed improvement of the deep neural network by adding American Society of Anesthesiologists (ASA) Physical Status Classification and robustness of the deep neural network to a reduced feature set. The networks were then compared to ASA Physical Status, logistic regression, and other published clinical scores including the Surgical Apgar, Preoperative Score to Predict Postoperative Mortality, Risk Quantification Index, and the Risk Stratification Index. Results In-hospital mortality in the training and test sets were 0.81% and 0.73%. The deep neural network with a reduced feature set and ASA Physical Status classification had the highest area under the receiver operating characteristics curve, 0.91 (95% CI, 0.88 to 0.93). The highest logistic regression area under the curve was found with a reduced feature set and ASA Physical Status (0.90, 95% CI, 0.87 to 0.93). The Risk Stratification Index had the highest area under the receiver operating characteristics curve, at 0.97 (95% CI, 0.94 to 0.99). Conclusions Deep neural networks can predict in-hospital mortality based on automatically extractable intraoperative data, but are not (yet) superior to existing methods.

scholarly journals Prediction of adsorption efficiencies of Ni (II) in aqueous solutions with perlite via artificial neural networks

2017 ◽  
Vol 43 (4) ◽  
pp. 26-32 ◽  
Author(s):  
Sinan Mehmet Turp
Keyword(s):  
Neural Network ◽  
Experimental Data ◽  
Neural Networks ◽  
Artificial Neural Network ◽  
Artificial Neural Networks ◽  
Back Propagation ◽  
Data Sets ◽  
Adsorption Efficiency ◽  
Back Propagation Algorithm ◽  
Artificial Neural

AbstractThis study investigates the estimated adsorption efficiency of artificial Nickel (II) ions with perlite in an aqueous solution using artificial neural networks, based on 140 experimental data sets. Prediction using artificial neural networks is performed by enhancing the adsorption efficiency with the use of Nickel (II) ions, with the initial concentrations ranging from 0.1 mg/L to 10 mg/L, the adsorbent dosage ranging from 0.1 mg to 2 mg, and the varying time of effect ranging from 5 to 30 mins. This study presents an artificial neural network that predicts the adsorption efficiency of Nickel (II) ions with perlite. The best algorithm is determined as a quasi-Newton back-propagation algorithm. The performance of the artificial neural network is determined by coefficient determination (R2), and its architecture is 3-12-1. The prediction shows that there is an outstanding relationship between the experimental data and the predicted values.

scholarly journals Analysis of the possibilities of using a deep neural networks as an image recognition technology to solve the problem of automating attendance tracking

2020 ◽  
pp. 15-26
Author(s):  
Anna Ilina ◽  
Vladimir Korenkov
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Quality Assessment ◽  
Image Recognition ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Facial Recognition ◽  
Manual Counting

The task of counting the number of people is relevant when conducting various types of events, which may include seminars, lectures, conferences, meetings, etc. Instead of monotonous manual counting of participants, it is much more effective to use facial recognition technology, which makes it possible not only to quickly count those present, but also to recognize each of them, which makes it possible to conduct further analysis of this data, identify patterns in them and predict. The research conducted in this paper determines the quality assessment of the use of facial recognition technology in images andvideo streams, based on the use of a deep neural network, to solve the problem of automating attendance tracking.

scholarly journals Building robust neural networks using different loss functions

2021 ◽  
pp. 67-82
Author(s):  
Maria Sivak ◽  
◽  
Vladimir Timofeev ◽  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Loss Function ◽  
Back Propagation ◽  
Loss Functions ◽  
Back Propagation Algorithm ◽  
Error Back Propagation ◽  
Propagation Algorithm ◽  
Error Back Propagation Algorithm ◽  
Parameter Values

The paper considers the problem of building robust neural networks using different robust loss functions. Applying such neural networks is reasonably when working with noisy data, and it can serve as an alternative to data preprocessing and to making neural network architecture more complex. In order to work adequately, the error back-propagation algorithm requires a loss function to be continuously or two-times differentiable. According to this requirement, two five robust loss functions were chosen (Andrews, Welsch, Huber, Ramsey and Fair). Using the above-mentioned functions in the error back-propagation algorithm instead of the quadratic one allows obtaining an entirely new class of neural networks. For investigating the properties of the built networks a number of computational experiments were carried out. Different values of outliers’ fraction and various numbers of epochs were considered. The first step included adjusting the obtained neural networks, which lead to choosing such values of internal loss function parameters that resulted in achieving the highest accuracy of a neural network. To determine the ranges of parameter values, a preliminary study was pursued. The results of the first stage allowed giving recommendations on choosing the best parameter values for each of the loss functions under study. The second stage dealt with comparing the investigated robust networks with each other and with the classical one. The analysis of the results shows that using the robust technique leads to a significant increase in neural network accuracy and in a learning rate.

Explainable Deep Learning Reproduces a ‘Professional Eye’ on the Diagnosis of Internal Disorders in Persimmon Fruit

2020 ◽  
Vol 61 (11) ◽  
pp. 1967-1973
Author(s):  
Takashi Akagi ◽  
Masanori Onishi ◽  
Kanae Masuda ◽  
Ryohei Kuroki ◽  
Kohei Baba ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Binary Classification ◽  
Network Models ◽  
Plant Biology ◽  
Neural Network Models ◽  
Persimmon Fruit

Abstract Recent rapid progress in deep neural network techniques has allowed recognition and classification of various objects, often exceeding the performance of the human eye. In plant biology and crop sciences, some deep neural network frameworks have been applied mainly for effective and rapid phenotyping. In this study, beyond simple optimizations of phenotyping, we propose an application of deep neural networks to make an image-based internal disorder diagnosis that is hard even for experts, and to visualize the reasons behind each diagnosis to provide biological interpretations. Here, we exemplified classification of calyx-end cracking in persimmon fruit by using five convolutional neural network models with various layer structures and examined potential analytical options involved in the diagnostic qualities. With 3,173 visible RGB images from the fruit apex side, the neural networks successfully made the binary classification of each degree of disorder, with up to 90% accuracy. Furthermore, feature visualizations, such as Grad-CAM and LRP, visualize the regions of the image that contribute to the diagnosis. They suggest that specific patterns of color unevenness, such as in the fruit peripheral area, can be indexes of calyx-end cracking. These results not only provided novel insights into indexes of fruit internal disorders but also proposed the potential applicability of deep neural networks in plant biology.

Artificial Neural Network Modelling for Waste

2011 ◽  
pp. 224-263 ◽  
Author(s):  
Eldon R. Rene ◽  
M. Estefanía López ◽  
María C. Veiga ◽  
Christian Kennes
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Artificial Neural Network ◽  
Wastewater Treatment ◽  
Back Propagation ◽  
Back Propagation Algorithm ◽  
Network Modelling ◽  
Waste Gas ◽  
Propagation Algorithm ◽  
Artificial Neural

Due to their inherent robustness, artificial neural network models have proven to be successful and have been used extensively in biological wastewater treatment applications. However, only recently, with the scientific advancements made in biological waste gas treatment systems, the application of neural networks have slowly gained the practical momentum for performance monitoring in this field. Simple neural models, after vigorous training and testing, are able to generalize the results of a wide range of operating conditions, with high prediction accuracy. This chapter gives a fundamental insight and overview of the process mechanism of different biological waste gas (biofilters, biotrickling filters, continuous stirred tank bioreactors and monolith bioreactors), and wastewater treatment systems (activated sludge process, trickling filter and sequencing batch reactors). The basic theory of artificial neural networks is explained with a clear understanding of the back propagation algorithm. A generalized neural network modelling procedure for waste treatment applications is outlined, and the role of back propagation algorithm network parameters is discussed. Anew, the application of neural networks for solving specific environmental problems is presented in the form of a literature review.

Application of neural networks in industrial production

2021 ◽  
pp. 10-17
Author(s):  
S. S. Yudachev ◽  
N. A. Gordienko ◽  
F. M. Bosy
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Effective Means ◽  
Back Propagation ◽  
Back Propagation Algorithm ◽  
Graphics Processors ◽  
Training Scheme ◽  
Network Training ◽  
The Neural Network

The article describes an algorithm for the synthesis of neural networks for controlling the gyrostabilizer. The neural network acts as an observer of the state vector. The role of such an observer is to provide feedback to the gyrostabilizer, which is illustrated in the article. Gyrostabilizer is a gyroscopic device designed to stabilize individual objects or devices, as well as to determine the angular deviations of objects. Gyrostabilizer systems will be more widely used, as they provide an effective means of motion control with a number of significant advantages for various designs. The article deals in detail with the issue of specific stage features of classical algorithms: selecting the network architecture, training the neural network, and verifying the results of feedback control. In recent years, neural networks have become an increasingly powerful tool in scientific computing. The universal approximation theorem states that a neural network can be constructed to approximate any given continuous function with the required accuracy. The back propagation algorithm also allows effectively optimizing the parameters when training a neural network. Due to the use of graphics processors, it is possible to perform efficient calculations for scientific and engineering tasks. The article presents the optimal configuration of the neural network, such as the depth of memory, the number of layers and neurons in these layers, as well as the functions of the activation layer. In addition, it provides data on dynamic systems to improve neural network training. An optimal training scheme is also provided.

Sign in / Sign up

Export Citation Format

Share Document