scholarly journals Neural Layer Bypassing Network

2022 ◽  
Author(s):  
Amogh Palasamudram
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Classification Models ◽  
Neural Network Architecture ◽  
Network Layers ◽  
Time Period ◽  
Computationally Intensive ◽  
Fully Connected ◽  
The Given

<p>This research introduces and evaluates the Neural Layer Bypassing Network (NLBN), a new neural network architecture to improve the speed and effectiveness of forward propagation in deep learning. This architecture utilizes 1 additional (fully connected) neural network layer after every layer in the main network. This new layer determines whether finishing the rest of the forward propagation is required to predict the output of the given input. To test the effectiveness of the NLBN, I programmed coding examples for this architecture with 3 different image classification models trained on 3 different datasets: MNIST Handwritten Digits Dataset, Horses or Humans Dataset, and Colorectal Histology Dataset. After training 1 standard convolutional neural network (CNN) and 1 NLBN per dataset (both of equivalent architectures), I performed 5 trials per dataset to analyze the performance of these two architectures. For the NLBN, I also collected data regarding the accuracy, time period, and speed of the network with respect to the percentage of the model the inputs are passed through. It was found that this architecture increases the speed of forward propagation by 6% - 25% while the accuracy tended to decrease by 0% - 4%; the results vary based on the dataset and structure of the model, but the increase in speed was normally at least twice the decrease in accuracy. In addition to the NLBN’s performance during predictions, it takes roughly 40% longer to train and requires more memory due to its complexity. However, the architecture can be made more efficient if integrated into TensorFlow libraries. Overall, by being able to autonomously skip neural network layers, this architecture can potentially be a foundation for neural networks to teach themselves to become more efficient for applications that require fast, accurate, and less computationally intensive predictions.<br></p>

scholarly journals Neural Networks for Driver Behavior Analysis

Electronics ◽  
2021 ◽  
Vol 10 (3) ◽  
pp. 342
Author(s):  
Fabio Martinelli ◽  
Fiammetta Marulli ◽  
Francesco Mercaldo ◽  
Antonella Santone
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Real Time ◽  
Real World ◽  
Experimental Analysis ◽  
Network Architecture ◽  
Driver Behavior ◽  
Neural Network Architecture ◽  
Real World Data ◽  
Fully Connected

The proliferation of info-entertainment systems in nowadays vehicles has provided a really cheap and easy-to-deploy platform with the ability to gather information about the vehicle under analysis. With the purpose to provide an architecture to increase safety and security in automotive context, in this paper we propose a fully connected neural network architecture considering position-based features aimed to detect in real-time: (i) the driver, (ii) the driving style and (iii) the path. The experimental analysis performed on real-world data shows that the proposed method obtains encouraging results.

scholarly journals Neural Layer Bypassing Network

2021 ◽  
Author(s):  
Amogh Palasamudram
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Image Classification ◽  
Network Architecture ◽  
Classification Models ◽  
Learning Models ◽  
Neural Network Architecture ◽  
Overall Performance ◽  
The Impact

<p>This research aims to introduce and evaluate a new neural network architecture to improve the speed and effectiveness of forward propagation in neural networks: the Neural Layer Bypassing Network (NLBN). The theory and workings of this architecture have been explained in this research paper, along with comparisons to other methods of increasing the efficacy of deep learning models. This research also includes code examples with 3 image classification models trained on different datasets and analyses the impact of the NLBN architecture on forward propagation. It was found that this architecture increases the speed of forward propagation and tends to slightly decrease the accuracy of the model. However, it takes longer to train and takes more memory. All in all, this architecture is a potential foundation for using deep learning to teach deep learning models to be more efficient. This includes skipping and re-propagating through layers to improve the overall performance of a model.</p><div><br></div>

SCORING MODELING BASED ON NEURAL NETWORKS FOR DETERMINING A BANK BORROWER'S RATING

2020 ◽  
Vol 2020 (10) ◽  
pp. 54-62
Author(s):  
Oleksii VASYLIEV ◽  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Statistical Data ◽  
Activation Function ◽  
Decision Making Process ◽  
Neural Network Architecture ◽  
Acceptable Accuracy ◽  
The Neural Network ◽  
Sigmoid Activation Function

The problem of applying neural networks to calculate ratings used in banking in the decision-making process on granting or not granting loans to borrowers is considered. The task is to determine the rating function of the borrower based on a set of statistical data on the effectiveness of loans provided by the bank. When constructing a regression model to calculate the rating function, it is necessary to know its general form. If so, the task is to calculate the parameters that are included in the expression for the rating function. In contrast to this approach, in the case of using neural networks, there is no need to specify the general form for the rating function. Instead, certain neural network architecture is chosen and parameters are calculated for it on the basis of statistical data. Importantly, the same neural network architecture can be used to process different sets of statistical data. The disadvantages of using neural networks include the need to calculate a large number of parameters. There is also no universal algorithm that would determine the optimal neural network architecture. As an example of the use of neural networks to determine the borrower's rating, a model system is considered, in which the borrower's rating is determined by a known non-analytical rating function. A neural network with two inner layers, which contain, respectively, three and two neurons and have a sigmoid activation function, is used for modeling. It is shown that the use of the neural network allows restoring the borrower's rating function with quite acceptable accuracy.

scholarly journals Reynolds averaged turbulence modelling using deep neural networks with embedded invariance

2016 ◽  
Vol 807 ◽  
pp. 155-166 ◽  
Author(s):  
Julia Ling ◽  
Andrew Kurzawski ◽  
Jeremy Templeton
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Reynolds Stress ◽  
Network Architecture ◽  
Eddy Viscosity ◽  
Deep Neural Networks ◽  
Test Cases ◽  
Neural Network Architecture ◽  
Stress Anisotropy ◽  
Anisotropy Tensor

There exists significant demand for improved Reynolds-averaged Navier–Stokes (RANS) turbulence models that are informed by and can represent a richer set of turbulence physics. This paper presents a method of using deep neural networks to learn a model for the Reynolds stress anisotropy tensor from high-fidelity simulation data. A novel neural network architecture is proposed which uses a multiplicative layer with an invariant tensor basis to embed Galilean invariance into the predicted anisotropy tensor. It is demonstrated that this neural network architecture provides improved prediction accuracy compared with a generic neural network architecture that does not embed this invariance property. The Reynolds stress anisotropy predictions of this invariant neural network are propagated through to the velocity field for two test cases. For both test cases, significant improvement versus baseline RANS linear eddy viscosity and nonlinear eddy viscosity models is demonstrated.

scholarly journals Towards Heterogeneous Multi-Agent Reinforcement Learning with Graph Neural Networks

2020 ◽  
Author(s):  
Douglas Meneghetti ◽  
Reinaldo Bianchi
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Communication Channels ◽  
Neural Network Architecture ◽  
Graph Representations ◽  
Labeled Graph ◽  
Multiple Agent ◽  
Multi Agent ◽  
Graph Neural Networks

This work proposes a neural network architecture that learns policies for multiple agent classes in a heterogeneous multi-agent reinforcement setting. The proposed network uses directed labeled graph representations for states, encodes feature vectors of different sizes for different entity classes, uses relational graph convolution layers to model different communication channels between entity types and learns distinct policies for different agent classes, sharing parameters wherever possible. Results have shown that specializing the communication channels between entity classes is a promising step to achieve higher performance in environments composed of heterogeneous entities.

scholarly journals Comparative Performance Analysis of Neural Network Real-Time Object Detections in Different Implementations

2020 ◽  
Vol 226 ◽  
pp. 02020
Author(s):  
Alexey V. Stadnik ◽  
Pavel S. Sazhin ◽  
Slavomir Hnatic
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Computer Vision ◽  
Performance Analysis ◽  
Object Detection ◽  
Real Time ◽  
Network Architecture ◽  
Neural Network Architecture ◽  
Comparative Performance

The performance of neural networks is one of the most important topics in the field of computer vision. In this work, we analyze the speed of object detection using the well-known YOLOv3 neural network architecture in different frameworks under different hardware requirements. We obtain results, which allow us to formulate preliminary qualitative conclusions about the feasibility of various hardware scenarios to solve tasks in real-time environments.

scholarly journals Optimizing the Simplicial-Map Neural Network Architecture

2021 ◽  
Vol 7 (9) ◽  
pp. 173
Author(s):  
Eduardo Paluzo-Hidalgo ◽  
Rocio Gonzalez-Diaz ◽  
Miguel A. Gutiérrez-Naranjo ◽  
Jónathan Heras
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Simplicial Complexes ◽  
Original Network ◽  
Neural Network Architecture ◽  
Simplicial Map ◽  
Classification Tool ◽  
Universal Approximators

Simplicial-map neural networks are a recent neural network architecture induced by simplicial maps defined between simplicial complexes. It has been proved that simplicial-map neural networks are universal approximators and that they can be refined to be robust to adversarial attacks. In this paper, the refinement toward robustness is optimized by reducing the number of simplices (i.e., nodes) needed. We have shown experimentally that such a refined neural network is equivalent to the original network as a classification tool but requires much less storage.

scholarly journals Identifikasi Penyakit Diabetes Millitus Menggunakan Jaringan Syaraf Tiruan Dengan Metode Perambatan-Balik (Backpropagation)

2018 ◽  
Author(s):  
Sutedi Sutedi
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Neural Network Architecture ◽  
Accuracy Rate ◽  
Layer 2 ◽  
Input Layer ◽  
Hidden Layer ◽  
Diabetes Melitus

Diabetes Melitus (DM) is dangerous disease that affect many of the variouslayer of work society. This disease is not easy to accurately recognized by thegeneral society. So we need to develop a system that can identify accurately. Systemis built using neural networks with backpropagation methods and the functionactivation sigmoid. Neural network architecture using 8 input layer, 2 output layerand 5 hidden layer. The results show that this methods succesfully clasifies datadiabetics and non diabetics with near 100% accuracy rate.

scholarly journals Baby Cry Detection in Domestic Environment using Convolutional Neural Networks

2020 ◽  
Vol 9 (7) ◽  
pp. 793-795
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Network Architecture ◽  
Web Application ◽  
Well Being ◽  
Emotion Detection ◽  
Neural Network Architecture ◽  
Domestic Environment ◽  
The Neural Network

In this paper we will identify a cry signals of infants and the explanation behind the screams below 0-6 months of segment age. Detection of baby cry signals is essential for the pre-processing of various applications involving crial analysis for baby caregivers, such as emotion detection. Since cry signals hold baby well-being information and can be understood to an extent by experienced parents and experts. We train and validate the neural network architecture for baby cry detection and also test the fastAI with the neural network. Trained neural networks will provide a model and this model can predict the reason behind the cry sound. Only the cry sounds are recognized, and alert the user automatically. Created a web application by responding and detecting different emotions including hunger, tired, discomfort, bellypain.

Application of convolutional neural networks for monitoring of marine objects

2020 ◽  
pp. 53-61
Author(s):  
Н.А. Полковникова ◽  
Е.В. Тузинкевич ◽  
А.Н. Попов
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Computer Vision ◽  
Convolutional Neural Network ◽  
Convolutional Neural Networks ◽  
Network Architecture ◽  
Research Process ◽  
Probability Of Detection ◽  
Neural Network Architecture ◽  
Deep Convolutional Neural Networks

В статье рассмотрены технологии компьютерного зрения на основе глубоких свёрточных нейронных сетей. Применение нейронных сетей особенно эффективно для решения трудно формализуемых задач. Разработана архитектура свёрточной нейронной сети применительно к задаче распознавания и классификации морских объектов на изображениях. В ходе исследования выполнен ретроспективный анализ технологий компьютерного зрения и выявлен ряд проблем, связанных с применением нейронных сетей: «исчезающий» градиент, переобучение и вычислительная сложность. При разработке архитектуры нейросети предложено использовать функцию активации RELU, обучение некоторых случайно выбранных нейронов и нормализацию с целью упрощения архитектуры нейросети. Сравнение используемых в нейросети функций активации ReLU, LeakyReLU, Exponential ReLU и SOFTMAX выполнено в среде Matlab R2020a. На основе свёрточной нейронной сети разработана программа на языке программирования Visual C# в среде MS Visual Studio для распознавания морских объектов. Программапредназначена для автоматизированной идентификации морских объектов, производит детектирование (нахождение объектов на изображении) и распознавание объектов с высокой вероятностью обнаружения. The article considers computer vision technologies based on deep convolutional neural networks. Application of neural networks is particularly effective for solving difficult formalized problems. As a result convolutional neural network architecture to the problem of recognition and classification of marine objects on images is implemented. In the research process a retrospective analysis of computer vision technologies was performed and a number of problems associated with the use of neural networks were identified: vanishing gradient, overfitting and computational complexity. To solve these problems in neural network architecture development, it was proposed to use RELU activation function, training some randomly selected neurons and normalization for simplification of neural network architecture. Comparison of ReLU, LeakyReLU, Exponential ReLU, and SOFTMAX activation functions used in the neural network implemented in Matlab R2020a.The computer program based on convolutional neural network for marine objects recognition implemented in Visual C# programming language in MS Visual Studio integrated development environment. The program is designed for automated identification of marine objects, produces detection (i.e., presence of objects on image), and objects recognition with high probability of detection.

Sign in / Sign up

Export Citation Format

Share Document