scholarly journals CommNets: Communicating Neural Network Architectures for Resource Constrained Systems

2019 ◽  
Vol 33 ◽  
pp. 9909-9910
Author(s):  
Prince M Abudu
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Network Architecture ◽  
Resource Constraints ◽  
Constrained Systems ◽  
Detection Accuracy ◽  
Network Architectures ◽  
Operating Environments ◽  
Hidden States

Applications that require heterogeneous sensor deployments continue to face practical challenges owing to resource constraints within their operating environments (i.e. energy efficiency, computational power and reliability). This has motivated the need for effective ways of selecting a sensing strategy that maximizes detection accuracy for events of interest using available resources and data-driven approaches. Inspired by those limitations, we ask a fundamental question: whether state-of-the-art Recurrent Neural Networks can observe different series of data and communicate their hidden states to collectively solve an objective in a distributed fashion. We realize our answer by conducting a series of systematic analyses of a Communicating Recurrent Neural Network architecture on varying time-steps, objective functions and number of nodes. The experimental setup we employ models tasks synonymous with those in Wireless Sensor Networks. Our contributions show that Recurrent Neural Networks can communicate through their hidden states and we achieve promising results.

scholarly journals A mixed-scale dense convolutional neural network for image analysis

2017 ◽  
Vol 115 (2) ◽  
pp. 254-259 ◽  
Author(s):  
Daniël M. Pelt ◽  
James A. Sethian
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Image Processing ◽  
Network Architecture ◽  
Training Data ◽  
Network Architectures ◽  
Feature Maps ◽  
Deep Convolutional Neural Networks ◽  
Single Set ◽  
Reduced Risk

Deep convolutional neural networks have been successfully applied to many image-processing problems in recent works. Popular network architectures often add additional operations and connections to the standard architecture to enable training deeper networks. To achieve accurate results in practice, a large number of trainable parameters are often required. Here, we introduce a network architecture based on using dilated convolutions to capture features at different image scales and densely connecting all feature maps with each other. The resulting architecture is able to achieve accurate results with relatively few parameters and consists of a single set of operations, making it easier to implement, train, and apply in practice, and automatically adapts to different problems. We compare results of the proposed network architecture with popular existing architectures for several segmentation problems, showing that the proposed architecture is able to achieve accurate results with fewer parameters, with a reduced risk of overfitting the training data.

scholarly journals Rumor Detection Based on SAGNN: Simplified Aggregation Graph Neural Networks

2021 ◽  
Vol 3 (1) ◽  
pp. 84-94
Author(s):  
Liang Zhang ◽  
Jingqun Li ◽  
Bin Zhou ◽  
Yan Jia
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Network Architectures ◽  
Neural Network Architecture ◽  
Convolutional Networks ◽  
Effective Manner ◽  
Deep Learning Network ◽  
Graph Neural Networks ◽  
Rumor Detection

Identifying fake news on media has been an important issue. This is especially true considering the wide spread of rumors on popular social networks such as Twitter. Various kinds of techniques have been proposed for automatic rumor detection. In this work, we study the application of graph neural networks for rumor classification at a lower level, instead of applying existing neural network architectures to detect rumors. The responses to true rumors and false rumors display distinct characteristics. This suggests that it is essential to capture such interactions in an effective manner for a deep learning network to achieve better rumor detection performance. To this end we present a simplified aggregation graph neural network architecture. Experiments on publicly available Twitter datasets demonstrate that the proposed network has performance on a par with or even better than that of state-of-the-art graph convolutional networks, while significantly reducing the computational complexity.

Convolutional Recurrent Neural Networks for Text Classification

2021 ◽  
Vol 32 (4) ◽  
pp. 65-82
Author(s):  
Shengfei Lyu ◽  
Jiaqi Liu
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Network ◽  
Recurrent Neural Network ◽  
Text Classification ◽  
Recurrent Neural Networks ◽  
Superior Performance ◽  
Novel Approach ◽  
Hidden States ◽  
Traditional Approaches

Recurrent neural network (RNN) and convolutional neural network (CNN) are two prevailing architectures used in text classification. Traditional approaches combine the strengths of these two networks by straightly streamlining them or linking features extracted from them. In this article, a novel approach is proposed to maintain the strengths of RNN and CNN to a great extent. In the proposed approach, a bi-directional RNN encodes each word into forward and backward hidden states. Then, a neural tensor layer is used to fuse bi-directional hidden states to get word representations. Meanwhile, a convolutional neural network is utilized to learn the importance of each word for text classification. Empirical experiments are conducted on several datasets for text classification. The superior performance of the proposed approach confirms its effectiveness.

A Low-Sensitivity Recurrent Neural Network

1998 ◽  
Vol 10 (1) ◽  
pp. 165-188 ◽  
Author(s):  
Andrew D. Back ◽  
Ah Chung Tsoi
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Recurrent Neural Network ◽  
Recurrent Neural Networks ◽  
Network Architecture ◽  
High Sensitivity ◽  
Feedforward Neural Networks ◽  
Parameter Sensitivity ◽  
Model Parameters ◽  
Low Sensitivity

The problem of high sensitivity in modeling is well known. Small perturbations in the model parameters may result in large, undesired changes in the model behavior. A number of authors have considered the issue of sensitivity in feedforward neural networks from a probabilistic perspective. Less attention has been given to such issues in recurrent neural networks. In this article, we present a new recurrent neural network architecture, that is capable of significantly improved parameter sensitivity properties compared to existing recurrent neural networks. The new recurrent neural network generalizes previous architectures by employing alternative discrete-time operators in place of the shift operator normally used. An analysis of the model demonstrates the existence of parameter sensitivity in recurrent neural networks and supports the proposed architecture. The new architecture performs significantly better than previous recurrent neural networks, as shown by a series of simple numerical experiments.

scholarly journals Suggestion and invention of recipes using bi-directional LSTMs-based frameworks

2021 ◽  
Vol 3 (5) ◽  
Author(s):  
Sai Nikhil Rao Gona ◽  
Himamsu Marellapudi
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Loss Function ◽  
Recurrent Neural Networks ◽  
Network Architecture ◽  
Loss Functions ◽  
Neural Network Architecture ◽  
Calorie Content ◽  
Proposed Model ◽  
Variational Autoencoder

AbstractChoosing which recipe to eat and which recipe to avoid isn’t that simple for anyone. It takes strenuous efforts and a lot of time for people to calculate the number of calories and P.H level of the dish. In this paper, we propose an ensemble neural network architecture that suggests recipes based on the taste of the person, P.H level and calorie content of the recipes. We also propose a bi-directional LSTMs-based variational autoencoder for generating new recipes. We have ensembled three bi-directional LSTM-based recurrent neural networks which can classify the recipes based on the taste of the person, P.H level of the recipe and calorie content of the recipe. The proposed model also predicts the taste ratings of the recipes for which we proposed a custom loss function which gave better results than the standard loss functions and the model also predicts the calorie content of the recipes. The bi-directional LSTMs-based variational autoencoder after being trained with the recipes which are fit for the person generates new recipes from the existing recipes. After training and testing the recurrent neural networks and the variational autoencoder, we have tested the model with 20 new recipes and got overwhelming results in the experimentation, the variational autoencoders generated a couple of new recipes, which are healthy to the specific person and will be liked by the specific person.

scholarly journals Rumor Detection Based on SAGNN: Simplified Aggregation Graph Neural Networks

2020 ◽  
Author(s):  
Liang Zhang ◽  
Jingqun Li ◽  
Bin Zhou ◽  
Yan Jia
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Network Architectures ◽  
Convolutional Networks ◽  
Effective Manner ◽  
The Media ◽  
Deep Learning Network ◽  
Graph Neural Networks ◽  
Rumor Detection

Identifying fake news on the media has been an important issue. This is especially true considering the wide spread of rumors on the popular social networks such as Twitter. Various kinds of techniques have been proposed for automatic rumor detection. In this work, we study the application of graph neural networks for rumor classification at a lower level, instead of applying existing neural network architectures to detect rumors. The responses to true rumors and false rumors display distinct characteristics. This suggests that it is essential to capture such interactions in an effective manner for a deep learning network to achieve better rumor detection performance. To this end we present a simplified aggregation graph neural network architecture. Experiments on publicly available Twitter datasets demonstrate that the proposed network has performance on a par with or even better than that of state-of-the-art graph convolutional networks, while significantly reducing the computational complexity.

scholarly journals Power signal disturbance classification using wavelet based neural network

2007 ◽  
Vol 4 (1) ◽  
pp. 71-83 ◽  
Author(s):  
S. Suja ◽  
Jovitha Jerome
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Recognition System ◽  
Discrete Wavelet ◽  
Detection Accuracy ◽  
Neural Network Architecture ◽  
The Third ◽  
Power Quality Disturbance ◽  
Main Components

In this paper, the power signal disturbances are detected using discrete wavelet transform (DWT) and categorized using neural networks. This paper presents a prototype of power quality disturbance recognition system. The prototype contains three main components. First a simulator is used to generate power signal disturbances. The second component is a detector which uses the technique of DWT to detect the power signal disturbances. DWT is used to extract disturbance features in the power signal. These coefficients obtained from DWT are further subjected to statistical manipulations for increasing the detection accuracy. The third component is neural network architecture to classify the power signal disturbances with increased accuracy of detection.

RANDOM MULTI-MODAL DEEP LEARNING IN THE PROBLEM OF IMAGE RECOGNITION

2021 ◽  
pp. 21-26
Author(s):  
А.И. Паршин ◽  
М.Н. Аралов ◽  
В.Ф. Барабанов ◽  
Н.И. Гребенникова
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
Image Recognition ◽  
Recurrent Neural Networks ◽  
Network Architecture ◽  
Deep Neural Networks ◽  
Machine Learning Algorithms ◽  
Complex Data ◽  
Advantages And Disadvantages

Задача распознавания изображений - одна из самых сложных в машинном обучении, требующая от исследователя как глубоких знаний, так и больших временных и вычислительных ресурсов. В случае использования нелинейных и сложных данных применяются различные архитектуры глубоких нейронных сетей, но при этом сложным вопросом остается проблема выбора нейронной сети. Основными архитектурами, используемыми повсеместно, являются свёрточные нейронные сети (CNN), рекуррентные нейронные сети (RNN), глубокие нейронные сети (DNN). На основе рекуррентных нейронных сетей (RNN) были разработаны сети с долгой краткосрочной памятью (LSTM) и сети с управляемыми реккурентными блоками (GRU). Каждая архитектура нейронной сети имеет свою структуру, свои настраиваемые и обучаемые параметры, обладает своими достоинствами и недостатками. Комбинируя различные виды нейронных сетей, можно существенно улучшить качество предсказания в различных задачах машинного обучения. Учитывая, что выбор оптимальной архитектуры сети и ее параметров является крайне трудной задачей, рассматривается один из методов построения архитектуры нейронных сетей на основе комбинации свёрточных, рекуррентных и глубоких нейронных сетей. Показано, что такие архитектуры превосходят классические алгоритмы машинного обучения The image recognition task is one of the most difficult in machine learning, requiring both deep knowledge and large time and computational resources from the researcher. In the case of using nonlinear and complex data, various architectures of deep neural networks are used but the problem of choosing a neural network remains a difficult issue. The main architectures used everywhere are convolutional neural networks (CNN), recurrent neural networks (RNN), deep neural networks (DNN). Based on recurrent neural networks (RNNs), Long Short Term Memory Networks (LSTMs) and Controlled Recurrent Unit Networks (GRUs) were developed. Each neural network architecture has its own structure, customizable and trainable parameters, and advantages and disadvantages. By combining different types of neural networks, you can significantly improve the quality of prediction in various machine learning problems. Considering that the choice of the optimal network architecture and its parameters is an extremely difficult task, one of the methods for constructing the architecture of neural networks based on a combination of convolutional, recurrent and deep neural networks is considered. We showed that such architectures are superior to classical machine learning algorithms

scholarly journals Learning to Learn Personalized Neural Network for Ventricular Arrhythmias Detection on Intracardiac EGMs

2021 ◽  
Author(s):  
Zhenge Jia ◽  
Zhepeng Wang ◽  
Feng Hong ◽  
Lichuan PING ◽  
Yiyu Shi ◽  
...  
Keyword(s):  
Neural Network ◽  
Network Architecture ◽  
Ventricular Arrhythmias ◽  
Resource Constraints ◽  
Learning Algorithm ◽  
Detection Methods ◽  
Model Parameters ◽  
Detection Accuracy ◽  
Implantable Cardioverter Defibrillators ◽  
Life Threatening

Life-threatening ventricular arrhythmias (VAs) detection on intracardiac electrograms (IEGMs) is essential to Implantable Cardioverter Defibrillators (ICDs). However, current VAs detection methods count on a variety of heuristic detection criteria, and require frequent manual interventions to personalize criteria parameters for each patient to achieve accurate detection. In this work, we propose a one-dimensional convolutional neural network (1D-CNN) based life-threatening VAs detection on IEGMs. The network architecture is elaborately designed to satisfy the extreme resource constraints of the ICD while maintaining high detection accuracy. We further propose a meta-learning algorithm with a novel patient-wise training tasks formatting strategy to personalize the 1D-CNN. The algorithm generates a well-generalized model initialization containing across-patient knowledge, and performs a quick adaptation of the model to the specific patient's IEGMs. In this way, a new patient could be immediately assigned with personalized 1D-CNN model parameters using limited input data. Compared with the conventional VAs detection method, the proposed method achieves 2.2% increased sensitivity for detecting VAs rhythm and 8.6% increased specificity for non-VAs rhythm.

SCORING MODELING BASED ON NEURAL NETWORKS FOR DETERMINING A BANK BORROWER'S RATING

2020 ◽  
Vol 2020 (10) ◽  
pp. 54-62
Author(s):  
Oleksii VASYLIEV ◽  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Statistical Data ◽  
Activation Function ◽  
Decision Making Process ◽  
Neural Network Architecture ◽  
Acceptable Accuracy ◽  
The Neural Network ◽  
Sigmoid Activation Function

The problem of applying neural networks to calculate ratings used in banking in the decision-making process on granting or not granting loans to borrowers is considered. The task is to determine the rating function of the borrower based on a set of statistical data on the effectiveness of loans provided by the bank. When constructing a regression model to calculate the rating function, it is necessary to know its general form. If so, the task is to calculate the parameters that are included in the expression for the rating function. In contrast to this approach, in the case of using neural networks, there is no need to specify the general form for the rating function. Instead, certain neural network architecture is chosen and parameters are calculated for it on the basis of statistical data. Importantly, the same neural network architecture can be used to process different sets of statistical data. The disadvantages of using neural networks include the need to calculate a large number of parameters. There is also no universal algorithm that would determine the optimal neural network architecture. As an example of the use of neural networks to determine the borrower's rating, a model system is considered, in which the borrower's rating is determined by a known non-analytical rating function. A neural network with two inner layers, which contain, respectively, three and two neurons and have a sigmoid activation function, is used for modeling. It is shown that the use of the neural network allows restoring the borrower's rating function with quite acceptable accuracy.

Sign in / Sign up

Export Citation Format

Share Document