scholarly journals Neural network event forecasting for robots with continuous training

2020 ◽  
pp. 33-42
Author(s):  
Vasiliy Osipov ◽  
Dmitriy Miloserdov
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Arima Model ◽  
Urban Traffic ◽  
Network Structures ◽  
Intelligent Robots ◽  
Layer Structures ◽  
Event Forecasting ◽  
Network Event

Introduction: High hopes for a significant expansion of human capabilities in various fields of activity are pinned on the creation and use of highly intelligent robots. To achieve this level of robot intelligence, it is necessary to successfully solve the problems of predicting the external environment and the state of the robots themselves. Solutions based on recurrent neural networks with controlled elements are promising neural network forecasting systems. Purpose: Search for appropriate neural network structures for predicting events. Development of approaches to controlling the associative call of information from a neural network memory. Methods: Computer simulation of recurrent neural networks with controlled elements and various structures of layers. Results: An improved method of neural network event forecasting with continuous robot training has been developed. This method allows you to predict events on either long or short samples of time series. In order to improve the forecasting accuracy, new rules have been proposed for controlling the associative call of information from the neural network memory. A software system has been developed which implements the proposed method and supports the emulation of neural networks with various layer structures. The possibilities of recurrent neural networks with linear or spiral layer structures are analyzed using the example of urban traffic flow forecasting. The gain of the proposed method in comparison with the ARIMA model for the MAPE indicator is from 4.1 to 7.4%. Among the studied neural network structures, the spiral structures have shown the highest accuracy, and linear structures have shown the lowest accuracy. Practical relevance: The results of the study can be used to improve the accuracy of event forecasting for intelligent robots.

scholarly journals Recurrent Neural Networks for Narrowband Signal Detection in the Time-Frequency Domain

2004 ◽  
Vol 213 ◽  
pp. 483-486
Author(s):  
David Brodrick ◽  
Douglas Taylor ◽  
Joachim Diederich
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Signal Detection ◽  
Frequency Domain ◽  
Recurrent Neural Networks ◽  
Radio Frequency Interference ◽  
Recurrent Networks ◽  
Time Frequency ◽  
Narrowband Signal ◽  
Radio Signals

A recurrent neural network was trained to detect the time-frequency domain signature of narrowband radio signals against a background of astronomical noise. The objective was to investigate the use of recurrent networks for signal detection in the Search for Extra-Terrestrial Intelligence, though the problem is closely analogous to the detection of some classes of Radio Frequency Interference in radio astronomy.

scholarly journals The interaction between structure and meaning in sentence comprehension: Recurrent neural networks and reading times

2019 ◽  
Author(s):  
Stefan L. Frank ◽  
John Hoeks
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Sentence Processing ◽  
Recurrent Neural Networks ◽  
Sentence Comprehension ◽  
Semantic Relations ◽  
Garden Path ◽  
Word Predictability ◽  
Syntactic Ambiguities ◽  
Potential Object

Recurrent neural network (RNN) models of sentence processing have recently displayed a remarkable ability to learn aspects of structure comprehension, as evidenced by their ability to account for reading times on sentences with local syntactic ambiguities (i.e., garden-path effects). Here, we investigate if these models can also simulate the effect of semantic appropriateness of the ambiguity's readings. RNNs-based estimates of surprisal of the disambiguating verb of sentences with an NP/S-coordination ambiguity (as in `The wizard guards the king and the princess protects ...') show identical patters to human reading times on the same sentences: Surprisal is higher on ambiguous structures than on their disambiguated counterparts and this effect is weaker, but not absent, in cases of poor thematic fit between the verb and its potential object (`The teacher baked the cake and the baker made ...'). These results show that an RNN is able to simultaneously learn about structural and semantic relations between words and suggest that garden-path phenomena may be more closely related to word predictability than traditionally assumed.

scholarly journals Binary and Multiclass Text Classification by Means of Separable Convolutional Neural Network

Inventions ◽  
2021 ◽  
Vol 6 (4) ◽  
pp. 70
Author(s):  
Elena Solovyeva ◽  
Ali Abdullah
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Network ◽  
Recurrent Neural Networks ◽  
Low Cost ◽  
Computational Cost ◽  
High Accuracy ◽  
Activation Functions ◽  
Fully Connected ◽  
Fully Connected Networks

In this paper, the structure of a separable convolutional neural network that consists of an embedding layer, separable convolutional layers, convolutional layer and global average pooling is represented for binary and multiclass text classifications. The advantage of the proposed structure is the absence of multiple fully connected layers, which is used to increase the classification accuracy but raises the computational cost. The combination of low-cost separable convolutional layers and a convolutional layer is proposed to gain high accuracy and, simultaneously, to reduce the complexity of neural classifiers. Advantages are demonstrated at binary and multiclass classifications of written texts by means of the proposed networks under the sigmoid and Softmax activation functions in convolutional layer. At binary and multiclass classifications, the accuracy obtained by separable convolutional neural networks is higher in comparison with some investigated types of recurrent neural networks and fully connected networks.

scholarly journals ELMAN-RECURRENT NEURAL NETWORK FOR LOAD SHEDDING OPTIMIZATION

SINERGI ◽  
2020 ◽  
Vol 24 (1) ◽  
pp. 29
Author(s):  
Widi Aribowo
Keyword(s):  
Neural Network ◽  
Genetic Algorithm ◽  
Neural Networks ◽  
Power System ◽  
Recurrent Neural Networks ◽  
Load Shedding ◽  
Feed Forward Neural Network ◽  
Linear Problems ◽  
Elman Recurrent Neural Network ◽  
And Control

Load shedding plays a key part in the avoidance of the power system outage. The frequency and voltage fluidity leads to the spread of a power system into sub-systems and leads to the outage as well as the severe breakdown of the system utility.  In recent years, Neural networks have been very victorious in several signal processing and control applications.  Recurrent Neural networks are capable of handling complex and non-linear problems. This paper provides an algorithm for load shedding using ELMAN Recurrent Neural Networks (RNN). Elman has proposed a partially RNN, where the feedforward connections are modifiable and the recurrent connections are fixed. The research is implemented in MATLAB and the performance is tested with a 6 bus system. The results are compared with the Genetic Algorithm (GA), Combining Genetic Algorithm with Feed Forward Neural Network (hybrid) and RNN. The proposed method is capable of assigning load releases needed and more efficient than other methods. 

scholarly journals Seismic velocity estimation: A deep recurrent neural-network approach

Geophysics ◽  
2019 ◽  
Vol 85 (1) ◽  
pp. U21-U29
Author(s):  
Gabriel Fabien-Ouellet ◽  
Rahul Sarkar
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Model Building ◽  
Seismic Velocity ◽  
Velocity Model ◽  
Velocity Estimation ◽  
Interval Velocity ◽  
Velocity Model Building ◽  
3D Velocity Model

Applying deep learning to 3D velocity model building remains a challenge due to the sheer volume of data required to train large-scale artificial neural networks. Moreover, little is known about what types of network architectures are appropriate for such a complex task. To ease the development of a deep-learning approach for seismic velocity estimation, we have evaluated a simplified surrogate problem — the estimation of the root-mean-square (rms) and interval velocity in time from common-midpoint gathers — for 1D layered velocity models. We have developed a deep neural network, whose design was inspired by the information flow found in semblance analysis. The network replaces semblance estimation by a representation built with a deep convolutional neural network, and then it performs velocity estimation automatically with recurrent neural networks. The network is trained with synthetic data to identify primary reflection events, rms velocity, and interval velocity. For a synthetic test set containing 1D layered models, we find that rms and interval velocity are accurately estimated, with an error of less than [Formula: see text] for the rms velocity. We apply the neural network to a real 2D marine survey and obtain accurate rms velocity predictions leading to a coherent stacked section, in addition to an estimation of the interval velocity that reproduces the main structures in the stacked section. Our results provide strong evidence that neural networks can estimate velocity from seismic data and that good performance can be achieved on real data even if the training is based on synthetics. The findings for the 1D problem suggest that deep convolutional encoders and recurrent neural networks are promising components of more complex networks that can perform 2D and 3D velocity model building.

scholarly journals Exploring Efficient Neural Architectures for Linguistic–Acoustic Mapping in Text-To-Speech

2019 ◽  
Vol 9 (16) ◽  
pp. 3391 ◽  
Author(s):  
Santiago Pascual ◽  
Joan Serrà ◽  
Antonio Bonafonte
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Recurrent Neural Network ◽  
Recurrent Neural Networks ◽  
Affine Transformations ◽  
Text To Speech ◽  
Recursive Structure ◽  
The One ◽  
Acoustic Mapping ◽  
Symbol Sequences

Conversion from text to speech relies on the accurate mapping from linguistic to acoustic symbol sequences, for which current practice employs recurrent statistical models such as recurrent neural networks. Despite the good performance of such models (in terms of low distortion in the generated speech), their recursive structure with intermediate affine transformations tends to make them slow to train and to sample from. In this work, we explore two different mechanisms that enhance the operational efficiency of recurrent neural networks, and study their performance–speed trade-off. The first mechanism is based on the quasi-recurrent neural network, where expensive affine transformations are removed from temporal connections and placed only on feed-forward computational directions. The second mechanism includes a module based on the transformer decoder network, designed without recurrent connections but emulating them with attention and positioning codes. Our results show that the proposed decoder networks are competitive in terms of distortion when compared to a recurrent baseline, whilst being significantly faster in terms of CPU and GPU inference time. The best performing model is the one based on the quasi-recurrent mechanism, reaching the same level of naturalness as the recurrent neural network based model with a speedup of 11.2 on CPU and 3.3 on GPU.

Adaptive Capability of Recurrent Neural Networks with Fixed Weights for Series-Parallel System Identification

2009 ◽  
Vol 21 (11) ◽  
pp. 3214-3227
Author(s):  
James Ting-Ho Lo
Keyword(s):  
Neural Network ◽  
Dynamical System ◽  
Neural Networks ◽  
System Identification ◽  
Recurrent Neural Network ◽  
Recurrent Neural Networks ◽  
Additional Input ◽  
Adaptive Capability ◽  
Environmental Process ◽  
Optimal Series

By a fundamental neural filtering theorem, a recurrent neural network with fixed weights is known to be capable of adapting to an uncertain environment. This letter reports some mathematical results on the performance of such adaptation for series-parallel identification of a dynamical system as compared with the performance of the best series-parallel identifier possible under the assumption that the precise value of the uncertain environmental process is given. In short, if an uncertain environmental process is observable (not necessarily constant) from the output of a dynamical system or constant (not necessarily observable), then a recurrent neural network exists as a series-parallel identifier of the dynamical system whose output approaches the output of an optimal series-parallel identifier using the environmental process as an additional input.

A VECTOR MATRIX REAL TIME RECURSIVE BACKPROPAGATION ALGORITHM FOR RECURRENT NEURAL NETWORKS THAT APPROXIMATE MULTI-VALUED PERIODIC FUNCTIONS

2009 ◽  
Vol 08 (04) ◽  
pp. 395-411 ◽  
Author(s):  
PETER STUBBERUD
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Real Time ◽  
Recurrent Neural Networks ◽  
Feedforward Neural Networks ◽  
Universal Function ◽  
Multiple Delays ◽  
Feedback Path ◽  
Vector Matrix ◽  
Universal Approximators

Unlike feedforward neural networks (FFNN) which can act as universal function approximators, recursive, or recurrent, neural networks can act as universal approximators for multi-valued functions. In this paper, a real time recursive backpropagation (RTRBP) algorithm in a vector matrix form is developed for a two-layer globally recursive neural network that has multiple delays in its feedback path. This algorithm has been evaluated on two GRNNs that approximate both an analytic and nonanalytic periodic multi-valued function that a feedforward neural network is not capable of approximating.

Sign in / Sign up

Export Citation Format

Share Document