scholarly journals Recurrent Neural Networks for Narrowband Signal Detection in the Time-Frequency Domain

2004 ◽  
Vol 213 ◽  
pp. 483-486
Author(s):  
David Brodrick ◽  
Douglas Taylor ◽  
Joachim Diederich
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Signal Detection ◽  
Frequency Domain ◽  
Recurrent Neural Networks ◽  
Radio Frequency Interference ◽  
Recurrent Networks ◽  
Time Frequency ◽  
Narrowband Signal ◽  
Radio Signals

A recurrent neural network was trained to detect the time-frequency domain signature of narrowband radio signals against a background of astronomical noise. The objective was to investigate the use of recurrent networks for signal detection in the Search for Extra-Terrestrial Intelligence, though the problem is closely analogous to the detection of some classes of Radio Frequency Interference in radio astronomy.

RECURRENT NEURAL NETWORKS FOR MUSICAL PITCH MEMORY AND CLASSIFICATION

2005 ◽  
Vol 14 (01n02) ◽  
pp. 329-342 ◽  
Author(s):  
JUDY A. FRANKLIN ◽  
KRYSTAL K. LOCKE
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Recurrent Neural Network ◽  
Recurrent Neural Networks ◽  
Short Term Memory ◽  
Recurrent Networks ◽  
Short Term ◽  
Term Memory ◽  
Musical Pitch ◽  
Long Short Term Memory

We present results from experiments in using several pitch representations for jazz-oriented musical tasks performed by a recurrent neural network. We have run experiments with several kinds of recurrent networks for this purpose, and have found that Long Short-term Memory networks provide the best results. We show that a new pitch representation called Circles of Thirds works as well as two other published representations for these tasks, yet it is more succinct and enables faster learning. We then discuss limited results using other types of networks on the same tasks.

scholarly journals Spatiotemporal Monitoring and Evaluation Method for Sand-Filling of Immersed Tube Tunnel Foundation

2021 ◽  
Vol 11 (3) ◽  
pp. 1084
Author(s):  
Peng Wu ◽  
Ailan Che
Keyword(s):  
Neural Network ◽  
Elastic Wave ◽  
Frequency Domain ◽  
Bp Neural Network ◽  
Evaluation Method ◽  
Monitoring And Evaluation ◽  
Strong Nonlinearity ◽  
Filling Process ◽  
Time Frequency ◽  
Feature Parameters

The sand-filling method has been widely used in immersed tube tunnel engineering. However, for the problem of monitoring during the sand-filling process, the traditional methods can be inadequate for evaluating the state of sand deposits in real-time. Based on the high efficiency of elastic wave monitoring, and the superiority of the backpropagation (BP) neural network on solving nonlinear problems, a spatiotemporal monitoring and evaluation method is proposed for the filling performance of foundation cushion. Elastic wave data were collected during the sand-filling process, and the waveform, frequency spectrum, and time–frequency features were analysed. The feature parameters of the elastic wave were characterized by the time domain, frequency domain, and time-frequency domain. By analysing the changes of feature parameters with the sand-filling process, the feature parameters exhibited dynamic and strong nonlinearity. The data of elastic wave feature parameters and the corresponding sand-filling state were trained to establish the evaluation model using the BP neural network. The accuracy of the trained network model reached 93%. The side holes and middle holes were classified and analysed, revealing the characteristics of the dynamic expansion of the sand deposit along the diffusion radius. The evaluation results are consistent with the pressure gauge monitoring data, indicating the effectiveness of the evaluation and monitoring model for the spatiotemporal performance of sand deposits. For the sand-filling and grouting engineering, the machine-learning method could offer a better solution for spatiotemporal monitoring and evaluation in a complex environment.

Recurrent Neural Networks with Small Weights Implement Definite Memory Machines

2003 ◽  
Vol 15 (8) ◽  
pp. 1897-1929 ◽  
Author(s):  
Barbara Hammer ◽  
Peter Tiňo
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Experimental Studies ◽  
Activation Function ◽  
Transition Function ◽  
Recurrent Network ◽  
Point Of View ◽  
Recurrent Networks ◽  
Contraction Parameter ◽  
Arbitrary Precision

Recent experimental studies indicate that recurrent neural networks initialized with “small” weights are inherently biased toward definite memory machines (Tiňno, Čerňanský, & Beňušková, 2002a, 2002b). This article establishes a theoretical counterpart: transition function of recurrent network with small weights and squashing activation function is a contraction. We prove that recurrent networks with contractive transition function can be approximated arbitrarily well on input sequences of unbounded length by a definite memory machine. Conversely, every definite memory machine can be simulated by a recurrent network with contractive transition function. Hence, initialization with small weights induces an architectural bias into learning with recurrent neural networks. This bias might have benefits from the point of view of statistical learning theory: it emphasizes one possible region of the weight space where generalization ability can be formally proved. It is well known that standard recurrent neural networks are not distribution independent learnable in the probably approximately correct (PAC) sense if arbitrary precision and inputs are considered. We prove that recurrent networks with contractive transition function with a fixed contraction parameter fulfill the so-called distribution independent uniform convergence of empirical distances property and hence, unlike general recurrent networks, are distribution independent PAC learnable.

scholarly journals The interaction between structure and meaning in sentence comprehension: Recurrent neural networks and reading times

2019 ◽  
Author(s):  
Stefan L. Frank ◽  
John Hoeks
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Sentence Processing ◽  
Recurrent Neural Networks ◽  
Sentence Comprehension ◽  
Semantic Relations ◽  
Garden Path ◽  
Word Predictability ◽  
Syntactic Ambiguities ◽  
Potential Object

Recurrent neural network (RNN) models of sentence processing have recently displayed a remarkable ability to learn aspects of structure comprehension, as evidenced by their ability to account for reading times on sentences with local syntactic ambiguities (i.e., garden-path effects). Here, we investigate if these models can also simulate the effect of semantic appropriateness of the ambiguity's readings. RNNs-based estimates of surprisal of the disambiguating verb of sentences with an NP/S-coordination ambiguity (as in `The wizard guards the king and the princess protects ...') show identical patters to human reading times on the same sentences: Surprisal is higher on ambiguous structures than on their disambiguated counterparts and this effect is weaker, but not absent, in cases of poor thematic fit between the verb and its potential object (`The teacher baked the cake and the baker made ...'). These results show that an RNN is able to simultaneously learn about structural and semantic relations between words and suggest that garden-path phenomena may be more closely related to word predictability than traditionally assumed.

scholarly journals Binary and Multiclass Text Classification by Means of Separable Convolutional Neural Network

Inventions ◽  
2021 ◽  
Vol 6 (4) ◽  
pp. 70
Author(s):  
Elena Solovyeva ◽  
Ali Abdullah
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Network ◽  
Recurrent Neural Networks ◽  
Low Cost ◽  
Computational Cost ◽  
High Accuracy ◽  
Activation Functions ◽  
Fully Connected ◽  
Fully Connected Networks

In this paper, the structure of a separable convolutional neural network that consists of an embedding layer, separable convolutional layers, convolutional layer and global average pooling is represented for binary and multiclass text classifications. The advantage of the proposed structure is the absence of multiple fully connected layers, which is used to increase the classification accuracy but raises the computational cost. The combination of low-cost separable convolutional layers and a convolutional layer is proposed to gain high accuracy and, simultaneously, to reduce the complexity of neural classifiers. Advantages are demonstrated at binary and multiclass classifications of written texts by means of the proposed networks under the sigmoid and Softmax activation functions in convolutional layer. At binary and multiclass classifications, the accuracy obtained by separable convolutional neural networks is higher in comparison with some investigated types of recurrent neural networks and fully connected networks.

scholarly journals ELMAN-RECURRENT NEURAL NETWORK FOR LOAD SHEDDING OPTIMIZATION

SINERGI ◽  
2020 ◽  
Vol 24 (1) ◽  
pp. 29
Author(s):  
Widi Aribowo
Keyword(s):  
Neural Network ◽  
Genetic Algorithm ◽  
Neural Networks ◽  
Power System ◽  
Recurrent Neural Networks ◽  
Load Shedding ◽  
Feed Forward Neural Network ◽  
Linear Problems ◽  
Elman Recurrent Neural Network ◽  
And Control

Load shedding plays a key part in the avoidance of the power system outage. The frequency and voltage fluidity leads to the spread of a power system into sub-systems and leads to the outage as well as the severe breakdown of the system utility.  In recent years, Neural networks have been very victorious in several signal processing and control applications.  Recurrent Neural networks are capable of handling complex and non-linear problems. This paper provides an algorithm for load shedding using ELMAN Recurrent Neural Networks (RNN). Elman has proposed a partially RNN, where the feedforward connections are modifiable and the recurrent connections are fixed. The research is implemented in MATLAB and the performance is tested with a 6 bus system. The results are compared with the Genetic Algorithm (GA), Combining Genetic Algorithm with Feed Forward Neural Network (hybrid) and RNN. The proposed method is capable of assigning load releases needed and more efficient than other methods. 

scholarly journals Seismic velocity estimation: A deep recurrent neural-network approach

Geophysics ◽  
2019 ◽  
Vol 85 (1) ◽  
pp. U21-U29
Author(s):  
Gabriel Fabien-Ouellet ◽  
Rahul Sarkar
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Model Building ◽  
Seismic Velocity ◽  
Velocity Model ◽  
Velocity Estimation ◽  
Interval Velocity ◽  
Velocity Model Building ◽  
3D Velocity Model

Applying deep learning to 3D velocity model building remains a challenge due to the sheer volume of data required to train large-scale artificial neural networks. Moreover, little is known about what types of network architectures are appropriate for such a complex task. To ease the development of a deep-learning approach for seismic velocity estimation, we have evaluated a simplified surrogate problem — the estimation of the root-mean-square (rms) and interval velocity in time from common-midpoint gathers — for 1D layered velocity models. We have developed a deep neural network, whose design was inspired by the information flow found in semblance analysis. The network replaces semblance estimation by a representation built with a deep convolutional neural network, and then it performs velocity estimation automatically with recurrent neural networks. The network is trained with synthetic data to identify primary reflection events, rms velocity, and interval velocity. For a synthetic test set containing 1D layered models, we find that rms and interval velocity are accurately estimated, with an error of less than [Formula: see text] for the rms velocity. We apply the neural network to a real 2D marine survey and obtain accurate rms velocity predictions leading to a coherent stacked section, in addition to an estimation of the interval velocity that reproduces the main structures in the stacked section. Our results provide strong evidence that neural networks can estimate velocity from seismic data and that good performance can be achieved on real data even if the training is based on synthetics. The findings for the 1D problem suggest that deep convolutional encoders and recurrent neural networks are promising components of more complex networks that can perform 2D and 3D velocity model building.

Sign in / Sign up

Export Citation Format

Share Document