scholarly journals Modelling a subregular bias in phonological learning with Recurrent Neural Networks

2021 ◽  
Vol 9 (1) ◽  
Author(s):  
Brandon Prickett
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Formal Language ◽  
Formal Language Theory ◽  
Language Theory ◽  
Phonological Learning

A number of experiments have demonstrated what seems to be a bias in human phonological learning for patterns that are simpler according to Formal Language Theory (Finley and Badecker 2008; Lai 2015; Avcu 2018). This paper demonstrates that a sequence-to-sequence neural network (Sutskever et al. 2014), which has no such restriction explicitly built into its architecture, can successfully capture this bias. These results suggest that a bias for patterns that are simpler according to Formal Language Theory may not need to be explicitly incorporated into models of phonological learning.

scholarly journals An Entropy Metric for Regular Grammar Classification and Learning with Recurrent Neural Networks

Entropy ◽  
2021 ◽  
Vol 23 (1) ◽  
pp. 127
Author(s):  
Kaixuan Zhang ◽  
Qinglong Wang ◽  
C. Lee Giles
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Formal Language ◽  
Language Theory ◽  
Order Information ◽  
Concentric Ring ◽  
Original Order ◽  
Internal Structures ◽  
Regular Grammar ◽  
Learning Research

Recently, there has been a resurgence of formal language theory in deep learning research. However, most research focused on the more practical problems of attempting to represent symbolic knowledge by machine learning. In contrast, there has been limited research on exploring the fundamental connection between them. To obtain a better understanding of the internal structures of regular grammars and their corresponding complexity, we focus on categorizing regular grammars by using both theoretical analysis and empirical evidence. Specifically, motivated by the concentric ring representation, we relaxed the original order information and introduced an entropy metric for describing the complexity of different regular grammars. Based on the entropy metric, we categorized regular grammars into three disjoint subclasses: the polynomial, exponential and proportional classes. In addition, several classification theorems are provided for different representations of regular grammars. Our analysis was validated by examining the process of learning grammars with multiple recurrent neural networks. Our results show that as expected more complex grammars are generally more difficult to learn.

scholarly journals Recurrent Neural Networks for Narrowband Signal Detection in the Time-Frequency Domain

2004 ◽  
Vol 213 ◽  
pp. 483-486
Author(s):  
David Brodrick ◽  
Douglas Taylor ◽  
Joachim Diederich
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Signal Detection ◽  
Frequency Domain ◽  
Recurrent Neural Networks ◽  
Radio Frequency Interference ◽  
Recurrent Networks ◽  
Time Frequency ◽  
Narrowband Signal ◽  
Radio Signals

A recurrent neural network was trained to detect the time-frequency domain signature of narrowband radio signals against a background of astronomical noise. The objective was to investigate the use of recurrent networks for signal detection in the Search for Extra-Terrestrial Intelligence, though the problem is closely analogous to the detection of some classes of Radio Frequency Interference in radio astronomy.

THE DEVELOPMENT OF FORMAL LANGUAGE THEORY SINCE 1956

1990 ◽  
Vol 01 (04) ◽  
pp. 355-368
Author(s):  
ROBERT McNAUGHTON
Keyword(s):  
Formal Language ◽  
Formal Languages ◽  
Early Years ◽  
Formal Language Theory ◽  
Language Theory ◽  
Personal Interest

This brief survey will discuss the early years of the theory of formal languages through about 1970, treating only the most fundamental of the concepts. The paper will conclude with a brief discussion of a small number of topics, the choice reflecting only the personal interest of the author.

scholarly journals The interaction between structure and meaning in sentence comprehension: Recurrent neural networks and reading times

2019 ◽  
Author(s):  
Stefan L. Frank ◽  
John Hoeks
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Sentence Processing ◽  
Recurrent Neural Networks ◽  
Sentence Comprehension ◽  
Semantic Relations ◽  
Garden Path ◽  
Word Predictability ◽  
Syntactic Ambiguities ◽  
Potential Object

Recurrent neural network (RNN) models of sentence processing have recently displayed a remarkable ability to learn aspects of structure comprehension, as evidenced by their ability to account for reading times on sentences with local syntactic ambiguities (i.e., garden-path effects). Here, we investigate if these models can also simulate the effect of semantic appropriateness of the ambiguity's readings. RNNs-based estimates of surprisal of the disambiguating verb of sentences with an NP/S-coordination ambiguity (as in `The wizard guards the king and the princess protects ...') show identical patters to human reading times on the same sentences: Surprisal is higher on ambiguous structures than on their disambiguated counterparts and this effect is weaker, but not absent, in cases of poor thematic fit between the verb and its potential object (`The teacher baked the cake and the baker made ...'). These results show that an RNN is able to simultaneously learn about structural and semantic relations between words and suggest that garden-path phenomena may be more closely related to word predictability than traditionally assumed.

scholarly journals Binary and Multiclass Text Classification by Means of Separable Convolutional Neural Network

Inventions ◽  
2021 ◽  
Vol 6 (4) ◽  
pp. 70
Author(s):  
Elena Solovyeva ◽  
Ali Abdullah
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Network ◽  
Recurrent Neural Networks ◽  
Low Cost ◽  
Computational Cost ◽  
High Accuracy ◽  
Activation Functions ◽  
Fully Connected ◽  
Fully Connected Networks

In this paper, the structure of a separable convolutional neural network that consists of an embedding layer, separable convolutional layers, convolutional layer and global average pooling is represented for binary and multiclass text classifications. The advantage of the proposed structure is the absence of multiple fully connected layers, which is used to increase the classification accuracy but raises the computational cost. The combination of low-cost separable convolutional layers and a convolutional layer is proposed to gain high accuracy and, simultaneously, to reduce the complexity of neural classifiers. Advantages are demonstrated at binary and multiclass classifications of written texts by means of the proposed networks under the sigmoid and Softmax activation functions in convolutional layer. At binary and multiclass classifications, the accuracy obtained by separable convolutional neural networks is higher in comparison with some investigated types of recurrent neural networks and fully connected networks.

scholarly journals ELMAN-RECURRENT NEURAL NETWORK FOR LOAD SHEDDING OPTIMIZATION

SINERGI ◽  
2020 ◽  
Vol 24 (1) ◽  
pp. 29
Author(s):  
Widi Aribowo
Keyword(s):  
Neural Network ◽  
Genetic Algorithm ◽  
Neural Networks ◽  
Power System ◽  
Recurrent Neural Networks ◽  
Load Shedding ◽  
Feed Forward Neural Network ◽  
Linear Problems ◽  
Elman Recurrent Neural Network ◽  
And Control

Load shedding plays a key part in the avoidance of the power system outage. The frequency and voltage fluidity leads to the spread of a power system into sub-systems and leads to the outage as well as the severe breakdown of the system utility.  In recent years, Neural networks have been very victorious in several signal processing and control applications.  Recurrent Neural networks are capable of handling complex and non-linear problems. This paper provides an algorithm for load shedding using ELMAN Recurrent Neural Networks (RNN). Elman has proposed a partially RNN, where the feedforward connections are modifiable and the recurrent connections are fixed. The research is implemented in MATLAB and the performance is tested with a 6 bus system. The results are compared with the Genetic Algorithm (GA), Combining Genetic Algorithm with Feed Forward Neural Network (hybrid) and RNN. The proposed method is capable of assigning load releases needed and more efficient than other methods. 

Sign in / Sign up

Export Citation Format

Share Document