Learning Morphology of Natural Language as a Finite-State Grammar

2017 ◽  
pp. 44-57
Author(s):  
Javad Nouri ◽  
Roman Yangarber
Keyword(s):  
Natural Language ◽  
Finite State ◽  
State Grammar ◽  
Finite State Grammar

Finite State Automata and Simple Recurrent Networks

1989 ◽  
Vol 1 (3) ◽  
pp. 372-381 ◽  
Author(s):  
Axel Cleeremans ◽  
David Servan-Schreiber ◽  
James L. McClelland
Keyword(s):  
Natural Language ◽  
Network Architecture ◽  
Intermediate Step ◽  
Recurrent Networks ◽  
Time Step ◽  
Long Distance ◽  
Finite State ◽  
Simple Recurrent Networks ◽  
State Grammar ◽  
Finite State Grammar

We explore a network architecture introduced by Elman (1988) for predicting successive elements of a sequence. The network uses the pattern of activation over a set of hidden units from time-step t−1, together with element t, to predict element t + 1. When the network is trained with strings from a particular finite-state grammar, it can learn to be a perfect finite-state recognizer for the grammar. When the network has a minimal number of hidden units, patterns on the hidden units come to correspond to the nodes of the grammar, although this correspondence is not necessary for the network to act as a perfect finite-state recognizer. We explore the conditions under which the network can carry information about distant sequential contingencies across intervening elements. Such information is maintained with relative ease if it is relevant at each intermediate step; it tends to be lost when intervening elements do not depend on it. At first glance this may suggest that such networks are not relevant to natural language, in which dependencies may span indefinite distances. However, embeddings in natural language are not completely independent of earlier information. The final simulation shows that long distance sequential contingencies can be encoded by the network even if only subtle statistical properties of embedded strings depend on the early information.

scholarly journals Hierarchical and Linear Sequence Processing: An Electrophysiological Exploration of Two Different Grammar Types

2006 ◽  
Vol 18 (11) ◽  
pp. 1829-1842 ◽  
Author(s):  
Jörg Bahlmann ◽  
Thomas C. Gunter ◽  
Angela D. Friederici
Keyword(s):  
Phrase Structure ◽  
Phrase Structure Grammar ◽  
Brain Potentials ◽  
Late Positivity ◽  
Sequence Processing ◽  
Linear Sequence ◽  
Artificial Grammars ◽  
Finite State ◽  
State Grammar ◽  
Finite State Grammar

The present study investigated the processing of two types of artificial grammars by means of event-related brain potentials. Two categories of meaningless CV syllables were applied in each grammar type. The two grammars differed with regard to the type of the underlying rule. The finite-state grammar (FSG) followed the rule (AB)n, thereby generating local transitions between As and Bs (e.g., n = 2, ABAB). The phrase structure grammar (PSG) followed the rule AnBn, thereby generating center-embedded structures in which the first A and the last B embed the middle elements (e.g., n = 2, [A[AB]B]). Two sequence lengths (n = 2, n = 4) were used. Violations of the structures were introduced at different positions of the syllable sequences. Early violations were situated at the beginning of a sequence, and late violations were placed at the end of a sequence. A posteriorly distributed early negativity elicited by violations was present only in FSG. This effect was interpreted as the possible reflection of a violated local expectancy. Moreover, both grammar-type violations elicited a late positivity. This positivity varied as a function of the violation position in PSG, but not in FSG. These findings suggest that the late positivity could reflect difficulty of integration in PSG sequences.

Abstraction Processes in Artificial Grammar Learning

1997 ◽  
Vol 50 (1) ◽  
pp. 216-252 ◽  
Author(s):  
David R. Shanks ◽  
Theresa Johnstone ◽  
Leo Staggs
Keyword(s):  
Incidental Learning ◽  
Memory Task ◽  
Artificial Grammar ◽  
Grammar Learning ◽  
Abstract Knowledge ◽  
Learning Procedure ◽  
Finite State ◽  
Surface Similarity ◽  
State Grammar ◽  
Finite State Grammar

Four experiments explored the extent to which abstract knowledge may underlie subjects’ performance when asked to judge the grammaticality of letter strings generated from an artificial grammar. In Experiments 1 and 2 subjects studied grammatical strings instantiated with one set of letters and were then tested on grammatical and ungrammatical strings formed either from the same or a changed letter-set. Even with a change of letter-set, subjects were found to be sensitive to a variety of violations of the grammar. In Experiments 3 and 4, the critical manipulation involved the way in which the training strings were studied: an incidental learning procedure was used for some subjects, and others engaged in an explicit code-breaking task to try to learn the rules of the grammar. When strings were generated from a biconditional (Experiment 4) but not from a standard finite-state grammar (Experiment 3), grammaticality judgements for test strings were independent of their surface similarity to specific studied strings. Overall, the results suggest that transfer in this simple memory task is mediated at least to some extent by abstract knowledge.

Sign in / Sign up

Export Citation Format

Share Document