Simple Recurrent Networks Learn Context-Free and Context-Sensitive Languages by Counting

2001 ◽  
Vol 13 (9) ◽  
pp. 2093-2118 ◽  
Author(s):  
Paul Rodriguez
Keyword(s):  
Finite State Machine ◽  
Recurrent Network ◽  
Psychological Model ◽  
Storage Mechanism ◽  
Context Sensitive ◽  
Simple Recurrent Network ◽  
Finite State ◽  
Simple Recurrent Networks ◽  
Context Free ◽  
Free Language

It has been shown that if a recurrent neural network (RNN) learns to process a regular language, one can extract a finite-state machine (FSM) by treating regions of phase-space as FSM states. However, it has also been shown that one can construct an RNN to implement Turing machines by using RNN dynamics as counters. But how does a network learn languages that require counting? Rodriguez, Wiles, and Elman (1999) showed that a simple recurrent network (SRN) can learn to process a simple context-free language (CFL) by counting up and down. This article extends that to show a range of language tasks in which an SRN develops solutions that not only count but also copy and store counting information. In one case, the network stores information like an explicit storage mechanism. In other cases, the network stores information more indirectly in trajectories that are sensitive to slight displacements that depend on context. In this sense, an SRN can learn analog computation as a set of interdependent counters. This demonstrates how SRNs may be an alternative psychological model of language or sequence processing.

Learning Nonregular Languages: A Comparison of Simple Recurrent Networks and LSTM

2002 ◽  
Vol 14 (9) ◽  
pp. 2039-2041 ◽  
Author(s):  
J. Schmidhuber ◽  
F. Gers ◽  
D. Eck
Keyword(s):  
Recent Article ◽  
Short Term Memory ◽  
Recurrent Networks ◽  
Short Term ◽  
Term Memory ◽  
Context Sensitive ◽  
Simple Recurrent Networks ◽  
Long Short Term Memory ◽  
Context Free

In response to Rodriguez's recent article (2001), we compare the performance of simple recurrent nets and long short-term memory recurrent nets on context-free and context-sensitive languages.

scholarly journals Growth in Baumslag–Solitar groups I: subgroups and rationality

2011 ◽  
Vol 14 ◽  
pp. 34-71 ◽  
Author(s):  
Eric M. Freden ◽  
Teresa Knudson ◽  
Jennifer Schofield
Keyword(s):  
Turing Machine ◽  
Time Complexity ◽  
Machine Construction ◽  
Convolution Product ◽  
Growth Series ◽  
Context Sensitive ◽  
Context Free Language ◽  
Time And Space Complexity ◽  
Context Free ◽  
Free Language

AbstractThe computation of growth series for the higher Baumslag–Solitar groups is an open problem first posed by de la Harpe and Grigorchuk. We study the growth of the horocyclic subgroup as the key to the overall growth of these Baumslag–Solitar groups BS(p,q), where 1<p<q. In fact, the overall growth series can be represented as a modified convolution product with one of the factors being based on the series for the horocyclic subgroup. We exhibit two distinct algorithms that compute the growth of the horocyclic subgroup and discuss the time and space complexity of these algorithms. We show that when p divides q, the horocyclic subgroup has a geodesic combing whose words form a context-free (in fact, one-counter) language. A theorem of Chomsky–Schützenberger allows us to compute the growth series for this subgroup, which is rational. When p does not divide q, we show that no geodesic combing for the horocyclic subgroup forms a context-free language, although there is a context-sensitive geodesic combing. We exhibit a specific linearly bounded Turing machine that accepts this language (with quadratic time complexity) in the case of BS(2,3) and outline the Turing machine construction in the general case.

Eagle

2021 ◽  
Vol 30 (4) ◽  
pp. 1-46
Author(s):  
Jingbo Lu ◽  
Dongjie He ◽  
Jingling Xue
Keyword(s):  
Programming Languages ◽  
Object Oriented ◽  
Object Oriented Programming ◽  
Reachability Problem ◽  
Combinatorial Explosion ◽  
Call Graph ◽  
Context Sensitive ◽  
Significant Performance ◽  
Context Free ◽  
Free Language

Object sensitivity is widely used as a context abstraction for computing the points-to information context-sensitively for object-oriented programming languages such as Java. Due to the combinatorial explosion of contexts in large object-oriented programs, k -object-sensitive pointer analysis (under k -limiting), denoted k -obj , is often inefficient even when it is scalable for small values of k , where k ⩽ 2 holds typically. A recent popular approach for accelerating k -obj trades precision for efficiency by instructing k -obj to analyze only some methods in a program context-sensitively, determined heuristically by a pre-analysis. In this article, we investigate how to develop a fundamentally different approach, Eagle , for designing a pre-analysis that can make k -obj run significantly faster while maintaining its precision. The novelty of Eagle is to enable k -obj to analyze a method with partial context sensitivity (i.e., context-sensitively for only some of its selected variables/allocation sites) by solving a context-free-language (CFL) reachability problem based on a new CFL-reachability formulation of k -obj . By regularizing one CFL for specifying field accesses and using another CFL for specifying method calls, we have formulated Eagle as a fully context-sensitive taint analysis (without k -limiting) that is both effective (by selecting the variables/allocation sites to be analyzed by k -obj context-insensitively so as to reduce the number of context-sensitive facts inferred by k -obj in the program) and efficient (by running linearly in terms of the number of pointer assignment edges in the program). As Eagle represents the first precision-preserving pre-analysis, our evaluation focuses on demonstrating its significant performance benefits in accelerating k -obj for a set of popular Java benchmarks and applications, with call graph construction, may-fail-casting, and polymorphic call detection as three important client analyses.

Type-based flow analysis and context-free language reachability

2008 ◽  
Vol 18 (5) ◽  
pp. 823-894 ◽  
Author(s):  
MANUEL FÄHNDRICH ◽  
JAKOB REHOF
Keyword(s):  
Data Structures ◽  
Flow Analysis ◽  
Equivalence Classes ◽  
Dataflow Analysis ◽  
Context Sensitive ◽  
Context Free Language ◽  
Novel Approach ◽  
Asymptotic Complexity ◽  
Context Free ◽  
Free Language

We present a novel approach to computing the context-sensitive flow of values through procedures and data structures. Our approach combines and extends techniques from two seemingly disparate areas: polymorphic subtyping and interprocedural dataflow analysis based on context-free language reachability. The resulting technique offers several advantages over previous approaches: it works directly on higher-order programs; provides demand-driven interprocedural queries; and improves the asymptotic complexity of a known algorithm based on polymorphic subtyping fromO(n8) toO(n3) for computing all queries. For intra-procedural flow restricted to equivalence classes, our algorithm yields linear inter-procedural flow queries.

scholarly journals ON PERIODIC POINTS OF FREE INVERSE MONOID HOMOMORPHISMS

2013 ◽  
Vol 23 (08) ◽  
pp. 1789-1803 ◽  
Author(s):  
EMANUELE RODARO ◽  
PEDRO V. SILVA
Keyword(s):  
Fixed Point ◽  
Periodic Point ◽  
Periodic Points ◽  
Inverse Monoid ◽  
Finitely Generated ◽  
Context Sensitive ◽  
Context Free Language ◽  
Context Free ◽  
Free Language ◽  
Free Inverse Monoid

It is proved that the periodic point submonoid of a free inverse monoid endomorphism is always finitely generated. Using Chomsky's hierarchy of languages, we prove that the fixed point submonoid of an endomorphism of a free inverse monoid can be represented by a context-sensitive language but, in general, it cannot be represented by a context-free language.

scholarly journals Optimizing over subsequences generates context-sensitive languages

2021 ◽  
Vol 9 ◽  
pp. 528-537
Author(s):  
Andrew Lamont
Keyword(s):  
Optimality Theory ◽  
Generative Capacity ◽  
Context Sensitive ◽  
Finite State ◽  
Minimal Modification ◽  
Context Free

Abstract Phonological generalizations are finite-state. While Optimality Theory is a popular framework for modeling phonology, it is known to generate non-finite-state mappings and languages. This paper demonstrates that Optimality Theory is capable of generating non-context-free languages, contributing to the characterization of its generative capacity. This is achieved with minimal modification to the theory as it is standardly employed.

Nondeterministic Ordered Restarting Automata

2018 ◽  
Vol 29 (04) ◽  
pp. 663-685 ◽  
Author(s):  
Kent Kwee ◽  
Friedrich Otto
Keyword(s):  
Regular Languages ◽  
Context Sensitive ◽  
Finite State ◽  
Emptiness Problem ◽  
Context Free

While (stateless) deterministic ordered restarting automata accept exactly the regular languages, it has been observed that nondeterministic ordered restarting automata (ORWW-automata for short) are more expressive. Here we show that the class of languages accepted by the latter automata is an abstract family of languages that is incomparable to the linear, the context-free, and the growing context-sensitive languages with respect to inclusion, and that the emptiness problem is decidable for these languages. In addition, we give a construction that turns a stateless ORWW-automaton into a nondeterministic finite-state acceptor for the same language.

scholarly journals Extended finite state models of language

1996 ◽  
Vol 2 (4) ◽  
pp. 287-290 ◽  
Author(s):  
ANDRÁS KORNAI
Keyword(s):  
Artificial Intelligence ◽  
Text Analysis ◽  
Language Modeling ◽  
Context Sensitive ◽  
The Past ◽  
Finite State ◽  
Wide Availability ◽  
State Models ◽  
Finite State Models ◽  
Context Free

In spite of the wide availability of more powerful (context free, mildly context sensitive, and even Turing-equivalent) formalisms, the bulk of the applied work on language and sublanguage modeling, especially for the purposes of recognition and topic search, is still performed by various finite state methods. In fact, the use of such methods in research labs as well as in applied work actually increased in the past five years. To bring together those developing and using extended finite state methods to text analysis, speech/OCR language modeling, and related CL and NLP tasks with those in AI and CS interested in analyzing and possibly extending the domain of finite state algorithms, a workshop was held in August 1996 in Budapest as part of the European Conference on Artificial Intelligence (ECAI'96).

Sign in / Sign up

Export Citation Format

Share Document