An Entropy Metric for Regular Grammar Classification and Learning with Recurrent Neural Networks

Recently, there has been a resurgence of formal language theory in deep learning research. However, most research focused on the more practical problems of attempting to represent symbolic knowledge by machine learning. In contrast, there has been limited research on exploring the fundamental connection between them. To obtain a better understanding of the internal structures of regular grammars and their corresponding complexity, we focus on categorizing regular grammars by using both theoretical analysis and empirical evidence. Specifically, motivated by the concentric ring representation, we relaxed the original order information and introduced an entropy metric for describing the complexity of different regular grammars. Based on the entropy metric, we categorized regular grammars into three disjoint subclasses: the polynomial, exponential and proportional classes. In addition, several classification theorems are provided for different representations of regular grammars. Our analysis was validated by examining the process of learning grammars with multiple recurrent neural networks. Our results show that as expected more complex grammars are generally more difficult to learn.

Phraseology and Identity

In recent decades, phraseology has ceased to be a marginal philological discipline and has become an important area in both discourse analysis (“repeated” or “prefabricated” discourse) and applied linguistics (translation and language teaching). The fact that idiomaticity is one of its most characteristic features creates an added difficulty for translators and second language learners, since knowledge of lexicon and regular grammar is not sufficient for the communicative mastery of a language and for the socialization of the speakers in the speech community concerned. This is where sociolinguistics comes into play: the (passive and active) use of phraseological units (PU) or phrasemes becomes a relevant sociolinguistic factor as an index of collective identity in Peirce's terms and as an indicator, a marker, or a stereotype, according to Labov. Furthermore, the conservation of repertories of PU throughout history, and especially proverbs and other expressive formulas, is a symbol of the linguistic and cultural personality of the Catalan language community.

Differentiable Grammars for Videos

This paper proposes a novel algorithm which learns a formal regular grammar from real-world continuous data, such as videos. Learning latent terminals, non-terminals, and production rules directly from continuous data allows the construction of a generative model capturing sequential structures with multiple possibilities. Our model is fully differentiable, and provides easily interpretable results which are important in order to understand the learned structures. It outperforms the state-of-the-art on several challenging datasets and is more accurate for forecasting future activities in videos. We plan to open-source the code.1

Static Watson-Crick Linear Grammars and Its Computational Power

DNA computing, or more generally, molecular computing, is a recent development on computations using biological molecules, instead of the traditional silicon-chips. Some computational models which are based on different operations of DNA molecules have been developed by using the concept of formal language theory. The operations of DNA molecules inspire various types of formal language tools which include sticker systems, grammars and automata. Recently, the grammar counterparts of Watson-Crick automata known as Watson-Crick grammars which consist of regular, linear and context-free grammars, are defined as grammar models that generate double-stranded strings using the important feature of Watson-Crick complementarity rule. In this research, a new variant of static Watson-Crick linear grammar is introduced as an extension of static Watson-Crick regular grammar. A static Watson-Crick linear grammar is a grammar counterpart of sticker system that generates the double-stranded strings and uses rule as in linear grammar. The main result of the paper is to determine some computational properties of static Watson-Crick linear grammars. Next, the hierarchy between static Watson-Crick languages, Watson-Crick languages, Chomsky languages and families of languages generated by sticker systems are presented.

Static Watson-Crick regular grammar

DNA computing, or more generally, molecular computing, is a recent development at the interface of computer science and molecular biology. In DNA computing, many computational models have been proposed in the framework of formal language theory and automata such as Watson-Crick grammars and sticker systems. A Watson-Crick grammar is a grammar model that generates double stranded strings, whereas a sticker system is a DNA computing model of the ligation and annealing operations over DNA strands using the Watson-Crick complementarity to form a complete double stranded DNA sequence. Most of the proposed DNA computing models make use of this concept, including the Watson-Crick grammars and sticker systems. Watson-Crick grammars and their variants can be explored using formal language theory which allows the development of new concepts of Watson-Crick grammars. In this research, a new variant of Watson-Crick grammar called a static Watson-Crick regular grammar is introduced as an analytical counterpart of sticker systems. The computation of a sticker system starts from a given set of incomplete double stranded sequence to form a complete double stranded sequence. Here, a static Watson-Crick regular grammar differs from a dynamic Watson-Crick regular grammar in generating double stranded strings: the latter grammar produces each strand string “independently” and only check for the Watson-Crick complementarity of a generated complete double stranded string at the end, while the former grammar generates both strand strings “dependently”, i.e., checking for the Watson-Crick complementarity for each complete substring. In this paper, computational properties of static Watson-Crick regular grammars are investigated to correlate with the Chomsky hierarchy and hierarchy of the families of dynamic Watson-Crick regular languages. The relationship between families of languages generated by static Watson-Crick regular grammars with several variants of sticker systems, Watson-Crick regular grammars and Chomsky grammars are presented by showing the hierarchy.