scholarly journals A General Technique to Train Language Models on Language Models

2005 ◽  
Vol 31 (2) ◽  
pp. 173-185 ◽  
Author(s):  
Mark-Jan Nederhof
Keyword(s):  
Second Language ◽  
Finite Automaton ◽  
Language Model ◽  
Language Models ◽  
General Technique ◽  
Context Free Grammar ◽  
N Gram ◽  
Context Free ◽  
Probabilistic Context

We show that under certain conditions, a language model can be trained on the basis of a second language model. The main instance of the technique trains a finite automaton on the basis of a probabilistic context-free grammar, such that the Kullback-Leibler distance between grammar and trained automaton is provably minimal. This is a substantial generalization of an existing algorithm to train an n-gram model on the basis of a probabilistic context-free grammar.

scholarly journals Searching for universal model of amyloid signaling motifs using probabilistic context-free grammars

2021 ◽  
Vol 22 (1) ◽  
Author(s):  
Witold Dyrka ◽  
Marlena Gąsior-Głogowska ◽  
Monika Szefczyk ◽  
Natalia Szulc
Keyword(s):  
Functional Relationship ◽  
High Sensitivity ◽  
Alternative Methods ◽  
Discriminative Power ◽  
Context Free Grammar ◽  
Protein Motifs ◽  
Functional Features ◽  
Universal Grammars ◽  
Context Free ◽  
Probabilistic Context

Abstract Background Amyloid signaling motifs are a class of protein motifs which share basic structural and functional features despite the lack of clear sequence homology. They are hard to detect in large sequence databases either with the alignment-based profile methods (due to short length and diversity) or with generic amyloid- and prion-finding tools (due to insufficient discriminative power). We propose to address the challenge with a machine learning grammatical model capable of generalizing over diverse collections of unaligned yet related motifs. Results First, we introduce and test improvements to our probabilistic context-free grammar framework for protein sequences that allow for inferring more sophisticated models achieving high sensitivity at low false positive rates. Then, we infer universal grammars for a collection of recently identified bacterial amyloid signaling motifs and demonstrate that the method is capable of generalizing by successfully searching for related motifs in fungi. The results are compared to available alternative methods. Finally, we conduct spectroscopy and staining analyses of selected peptides to verify their structural and functional relationship. Conclusions While the profile HMMs remain the method of choice for modeling homologous sets of sequences, PCFGs seem more suitable for building meta-family descriptors and extrapolating beyond the seed sample.

scholarly journals Knowledge Sources for Constituent Parsing of German, a Morphologically Rich and Less-Configurational Language

2013 ◽  
Vol 39 (1) ◽  
pp. 57-85 ◽  
Author(s):  
Alexander Fraser ◽  
Helmut Schmid ◽  
Richárd Farkas ◽  
Renjing Wang ◽  
Hinrich Schütze
Keyword(s):  
State Of The Art ◽  
Lessons Learned ◽  
Knowledge Sources ◽  
Lexical Knowledge ◽  
Context Free Grammar ◽  
The Impact ◽  
Context Free ◽  
Probabilistic Context

We study constituent parsing of German, a morphologically rich and less-configurational language. We use a probabilistic context-free grammar treebank grammar that has been adapted to the morphologically rich properties of German by markovization and special features added to its productions. We evaluate the impact of adding lexical knowledge. Then we examine both monolingual and bilingual approaches to parse reranking. Our reranking parser is the new state of the art in constituency parsing of the TIGER Treebank. We perform an analysis, concluding with lessons learned, which apply to parsing other morphologically rich and less-configurational languages.

INTEGRATION OF n-GRAM LANGUAGE MODELS IN MULTIPLE CLASSIFIER SYSTEMS FOR OFFLINE HANDWRITTEN TEXT LINE RECOGNITION

2008 ◽  
Vol 22 (07) ◽  
pp. 1301-1321 ◽  
Author(s):  
ROMAN BERTOLAMI ◽  
HORST BUNKE
Keyword(s):  
Language Model ◽  
Language Models ◽  
Combination Method ◽  
Text Line ◽  
Multiple Classifier Systems ◽  
Classifier Systems ◽  
Handwritten Text ◽  
Handwritten Text Recognition ◽  
Multiple Classifier ◽  
N Gram

Current multiple classifier systems for unconstrained handwritten text recognition do not provide a straightforward way to utilize language model information. In this paper, we describe a generic method to integrate a statistical n-gram language model into the combination of multiple offline handwritten text line recognizers. The proposed method first builds a word transition network and then rescores this network with an n-gram language model. Experimental evaluation conducted on a large dataset of offline handwritten text lines shows that the proposed approach improves the recognition accuracy over a reference system as well as over the original combination method that does not include a language model.

scholarly journals Pushdown Automata in Statistical Machine Translation

2014 ◽  
Vol 40 (3) ◽  
pp. 687-723 ◽  
Author(s):  
Cyril Allauzen ◽  
Bill Byrne ◽  
Adrià de Gispert ◽  
Gonzalo Iglesias ◽  
Michael Riley
Keyword(s):  
Machine Translation ◽  
Large Scale ◽  
Complexity Analysis ◽  
Statistical Machine Translation ◽  
Language Model ◽  
General Purpose ◽  
Language Models ◽  
Experimental Conditions ◽  
Context Free ◽  
Pushdown Automata

This article describes the use of pushdown automata (PDA) in the context of statistical machine translation and alignment under a synchronous context-free grammar. We use PDAs to compactly represent the space of candidate translations generated by the grammar when applied to an input sentence. General-purpose PDA algorithms for replacement, composition, shortest path, and expansion are presented. We describe HiPDT, a hierarchical phrase-based decoder using the PDA representation and these algorithms. We contrast the complexity of this decoder with a decoder based on a finite state automata representation, showing that PDAs provide a more suitable framework to achieve exact decoding for larger synchronous context-free grammars and smaller language models. We assess this experimentally on a large-scale Chinese-to-English alignment and translation task. In translation, we propose a two-pass decoding strategy involving a weaker language model in the first-pass to address the results of PDA complexity analysis. We study in depth the experimental conditions and tradeoffs in which HiPDT can achieve state-of-the-art performance for large-scale SMT.

Sign in / Sign up

Export Citation Format

Share Document