scholarly journals Incremental Tree Substitution Grammar for Parsing and Sentence Prediction

2013 ◽  
Vol 1 ◽  
pp. 111-124
Author(s):  
Federico Sangati ◽  
Frank Keller
Keyword(s):  
Language Modeling ◽  
Incremental Processing ◽  
N Gram ◽  
Incremental Interpretation

In this paper, we present the first incremental parser for Tree Substitution Grammar (TSG). A TSG allows arbitrarily large syntactic fragments to be combined into complete trees; we show how constraints (including lexicalization) can be imposed on the shape of the TSG fragments to enable incremental processing. We propose an efficient Earley-based algorithm for incremental TSG parsing and report an F-score competitive with other incremental parsers. In addition to whole-sentence F-score, we also evaluate the partial trees that the parser constructs for sentence prefixes; partial trees play an important role in incremental interpretation, language modeling, and psycholinguistics. Unlike existing parsers, our incremental TSG parser can generate partial trees that include predictions about the upcoming words in a sentence. We show that it outperforms an n-gram model in predicting more than one upcoming word.

scholarly journals Profiling household appliance electricity usage with N-gram language modeling

2016 ◽  
Author(s):  
Daoyuan Li ◽  
Tegawende F. Bissyande ◽  
Sylvain Kubler ◽  
Jacques Klein ◽  
Yves Le Traon
Keyword(s):  
Language Modeling ◽  
Household Appliance ◽  
N Gram

Discriminative n-gram language modeling

2007 ◽  
Vol 21 (2) ◽  
pp. 373-392 ◽  
Author(s):  
Brian Roark ◽  
Murat Saraclar ◽  
Michael Collins
Keyword(s):  
Language Modeling ◽  
N Gram

Web Application for Romanian Language Phonetic Transcription

MACRo 2015 ◽  
2017 ◽  
Vol 2 (1) ◽  
pp. 1-10
Author(s):  
József Domokos ◽  
Zsolt Attila Szakács
Keyword(s):  
Web Service ◽  
Web Application ◽  
Language Model ◽  
Language Modeling ◽  
Phonetic Transcription ◽  
Finite State ◽  
Finite State Transducer ◽  
Model Training ◽  
N Gram ◽  
Pronunciation Dictionary

AbstractThis paper presents a Romanian language phonetic transcription web service and application built using Java technologies, on the top of the Phonetisaurus G2P, a Word Finite State Transducer (WFST)-driven Grapheme-to-Phoneme Conversion toolkit.We used NaviRO Romanian language pronunciation dictionary for WFST model training, and MIT Language Modeling (MITLM) toolkit to estimate the needed joint sequence n-gram language model.Dictionary evaluation tests are also included in the paper.The service can be accessed for educational, research and other non-commercial usage at http://users.utcluj.ro/~jdomokos/naviro/.

USING GRAPHEME n-GRAMS IN SPELLING CORRECTION AND AUGMENTATIVE TYPING SYSTEMS

2008 ◽  
Vol 04 (01) ◽  
pp. 87-106
Author(s):  
ALKET MEMUSHAJ ◽  
TAREK M. SOBH
Keyword(s):  
Natural Language Processing ◽  
Language Processing ◽  
Computational Efficiency ◽  
Probabilistic Models ◽  
Language Modeling ◽  
Language Models ◽  
Text Corpora ◽  
N Gram ◽  
Changes Over Time ◽  
Over Time

Probabilistic language models have gained popularity in Natural Language Processing due to their ability to successfully capture language structures and constraints with computational efficiency. Probabilistic language models are flexible and easily adapted to language changes over time as well as to some new languages. Probabilistic language models can be trained and their accuracy strongly related to the availability of large text corpora. In this paper, we investigate the usability of grapheme probabilistic models, specifically grapheme n-grams models in spellchecking as well as augmentative typing systems. Grapheme n-gram models require substantially smaller training corpora and that is one of the main drivers for this thesis in which we build grapheme n-gram language models for the Albanian language. There are presently no available Albanian language corpora to be used for probabilistic language modeling. Our technique attempts to augment spellchecking and typing systems by utilizing grapheme n-gram language models in improving suggestion accuracy in spellchecking and augmentative typing systems. Our technique can be implemented in a standalone tool or incorporated in another tool to offer additional selection/scoring criteria.

A study of n-gram and decision tree letter language modeling methods

1998 ◽  
Vol 24 (3) ◽  
pp. 171-192 ◽  
Author(s):  
Gerasimos Potamianos ◽  
Frederick Jelinek
Keyword(s):  
Decision Tree ◽  
Language Modeling ◽  
Modeling Methods ◽  
N Gram

SUITE OF TOOLS FOR STATISTICAL N-GRAM LANGUAGE MODELING FOR PATTERN MINING IN WHOLE GENOME SEQUENCES

2012 ◽  
Vol 10 (06) ◽  
pp. 1250016 ◽  
Author(s):  
MADHAVI K. GANAPATHIRAJU ◽  
ASIA D. MITCHELL ◽  
MOHAMED THAHIR ◽  
KAMIYA MOTWANI ◽  
SESHAN ANANTHASUBRAMANIAN
Keyword(s):  
Genome Sequence ◽  
Pattern Mining ◽  
Genomic Sequence ◽  
Repetitive Sequences ◽  
Language Modeling ◽  
Primary Component ◽  
Whole Genome Sequence ◽  
Whole Genome ◽  
Genome Sequences ◽  
N Gram

Genome sequences contain a number of patterns that have biomedical significance. Repetitive sequences of various kinds are a primary component of most of the genomic sequence patterns. We extended the suffix-array based Biological Language Modeling Toolkit to compute n-gram frequencies as well as n-gram language-model based perplexity in windows over the whole genome sequence to find biologically relevant patterns. We present the suite of tools and their application for analysis on whole human genome sequence.

Sign in / Sign up

Export Citation Format

Share Document