Unsupervised learning of part-of-speech guessing rules

1996 ◽  
Vol 2 (2) ◽  
pp. 111-136 ◽  
Author(s):  
ANDREI MIKHEEV
Keyword(s):  
Training Data ◽  
Evaluation Methodology ◽  
Parts Of Speech ◽  
Part Of Speech Tagging ◽  
Part Of Speech ◽  
Substantial Problem ◽  
Rule Sets ◽  
Induction Experiment ◽  
Unknown Words ◽  
The Impact

Words unknown to the lexicon present a substantial problem to part-of-speech tagging. In this paper we present a technique for fully unsupervised acquisition of rules which guess possible parts of speech for unknown words. This technique does not require specially prepared training data, and uses instead the lexicon supplied with a tagger and word frequencies collected from a raw corpus. Three complimentary sets of word-guessing rules are statistically induced: prefix morphological rules, suffix morphological rules and ending guessing rules. The acquisition process is strongly associated with guessing-rule evaluation methodology which is solely dedicated to the performance of part-of-speech guessers. Using the proposed technique a guessing-rule induction experiment was performed on the Brown Corpus data and rule-sets, with a highly competitive performance, were produced and compared with the state-of-the-art. To evaluate the impact of the word-guessing component on the overall tagging performance, it was integrated into a stochastic and a rule-based tagger and applied to texts with unknown words.

scholarly journals To Batch or Not to Batch? Comparing Batching and Curriculum Learning Strategies across Tasks and Datasets

Mathematics ◽  
2021 ◽  
Vol 9 (18) ◽  
pp. 2234
Author(s):  
Laura Burdick ◽  
Jonathan K. Kummerfeld ◽  
Rada Mihalcea
Keyword(s):  
Learning Strategies ◽  
Language Processing ◽  
Training Data ◽  
Systematic Analysis ◽  
Part Of Speech Tagging ◽  
Part Of Speech ◽  
Multiple Datasets ◽  
Processing Architectures ◽  
The Impact ◽  
Speech Tagging

Many natural language processing architectures are greatly affected by seemingly small design decisions, such as batching and curriculum learning (how the training data are ordered during training). In order to better understand the impact of these decisions, we present a systematic analysis of different curriculum learning strategies and different batching strategies. We consider multiple datasets for three tasks: text classification, sentence and phrase similarity, and part-of-speech tagging. Our experiments demonstrate that certain curriculum learning and batching decisions do increase performance substantially for some tasks.

scholarly journals The Impact of Arabic Part of Speech Tagging on Sentiment Analysis: A New Corpus and Deep Learning Approach

2021 ◽  
Vol 184 ◽  
pp. 148-155
Author(s):  
Abdul Munem Nerabie ◽  
Manar AlKhatib ◽  
Sujith Samuel Mathew ◽  
May El Barachi ◽  
Farhad Oroumchian
Keyword(s):  
Deep Learning ◽  
Sentiment Analysis ◽  
Learning Approach ◽  
Part Of Speech Tagging ◽  
Part Of Speech ◽  
The Impact ◽  
Speech Tagging

scholarly journals Learning Task-Specific Representation for Novel Words in Sequence Labeling

2019 ◽  
Author(s):  
Minlong Peng ◽  
Qi Zhang ◽  
Xiaoyu Xing ◽  
Tao Gui ◽  
Jinlan Fu ◽  
...  
Keyword(s):  
Empirical Studies ◽  
Named Entity Recognition ◽  
Learning Task ◽  
Training Data ◽  
Entity Recognition ◽  
Named Entity ◽  
Part Of Speech Tagging ◽  
Sequence Labeling ◽  
Part Of Speech ◽  
Word Representation

Word representation is a key component in neural-network-based sequence labeling systems. However, representations of unseen or rare words trained on the end task are usually poor for appreciable performance. This is commonly referred to as the out-of-vocabulary (OOV) problem. In this work, we address the OOV problem in sequence labeling using only training data of the task. To this end, we propose a novel method to predict representations for OOV words from their surface-forms (e.g., character sequence) and contexts. The method is specifically designed to avoid the error propagation problem suffered by existing approaches in the same paradigm. To evaluate its effectiveness, we performed extensive empirical studies on four part-of-speech tagging (POS) tasks and four named entity recognition (NER) tasks. Experimental results show that the proposed method can achieve better or competitive performance on the OOV problem compared with existing state-of-the-art methods.

Fuzzy network model for part-of-speech tagging under small training data

1996 ◽  
Vol 2 (2) ◽  
pp. 95-110 ◽  
Author(s):  
JAE-HOON KIM ◽  
GIL CHANG KIM
Keyword(s):  
Network Model ◽  
Hidden Markov ◽  
Training Data ◽  
Rule Based ◽  
Part Of Speech Tagging ◽  
Part Of Speech ◽  
Network Approaches ◽  
Fuzzy Network ◽  
Speech Tagging ◽  
Better Than

Recently, most part-of-speech tagging approaches, such as rule-based, probabilistic and neural network approaches, have shown very promising results. In this paper, we are particularly interested in probabilistic approaches, which usually require lots of training data to get reliable probabilities. We alleviate such a restriction of probabilistic approaches by introducing a fuzzy network model to provide a method for estimating more reliable parameters of a model under a small amount of training data. Experiments with the Brown corpus show that the performance of the fuzzy network model is much better than that of the hidden Markov model under a limited amount of training data.

scholarly journals FLORS: Fast and Simple Domain Adaptation for Part-of-Speech Tagging

2014 ◽  
Vol 2 ◽  
pp. 15-26 ◽  
Author(s):  
Tobias Schnabel ◽  
Hinrich Schütze
Keyword(s):  
Domain Adaptation ◽  
Part Of Speech Tagging ◽  
Part Of Speech ◽  
Unknown Words ◽  
Speech Tagging

We present FLORS, a new part-of-speech tagger for domain adaptation. FLORS uses robust representations that work especially well for unknown words and for known words with unseen tags. FLORS is simpler and faster than previous domain adaptation methods, yet it has significantly better accuracy than several baselines.

scholarly journals Modernising historical Slovene words

2015 ◽  
Vol 22 (6) ◽  
pp. 881-905 ◽  
Author(s):  
YVES SCHERRER ◽  
TOMAŽ ERJAVEC
Keyword(s):  
Statistical Machine Translation ◽  
Training Data ◽  
Original Text ◽  
Step Method ◽  
Part Of Speech Tagging ◽  
Text Annotation ◽  
Part Of Speech ◽  
Speech Tagging ◽  
Normalisation Method

AbstractWe propose a language-independent word normalisation method and exemplify it on modernising historical Slovene words. Our method relies on character-level statistical machine translation (CSMT) and uses only shallow knowledge. We present relevant data on historical Slovene, consisting of two (partially) manually annotated corpora and the lexicons derived from these corpora, containing historical word–modern word pairs. The two lexicons are disjoint, with one serving as the training set containing 40,000 entries, and the other as a test set with 20,000 entries. The data spans the years 1750–1900, and the lexicons are split into fifty-year slices, with all the experiments carried out separately on the three time periods. We perform two sets of experiments. In the first one – a supervised setting – we build a CSMT system using the lexicon of word pairs as training data. In the second one – an unsupervised setting – we simulate a scenario in which word pairs are not available. We propose a two-step method where we first extract a noisy list of word pairs by matching historical words with cognate modern words, and then train a CSMT system on these pairs. In both sets of experiments, we also optionally make use of a lexicon of modern words to filter the modernisation hypotheses. While we show that both methods produce significantly better results than the baselines, their accuracy and which method works best strongly correlates with the age of the texts, meaning that the choice of the best method will depend on the properties of the historical language which is to be modernised. As an extrinsic evaluation, we also compare the quality of part-of-speech tagging and lemmatisation directly on historical text and on its modernised words. We show that, depending on the age of the text, annotation on modernised words also produces significantly better results than annotation on the original text.

scholarly journals Pelabelan Kelas Kata Bahasa Jawa Menggunakan Hidden Markov Model

2020 ◽  
Vol 2 (2) ◽  
pp. 71-83
Author(s):  
Mohammad Mursyit ◽  
Aji Prasetya Wibawa ◽  
Ilham Ari Elbaith Zaeni ◽  
Harits Ar Rosyid
Keyword(s):  
Short Stories ◽  
Markov Model ◽  
Hidden Markov Model ◽  
Hidden Markov ◽  
Part Of Speech Tagging ◽  
Pos Tagging ◽  
Part Of Speech ◽  
Improve Accuracy ◽  
Unknown Words ◽  
Speech Tagging

Part of Speech Tagging atau POS Tagging adalah proses memberikan label pada setiap kata dalam sebuah kalimat secara otomatis. Penelitian ini menggunakan algoritma Hidden Markov Model (HMM) untuk proses POS Tagging. Perlakuan untuk unknown words menggunakan Most Probable POS-Tag. Dataset yang digunakan berupa 10 cerita pendek berbahasa Jawa terdiri dari 10.180 kata yang telah diberikan tagsetBahasa Jawa. Pada penelitian ini proses POS Tagging menggunakan dua skenario. Skenario pertama yaitu menggunakan algoritma Hidden Markov Model (HMM) tanpa menggunakan perlakuan untuk unknown words. Skenario yang kedua menggunakan HMM dan Most Probable POS-Tag untuk perlakuan unknown words. Hasil menunjukan skenario pertama menghasilkan akurasi sebesar 45.5% dan skenario kedua menghasilkan akurasi sebesar 70.78%. Most Probable POS-Tag dapat meningkatkan akurasi pada POS Tagging tetapi tidak selalu menunjukan hasil yang benar dalam pemberian label. Most Probable POS-Tag dapat menghilangkan probabilitas bernilai Nol dari POS Tagging Hidden Markov Model. Hasil penelitian ini menunjukan bahwa POS Tagging dengan menggunakan Hidden Markov Model dipengaruhi oleh perlakuan terhadap unknown words, perbendaharaan kata dan hubungan label kata pada dataset.  Part of Speech Tagging or POS Tagging is the process of automatically giving labels to each word in a sentence. This study uses the Hidden Markov Model (HMM) algorithm for the POS Tagging process. Treatment for unknown words uses the Most Probable POS-Tag. The dataset used is in the form of 10 short stories in Javanese consisting of 10,180 words which have been given the Javanese tagset. In this study, the POS Tagging process uses two scenarios. The first scenario is using the Hidden Markov Model (HMM) algorithm without using treatment for unknown words. The second scenario uses HMM and Most Probable POS-Tag for treatment of unknown words. The results show that the first scenario produces an accuracy of 45.5% and the second scenario produces an accuracy of 70.78%. Most Probable POS-Tag can improve accuracy in POS Tagging but does not always produce correct labels. Most Probable POS-Tag can remove zero-value probability from POS Tagging Hidden Markov Model. The results of this study indicate that POS Tagging using the Hidden Markov Model is influenced by the treatment of unknown words, vocabulary and word label relationships in the dataset.

scholarly journals Hidden Markov Based Mathematical Model dedicated to Extract Ingredients from Recipe Text

2019 ◽  
Author(s):  
Zied Baklouti
Keyword(s):  
Mathematical Model ◽  
Language Processing ◽  
Hidden Markov ◽  
Stochastic Methods ◽  
Part Of Speech Tagging ◽  
Pos Tagging ◽  
Part Of Speech ◽  
Unknown Words ◽  
High Level ◽  
Speech Tagging

Natural Language Processing (NLP) is a branch of machine learning that gives the machines the ability to decode human languages. Part-of-speech tagging (POS tagging) is a preprocessing task that requires an annotated corpora. Rule-based and stochastic methods showed great results for POS tag prediction. On this work, I performed a mathematical model based on Hidden Markov structures and I obtained a high level accuracy of ingredients extracted from text recipe which is a performance greater than what traditional methods could make without unknown words consideration.

Sign in / Sign up

Export Citation Format

Share Document