Part of Speech (PoS) Tagging for Konkani Language Using HMM

2022 ◽  
pp. 601-609
Author(s):  
Annie Rajan ◽  
Ambuja Salgaonkar
Keyword(s):  
Pos Tagging ◽  
Part Of Speech

Lexical Rule and Lexicon Effect for Part of Speech Tagging Bahasa Madura

2018 ◽  
Vol 18 (1) ◽  
pp. 65-72
Author(s):  
Nindian Puspa Dewi ◽  
Ubaidi Ubaidi
Keyword(s):  
Text Processing ◽  
Part Of Speech Tagging ◽  
Pos Tagging ◽  
Part Of Speech ◽  
Speech Tagging ◽  
Bahasa Indonesia

POS Tagging adalah dasar untuk pengembangan Text Processing suatu bahasa. Dalam penelitian ini kita meneliti pengaruh penggunaan lexicon dan perubahan morfologi kata dalam penentuan tagset yang tepat untuk suatu kata. Aturan dengan pendekatan morfologi kata seperti awalan, akhiran, dan sisipan biasa disebut sebagai lexical rule. Penelitian ini menerapkan lexical rule hasil learner dengan menggunakan algoritma Brill Tagger. Bahasa Madura adalah bahasa daerah yang digunakan di Pulau Madura dan beberapa pulau lainnya di Jawa Timur. Objek penelitian ini menggunakan Bahasa Madura yang memiliki banyak sekali variasi afiksasi dibandingkan dengan Bahasa Indonesia. Pada penelitian ini, lexicon selain digunakan untuk pencarian kata dasar Bahasa Madura juga digunakan sebagai salah satu tahap pemberian POS Tagging. Hasil ujicoba dengan menggunakan lexicon mencapai akurasi yaitu 86.61% sedangkan jika tidak menggunakan lexicon hanya mencapai akurasi 28.95 %. Dari sini dapat disimpulkan bahwa ternyata lexicon sangat berpengaruh terhadap POS Tagging.

scholarly journals Generation of Cross-Lingual Word Vectors for Low-Resourced Languages Using Deep Learning and Topological Metrics in a Data-Efficient Way

Electronics ◽  
2021 ◽  
Vol 10 (12) ◽  
pp. 1372
Author(s):  
Sanjanasri JP ◽  
Vijay Krishna Menon ◽  
Soman KP ◽  
Rajendran S ◽  
Agnieszka Wolk
Keyword(s):  
Deep Learning ◽  
Language Processing ◽  
Semantic Space ◽  
Semantic Interpretation ◽  
Learning Approaches ◽  
Qualitative Comparison ◽  
Bilingual Dictionary ◽  
Pos Tagging ◽  
Part Of Speech ◽  
Cross Lingual

Linguists have been focused on a qualitative comparison of the semantics from different languages. Evaluation of the semantic interpretation among disparate language pairs like English and Tamil is an even more formidable task than for Slavic languages. The concept of word embedding in Natural Language Processing (NLP) has enabled a felicitous opportunity to quantify linguistic semantics. Multi-lingual tasks can be performed by projecting the word embeddings of one language onto the semantic space of the other. This research presents a suite of data-efficient deep learning approaches to deduce the transfer function from the embedding space of English to that of Tamil, deploying three popular embedding algorithms: Word2Vec, GloVe and FastText. A novel evaluation paradigm was devised for the generation of embeddings to assess their effectiveness, using the original embeddings as ground truths. Transferability across other target languages of the proposed model was assessed via pre-trained Word2Vec embeddings from Hindi and Chinese languages. We empirically prove that with a bilingual dictionary of a thousand words and a corresponding small monolingual target (Tamil) corpus, useful embeddings can be generated by transfer learning from a well-trained source (English) embedding. Furthermore, we demonstrate the usability of generated target embeddings in a few NLP use-case tasks, such as text summarization, part-of-speech (POS) tagging, and bilingual dictionary induction (BDI), bearing in mind that those are not the only possible applications.

scholarly journals Research on comment target extracting in Chinese online shopping platform

2018 ◽  
Vol 2 (3) ◽  
pp. 247-258
Author(s):  
Zhishuo Liu ◽  
Qianhui Shen ◽  
Jingmiao Ma ◽  
Ziqi Dong
Keyword(s):  
Association Rule ◽  
Online Shopping ◽  
Low Frequency ◽  
Online Reviews ◽  
Content Type ◽  
Pos Tagging ◽  
Part Of Speech ◽  
Frequency Feature ◽  
Practical Implications ◽  
Data Reading

Purpose This paper aims to extract the comment targets in Chinese online shopping platform. Design/methodology/approach The authors first collect the comment texts, word segmentation, part-of-speech (POS) tagging and extracted feature words twice. Then they cluster the evaluation sentence and find the association rules between the evaluation words and the evaluation object. At the same time, they establish the association rule table. Finally, the authors can mine the evaluation object of comment sentence according to the evaluation word and the association rule table. At last, they obtain comment data from Taobao and demonstrate that the method proposed in this paper is effective by experiment. Findings The extracting comment target method the authors proposed in this paper is effective. Research limitations/implications First, the study object of extracting implicit features is review clauses, and not considering the context information, which may affect the accuracy of the feature excavation to a certain degree. Second, when extracting feature words, the low-frequency feature words are not considered, but some low-frequency feature words also contain effective information. Practical implications Because of the mass online reviews data, reading every comment one by one is impossible. Therefore, it is important that research on handling product comments and present useful or interest comments for clients. Originality/value The extracting comment target method the authors proposed in this paper is effective.

scholarly journals A Cascaded Unsupervised Model for PoS Tagging

2021 ◽  
Vol 20 (1) ◽  
pp. 1-23
Author(s):  
Necva Bölücü ◽  
Burcu Can
Keyword(s):  
Linear Model ◽  
Language Processing ◽  
Bayesian Model ◽  
Linear Models ◽  
Syntactic Category ◽  
Semantic Parsing ◽  
Pos Tagging ◽  
Part Of Speech ◽  
Sentence Level ◽  
Log Linear

Part of speech (PoS) tagging is one of the fundamental syntactic tasks in Natural Language Processing, as it assigns a syntactic category to each word within a given sentence or context (such as noun, verb, adjective, etc.). Those syntactic categories could be used to further analyze the sentence-level syntax (e.g., dependency parsing) and thereby extract the meaning of the sentence (e.g., semantic parsing). Various methods have been proposed for learning PoS tags in an unsupervised setting without using any annotated corpora. One of the widely used methods for the tagging problem is log-linear models. Initialization of the parameters in a log-linear model is very crucial for the inference. Different initialization techniques have been used so far. In this work, we present a log-linear model for PoS tagging that uses another fully unsupervised Bayesian model to initialize the parameters of the model in a cascaded framework. Therefore, we transfer some knowledge between two different unsupervised models to leverage the PoS tagging results, where a log-linear model benefits from a Bayesian model’s expertise. We present results for Turkish as a morphologically rich language and for English as a comparably morphologically poor language in a fully unsupervised framework. The results show that our framework outperforms other unsupervised models proposed for PoS tagging.

Towards a Historical Treebank of Middle and Early Modern Welsh, Part I: Workflow and POS Tagging

2021 ◽  
Vol 22 (1) ◽  
pp. 125-154
Author(s):  
Marieke Meelen ◽  
David Willis
Keyword(s):  
Early Modern ◽  
Pos Tagging ◽  
Part Of Speech ◽  
European Languages ◽  
Welsh Language ◽  
Word Boundaries ◽  
Exhaustive Extraction ◽  
Grammatical Structures

This article introduces the working methods of the Parsed Historical Corpus of the Welsh Language (PARSHCWL). The corpus is designed to provide researchers with a tool for automatic exhaustive extraction of instances of grammatical structures from Middle and Modern Welsh texts in a way comparable to similar tools that already exist for various European languages. The major features of the corpus are outlined, along with the overall architecture of the workflow needed for a team of researchers to produce it. In this paper, the two first stages of the process, namely pre-processing of texts and automated part-of-speech (POS) tagging are discussed in some detail, focusing in particular on major issues involved in defining word boundaries and in defining a robust and useful tagset.

scholarly journals Part-of-Speech Tagging via Deep Neural Networks for Northern-Ethiopic Languages

2020 ◽  
Vol 49 (4) ◽  
pp. 482-494
Author(s):  
Jurgita Kapočiūtė-Dzikienė ◽  
Senait Gebremichael Tesfagergish
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Language Processing ◽  
Deep Neural Networks ◽  
Short Term Memory ◽  
Parameter Tuning ◽  
Feed Forward Neural Network ◽  
Pos Tagging ◽  
Part Of Speech ◽  
Pos Tagger

Deep Neural Networks (DNNs) have proven to be especially successful in the area of Natural Language Processing (NLP) and Part-Of-Speech (POS) tagging—which is the process of mapping words to their corresponding POS labels depending on the context. Despite recent development of language technologies, low-resourced languages (such as an East African Tigrinya language), have received too little attention. We investigate the effectiveness of Deep Learning (DL) solutions for the low-resourced Tigrinya language of the Northern-Ethiopic branch. We have selected Tigrinya as the testbed example and have tested state-of-the-art DL approaches seeking to build the most accurate POS tagger. We have evaluated DNN classifiers (Feed Forward Neural Network – FFNN, Long Short-Term Memory method – LSTM, Bidirectional LSTM, and Convolutional Neural Network – CNN) on a top of neural word2vec word embeddings with a small training corpus known as Nagaoka Tigrinya Corpus. To determine the best DNN classifier type, its architecture and hyper-parameter set both manual and automatic hyper-parameter tuning has been performed. BiLSTM method was proved to be the most suitable for our solving task: it achieved the highest accuracy equal to 92% that is 65% above the random baseline.

scholarly journals Pengaruh Part of Speech Tagging Berbasis Aturan dan Distribusi Probabilitas Maximum Entropy untuk Bahasa Jawa Krama

2016 ◽  
Vol 7 (4) ◽  
Author(s):  
Hafiz Ridha Pramudita ◽  
Ema Utami ◽  
Armadyah Amborowati
Keyword(s):  
Maximum Entropy ◽  
Syntactic Category ◽  
Rule Based ◽  
Part Of Speech Tagging ◽  
Pos Tagging ◽  
Part Of Speech ◽  
Speech Tagging ◽  
Local Languages

Abstract. Javanese language is one of the local languages in Indonesia, which is used by most of the population of Indonesia. The language has complex grammar to embrace the values of decency that is determined by the use of words containing courtesy known as Raos Alus. Every word in the Javanese belongs to a certain part of speech like what happens to other languages. Part of Speech (POS) tagging is a process to set syntactic category in a word such as nouns, verbs, or adjectives to every word in the document or text. This study examined the POS Tagging with Maximum Entropy and Rule Based for Javanese Krama—Higher Javanese--by using the Open NLP library to measure the maximum entropy. The results obtained are Maximum Entropy and Rule Based can be used for POS Tagging on Javanese Krama with the highest accuracy of 97.67%.Keywords: POS Tagging, NLP, Maximum Entropy, Rule Based, Javanese Krama LanguageAbstrak. Bahasa Jawa merupakan salah satu bahasa daerah di Indonesia yang dipakai oleh sebagian besar penduduk Indonesia. Bahasa Jawa memiliki tata bahasa yang kompleks karena menganut nilai-nilai kesopanan yang ditentukan berdasarkan penggunaan dengan kata-kata yang mengandung raos alus (rasa sopan). Setiap kata dalam Bahasa Jawa memiliki jenis kata atau part of speech tertentu seperti halnya dengan bahasa-bahasa lain. POS tagging merupakah bagian penting dari cakupan bidang ilmu Natural Languange Processing (NLP). Penelitian ini menguji POS Tagging dengan Berbasis Aturan dan distribusi probabilitas Maximum Entropy pada Bahasa Jawa Krama menggunakan library OpenNLP untuk mengukur maximum entropy. Hasil yang diperoleh adalah Maximum Entropy dan Rule Based dapat digunakan untuk POSTagging pada Bahasa Jawa Krama dengan akurasi tertinggi 97,67%.Kata Kunci: POS Tagging, NLP, Maximum Entropy, Rule Based, Bahasa Jawa Krama

Chinese POS Disambiguation and Unknown Word Guessing with Lexicalized HMMs

2009 ◽  
pp. 1595-1607
Author(s):  
Guohong Fu ◽  
Kang-Kwong Luke
Keyword(s):  
Contextual Information ◽  
Word Formation ◽  
Morphological Features ◽  
Single Task ◽  
Unknown Word ◽  
Pos Tagging ◽  
Part Of Speech ◽  
Peking University ◽  
The Way

This article presents a lexicalized HMM-based approach to Chinese part-of-speech (POS) disambiguation and unknown word guessing (UWG). In order to explore word-internal morphological features for Chinese POS tagging, four types of pattern tags are defined to indicate the way lexicon words are used in a segmented sentence. Such patterns are combined further with POS tags. Thus, Chinese POS disambiguation and UWG can be unified as a single task of assigning each known word to input a proper hybrid tag. Furthermore, a uniformly lexicalized HMM-based tagger also is developed to perform this task, which can incorporate both internal word-formation patterns and surrounding contextual information for Chinese POS tagging under the framework of HMMs. Experiments on the Peking University Corpus indicate that the tagging precision can be improved with efficiency by the proposed approach.

A Tagging Approach to Extract Security Requirements in Non-Traditional Software Development Processes

2014 ◽  
Vol 5 (4) ◽  
pp. 31-47 ◽  
Author(s):  
Annette Tetmeyer ◽  
Daniel Hein ◽  
Hossein Saiedian
Keyword(s):  
Software Development ◽  
Software Security ◽  
Security Requirements ◽  
Software Systems ◽  
Small Organizations ◽  
Pos Tagging ◽  
Part Of Speech ◽  
Security Requirements Engineering ◽  
Development Processes

While software security has become an expectation, stakeholders often have difficulty expressing such expectations. Elaborate (and expensive) frameworks to identify, analyze, validate and incorporate security requirements for large software systems (and organizations) have been proposed, however, small organizations working within short development lifecycles and minimal resources cannot justify such frameworks and often need a light and practical approach to security requirements engineering that can be easily integrated into their existing development processes. This work presents an approach for eliciting, analyzing, prioritizing and developing security requirements which can be integrated into existing software development lifecycles for small organizations. The approach is based on identifying candidate security goals using part of speech (POS) tagging, categorizing security goals based on canonical security definitions, and understanding the stakeholder goals to develop preliminary security requirements and to prioritize them. It uses a case study to validate the feasibility and effectiveness of the proposed approach.

Part-of-speech tagging of Modern Hebrew text

2008 ◽  
Vol 14 (2) ◽  
pp. 223-251 ◽  
Author(s):  
ROY BAR-HAIM ◽  
KHALIL SIMA'AN ◽  
YOAD WINTER
Keyword(s):  
Markov Models ◽  
Full Generality ◽  
Semitic Languages ◽  
Part Of Speech Tagging ◽  
Pos Tagging ◽  
Part Of Speech ◽  
Word Level ◽  
Modern Hebrew ◽  
Architectural Decision ◽  
Pos Tagger

AbstractWords in Semitic texts often consist of a concatenation ofword segments, each corresponding to a part-of-speech (POS) category. Semitic words may be ambiguous with regard to their segmentation as well as to the POS tags assigned to each segment. When designing POS taggers for Semitic languages, a major architectural decision concerns the choice of the atomic input tokens (terminal symbols). If the tokenization is at the word level, the output tags must be complex, and represent both the segmentation of the word and the POS tag assigned to each word segment. If the tokenization is at the segment level, the input itself must encode the different alternative segmentations of the words, while the output consists of standard POS tags. Comparing these two alternatives is not trivial, as the choice between them may have global effects on the grammatical model. Moreover, intermediate levels of tokenization between these two extremes are conceivable, and, as we aim to show, beneficial. To the best of our knowledge, the problem of tokenization for POS tagging of Semitic languages has not been addressed before in full generality. In this paper, we study this problem for the purpose of POS tagging of Modern Hebrew texts. After extensive error analysis of the two simple tokenization models, we propose a novel, linguistically motivated, intermediate tokenization model that gives better performance for Hebrew over the two initial architectures. Our study is based on the well-known hidden Markov models (HMMs). We start out from a manually devised morphological analyzer and a very small annotated corpus, and describe how to adapt an HMM-based POS tagger for both tokenization architectures. We present an effective technique for smoothing the lexical probabilities using an untagged corpus, and a novel transformation for casting the segment-level tagger in terms of a standard, word-level HMM implementation. The results obtained using our model are on par with the best published results on Modern Standard Arabic, despite the much smaller annotated corpus available for Modern Hebrew.

Sign in / Sign up

Export Citation Format

Share Document