Part-of-Speech Tagging Using Long Short Term Memory (LSTM): Amazigh Text Written in Tifinaghe Characters

The need to capture intra-word information in natural language processing (NLP) tasks has inspired research in learning various word representations at word, character, or morpheme levels, but little attention has been given to syllables from a syllabic alphabet. Motivated by the success of compositional models in morphological languages, we present a Convolutional-long short term memory (Conv-LSTM) model for constructing Swahili word representation vectors from syllables. The unified architecture addresses the word agglutination and polysemous nature of Swahili by extracting high-level syllable features using a convolutional neural network (CNN) and then composes quality word embeddings with a long short term memory (LSTM). The word embeddings are then validated using a syllable-aware language model ( 31.267 ) and a part-of-speech (POS) tagging task ( 98.78 ), both yielding very competitive results to the state-of-art models in their respective domains. We further validate the language model using Xhosa and Shona, which are syllabic-based languages. The novelty of the study is in its capability to construct quality word embeddings from syllables using a hybrid model that does not use max-over-pool common in CNN and then the exploitation of these embeddings in POS tagging. Therefore, the study plays a crucial role in the processing of agglutinative and syllabic-based languages by contributing quality word embeddings from syllable embeddings, a robust Conv–LSTM model that learns syllables for not only language modeling and POS tagging, but also for other downstream NLP tasks.

Download Full-text

Towards Tokenization and Part-of-Speech Tagging for Khmer: Data and Discussion

ACM Transactions on Asian and Low-Resource Language Information Processing ◽

10.1145/3464378 ◽

2021 ◽

Vol 20 (6) ◽

pp. 1-16

Author(s):

Hour Kaing ◽

Chenchen Ding ◽

Masao Utiyama ◽

Eiichiro Sumita ◽

Sethserey Sam ◽

...

Keyword(s):

Short Term Memory ◽

Conditional Random Field ◽

Support Vector ◽

Part Of Speech Tagging ◽

Pos Tagging ◽

Part Of Speech ◽

Long Short Term Memory ◽

Syntactic Annotation ◽

Near Future ◽

Speech Tagging

As a highly analytic language, Khmer has considerable ambiguities in tokenization and part-of-speech (POS) tagging processing. This topic is investigated in this study. Specifically, a 20,000-sentence Khmer corpus with manual tokenization and POS-tagging annotation is released after a series of work over the last 4 years. This is the largest morphologically annotated Khmer dataset as of 2020, when this article was prepared. Based on the annotated data, experiments were conducted to establish a comprehensive benchmark on the automatic processing of tokenization and POS-tagging for Khmer. Specifically, a support vector machine, a conditional random field (CRF) , a long short-term memory (LSTM) -based recurrent neural network, and an integrated LSTM-CRF model have been investigated and discussed. As a primary conclusion, processing at morpheme-level is satisfactory for the provided data. However, it is intrinsically difficult to identify further grammatical constituents of compounds or phrases because of the complex analytic features of the language. Syntactic annotation and automatic parsing for Khmer will be scheduled in the near future.

Download Full-text

Part-of-Speech-Based Long Short-Term Memory Network for Learning Sentence Representations

IEEE Access ◽

10.1109/access.2019.2911983 ◽

2019 ◽

Vol 7 ◽

pp. 51810-51816 ◽

Cited By ~ 3

Author(s):

Wenhao Zhu ◽

Tengjun Yao ◽

Wu Zhang ◽

Baogang Wei

Keyword(s):

Short Term Memory ◽

Short Term ◽

Term Memory ◽

Part Of Speech ◽

Memory Network ◽

Long Short Term Memory

Download Full-text

Sleep Breathing Disorders Detection with Bioradar Using a Long Short-Term Memory Network

2020 XXXIIIrd General Assembly and Scientific Symposium of the International Union of Radio Science ◽

10.23919/ursigass49373.2020.9232203 ◽

2020 ◽

Author(s):

Lesya Anishchenko ◽

Ludmila Korostovtseva ◽

Mikhail Bochkarev ◽

Yurii Sviryaev

Keyword(s):

Short Term Memory ◽

Short Term ◽

Term Memory ◽

Sleep Breathing Disorders ◽

Breathing Disorders ◽

Memory Network ◽

Long Short Term Memory

Download Full-text

Incorporating Financial News for Forecasting Bitcoin Prices Based on Long Short-Term Memory Networks

SSRN Electronic Journal ◽

10.2139/ssrn.3733398 ◽

2020 ◽

Author(s):

Abdolreza Nazemi ◽

Johannes Jakubik ◽

Andreas Geyer-Schulz ◽

Frank J. Fabozzi

Keyword(s):

Short Term Memory ◽

Short Term ◽

Term Memory ◽

Financial News ◽

Long Short Term Memory

Download Full-text