scholarly journals Part-of-Speech Tagging via Deep Neural Networks for Northern-Ethiopic Languages

2020 ◽  
Vol 49 (4) ◽  
pp. 482-494
Author(s):  
Jurgita Kapočiūtė-Dzikienė ◽  
Senait Gebremichael Tesfagergish
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Language Processing ◽  
Deep Neural Networks ◽  
Short Term Memory ◽  
Parameter Tuning ◽  
Feed Forward Neural Network ◽  
Pos Tagging ◽  
Part Of Speech ◽  
Pos Tagger

Deep Neural Networks (DNNs) have proven to be especially successful in the area of Natural Language Processing (NLP) and Part-Of-Speech (POS) tagging—which is the process of mapping words to their corresponding POS labels depending on the context. Despite recent development of language technologies, low-resourced languages (such as an East African Tigrinya language), have received too little attention. We investigate the effectiveness of Deep Learning (DL) solutions for the low-resourced Tigrinya language of the Northern-Ethiopic branch. We have selected Tigrinya as the testbed example and have tested state-of-the-art DL approaches seeking to build the most accurate POS tagger. We have evaluated DNN classifiers (Feed Forward Neural Network – FFNN, Long Short-Term Memory method – LSTM, Bidirectional LSTM, and Convolutional Neural Network – CNN) on a top of neural word2vec word embeddings with a small training corpus known as Nagaoka Tigrinya Corpus. To determine the best DNN classifier type, its architecture and hyper-parameter set both manual and automatic hyper-parameter tuning has been performed. BiLSTM method was proved to be the most suitable for our solving task: it achieved the highest accuracy equal to 92% that is 65% above the random baseline.

scholarly journals Multiple Source Localization in a Shallow Water Waveguide Exploiting Subarray Beamforming and Deep Neural Networks

Sensors ◽  
2019 ◽  
Vol 19 (21) ◽  
pp. 4768 ◽  
Author(s):  
Zhaoqiong Huang ◽  
Ji Xu ◽  
Zaixiao Gong ◽  
Haibin Wang ◽  
Yonghong Yan
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Shallow Water ◽  
Source Localization ◽  
Deep Neural Networks ◽  
Short Term Memory ◽  
Direction Finding ◽  
Multiple Source ◽  
Feed Forward Neural Network ◽  
Subarray Beamforming

Deep neural networks (DNNs) have been shown to be effective for single sound source localization in shallow water environments. However, multiple source localization is a more challenging task because of the interactions among multiple acoustic signals. This paper proposes a framework for multiple source localization on underwater horizontal arrays using deep neural networks. The two-stage DNNs are adopted to determine both the directions and ranges of multiple sources successively. A feed-forward neural network is trained for direction finding, while the long short term memory recurrent neural network is used for source ranging. Particularly, in the source ranging stage, we perform subarray beamforming to extract features of sources that are detected by the direction finding stage, because subarray beamforming can enhance the mixed signal to the desired direction while preserving the horizontal-longitudinal correlations of the acoustic field. In this way, a universal model trained in the single-source scenario can be applied to multi-source scenarios with arbitrary numbers of sources. Both simulations and experiments in a range-independent shallow water environment of SWellEx-96 Event S5 are given to demonstrate the effectiveness of the proposed method.

Part-of-Speech (POS) Tagging Using Deep Learning-Based Approaches on the Designed Khasi POS Corpus

2022 ◽  
Vol 21 (3) ◽  
pp. 1-24
Author(s):  
Sunita Warjri ◽  
Partha Pakray ◽  
Saralin A. Lyngdoh ◽  
Arnab Kumar Maji
Keyword(s):  
Deep Learning ◽  
Natural Language ◽  
Computational Linguistics ◽  
Language Processing ◽  
Research Work ◽  
Pos Tagging ◽  
Part Of Speech ◽  
Corpus Size ◽  
Increase In Accuracy ◽  
Pos Tagger

Part-of-speech (POS) tagging is one of the research challenging fields in natural language processing (NLP). It requires good knowledge of a particular language with large amounts of data or corpora for feature engineering, which can lead to achieving a good performance of the tagger. Our main contribution in this research work is the designed Khasi POS corpus. Till date, there has been no form of any kind of Khasi corpus developed or formally developed. In the present designed Khasi POS corpus, each word is tagged manually using the designed tagset. Methods of deep learning have been used to experiment with our designed Khasi POS corpus. The POS tagger based on BiLSTM, combinations of BiLSTM with CRF, and character-based embedding with BiLSTM are presented. The main challenges of understanding and handling Natural Language toward Computational linguistics to encounter are anticipated. In the presently designed corpus, we have tried to solve the problems of ambiguities of words concerning their context usage, and also the orthography problems that arise in the designed POS corpus. The designed Khasi corpus size is around 96,100 tokens and consists of 6,616 distinct words. Initially, while running the first few sets of data of around 41,000 tokens in our experiment the taggers are found to yield considerably accurate results. When the Khasi corpus size has been increased to 96,100 tokens, we see an increase in accuracy rate and the analyses are more pertinent. As results, accuracy of 96.81% is achieved for the BiLSTM method, 96.98% for BiLSTM with CRF technique, and 95.86% for character-based with LSTM. Concerning substantial research from the NLP perspectives for Khasi, we also present some of the recently existing POS taggers and other NLP works on the Khasi language for comparative purposes.

Learning Syllables Using Conv-LSTM Model for Swahili Word Representation and Part-of-speech Tagging

2021 ◽  
Vol 20 (4) ◽  
pp. 1-25
Author(s):  
Casper Shikali Shivachi ◽  
Refuoe Mokhosi ◽  
Zhou Shijie ◽  
Liu Qihe
Keyword(s):  
Language Processing ◽  
Short Term Memory ◽  
Language Model ◽  
Word Embeddings ◽  
Short Term ◽  
Term Memory ◽  
Pos Tagging ◽  
Part Of Speech ◽  
Word Representation ◽  
Long Short Term Memory

The need to capture intra-word information in natural language processing (NLP) tasks has inspired research in learning various word representations at word, character, or morpheme levels, but little attention has been given to syllables from a syllabic alphabet. Motivated by the success of compositional models in morphological languages, we present a Convolutional-long short term memory (Conv-LSTM) model for constructing Swahili word representation vectors from syllables. The unified architecture addresses the word agglutination and polysemous nature of Swahili by extracting high-level syllable features using a convolutional neural network (CNN) and then composes quality word embeddings with a long short term memory (LSTM). The word embeddings are then validated using a syllable-aware language model ( 31.267 ) and a part-of-speech (POS) tagging task ( 98.78 ), both yielding very competitive results to the state-of-art models in their respective domains. We further validate the language model using Xhosa and Shona, which are syllabic-based languages. The novelty of the study is in its capability to construct quality word embeddings from syllables using a hybrid model that does not use max-over-pool common in CNN and then the exploitation of these embeddings in POS tagging. Therefore, the study plays a crucial role in the processing of agglutinative and syllabic-based languages by contributing quality word embeddings from syllable embeddings, a robust Conv–LSTM model that learns syllables for not only language modeling and POS tagging, but also for other downstream NLP tasks.

scholarly journals Improving accuracy of Part-of-Speech (POS) tagging using hidden markov model and morphological analysis for Myanmar Language

2020 ◽  
Vol 10 (2) ◽  
pp. 2023
Author(s):  
Dim Lam Cing ◽  
Khin Mar Soe
Keyword(s):  
Language Processing ◽  
High Performance ◽  
Markov Models ◽  
Hidden Markov ◽  
Morphological Structure ◽  
Word Segmentation ◽  
Pos Tagging ◽  
Part Of Speech ◽  
Improving Accuracy ◽  
Pos Tagger

In Natural Language Processing (NLP), Word segmentation and Part-of-Speech (POS) tagging are fundamental tasks. The POS information is also necessary in NLP’s preprocessing work applications such as machine translation (MT), information retrieval (IR), etc. Currently, there are many research efforts in word segmentation and POS tagging developed separately with different methods to get high performance and accuracy. For Myanmar Language, there are also separate word segmentors and POS taggers based on statistical approaches such as Neural Network (NN) and Hidden Markov Models (HMMs). But, as the Myanmar language's complex morphological structure, the OOV problem still exists. To keep away from error and improve segmentation by utilizing POS data, segmentation and labeling should be possible at the same time.The main goal of developing POS tagger for any Language is to improve accuracy of tagging and remove ambiguity in sentences due to language structure. This paper focuses on developing word segmentation and Part-of- Speech (POS) Tagger for Myanmar Language. This paper presented the comparison of separate word segmentation and POS tagging with joint word segmentation and POS tagging.

scholarly journals Using Machine Learning Methods in the Financial Market for Technical Analysis Based on Hybrid Models

2020 ◽  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Hybrid Model ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Short Term Memory ◽  
Short Term ◽  
Term Memory ◽  
Input Structure ◽  
Long Short Term Memory

In this study, it is presented a new hybrid model based on deep neural networks to predict the direction and magnitude of the Forex market movement in the short term. The overall model presented is based on the scalping strategy and is provided for high frequency transactions. The proposed hybrid model is based on a combination of three models based on deep neural networks. The first model is a deep neural network with a multi-input structure consisting of a combination of Long Short Term Memory layers. The second model is a deep neural network with a multi-input structure made of a combination of one-dimensional Convolutional Neural network layers. The third model has a simpler structure and is a multi-input model of the Multi-Layer Perceptron layers. The overall model was also a model based on the majority vote of three top models. This study showed that models based on Long Short-Term Memory layers provided better results than the other models and even hybrid models with more than 70% accurate.

scholarly journals Viability of Neural Networks for Core Technologies for Resource-Scarce Languages

Information ◽  
2020 ◽  
Vol 11 (1) ◽  
pp. 41
Author(s):  
Melinda Loubser ◽  
Martin J. Puttkammer
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
South African ◽  
Language Processing ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Neural Network Models ◽  
African Languages ◽  
Pos Tagging

In this paper, the viability of neural network implementations of core technologies (the focus of this paper is on text technologies) for 10 resource-scarce South African languages is evaluated. Neural networks are increasingly being used in place of other machine learning methods for many natural language processing tasks with good results. However, in the South African context, where most languages are resource-scarce, very little research has been done on neural network implementations of core language technologies. In this paper, we address this gap by evaluating neural network implementations of four core technologies for ten South African languages. The technologies we address are part of speech tagging, named entity recognition, compound analysis and lemmatization. Neural architectures that performed well on similar tasks in other settings were implemented for each task and the performance was assessed in comparison with currently used machine learning implementations of each technology. The neural network models evaluated perform better than the baselines for compound analysis, are viable and comparable to the baseline on most languages for POS tagging and NER, and are viable, but not on par with the baseline, for Afrikaans lemmatization.

scholarly journals MulTed: a multilingual aligned and tagged parallel corpus

2020 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Imad Zeroual ◽  
Abdelhak Lakhouaja
Keyword(s):  
Language Processing ◽  
Corpus Linguistics ◽  
Statistical Machine Translation ◽  
Language Recognition ◽  
Parallel Corpora ◽  
Pos Tagging ◽  
Part Of Speech ◽  
Language Studies ◽  
Mapping Process ◽  
Pos Tagger

Recently, more data-driven approaches are demanding multilingual parallel resources primarily in the cross-language studies. To meet these demands, building multilingual parallel corpora are becoming the focus of many Natural Language Processing (NLP) scientific groups. Unlike monolingual corpora, the number of available multilingual parallel corpora is limited. In this paper, the MulTed, a corpus of subtitles extracted from TEDx talks is introduced. It is multilingual, Part of Speech (PoS) tagged, and bilingually sentence-aligned with English as a pivot language. This corpus is designed for many NLP applications, where the sentence-alignment, the PoS tagging, and the size of corpora are influential such as statistical machine translation, language recognition, and bilingual dictionary generation. Currently, the corpus has subtitles that cover 1100 talks available in over 100 languages. The subtitles are classified based on a variety of topics such as Business, Education, and Sport. Regarding the PoS tagging, the Treetagger, a language-independent PoS tagger, is used; then, to make the PoS tagging maximally useful, a mapping process to a universal common tagset is performed. Finally, we believe that making the MulTed corpus available for a public use can be a significant contribution to the literature of NLP and corpus linguistics, especially for under-resourced languages.

scholarly journals Sound Event Detection Using Derivative Features in Deep Neural Networks

2020 ◽  
Vol 10 (14) ◽  
pp. 4911
Author(s):  
Jin-Yeol Kwak ◽  
Yong-Joo Chung
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Event Detection ◽  
Network Architecture ◽  
Deep Neural Networks ◽  
Audio Signal ◽  
Feed Forward Neural Network ◽  
Sound Event ◽  
Audio Data ◽  
Sound Event Detection

We propose using derivative features for sound event detection based on deep neural networks. As input to the networks, we used log-mel-filterbank and its first and second derivative features for each frame of the audio signal. Two deep neural networks were used to evaluate the effectiveness of these derivative features. Specifically, a convolutional recurrent neural network (CRNN) was constructed by combining a convolutional neural network and a recurrent neural networks (RNN) followed by a feed-forward neural network (FNN) acting as a classification layer. In addition, a mean-teacher model based on an attention CRNN was used. Both models had an average pooling layer at the output so that weakly labeled and unlabeled audio data may be used during model training. Under the various training conditions, depending on the neural network architecture and training set, the use of derivative features resulted in a consistent performance improvement by using the derivative features. Experiments on audio data from the Detection and Classification of Acoustic Scenes and Events 2018 and 2019 challenges indicated that a maximum relative improvement of 16.9% was obtained in terms of the F-score.

scholarly journals Attentive Tensor Product Learning

2019 ◽  
Vol 33 ◽  
pp. 1344-1351 ◽  
Author(s):  
Qiuyuan Huang ◽  
Li Deng ◽  
Dapeng Wu ◽  
Chang Liu ◽  
Xiaodong He
Keyword(s):  
Deep Learning ◽  
Natural Language ◽  
Tensor Product ◽  
Language Processing ◽  
Short Term Memory ◽  
Feedforward Neural Networks ◽  
Grammatical Structure ◽  
Symbolic Model ◽  
Pos Tagging ◽  
Part Of Speech

This paper proposes a novel neural architecture — Attentive Tensor Product Learning (ATPL) — to represent grammatical structures of natural language in deep learning models. ATPL exploits Tensor Product Representations (TPR), a structured neural-symbolic model developed in cognitive science, to integrate deep learning with explicit natural language structures and rules. The key ideas of ATPL are: 1) unsupervised learning of role-unbinding vectors of words via the TPR-based deep neural network; 2) the use of attention modules to compute TPR; and 3) the integration of TPR with typical deep learning architectures including long short-term memory and feedforward neural networks. The novelty of our approach lies in its ability to extract the grammatical structure of a sentence by using role-unbinding vectors, which are obtained in an unsupervised manner. Our ATPL approach is applied to 1) image captioning, 2) part of speech (POS) tagging, and 3) constituency parsing of a natural language sentence. The experimental results demonstrate the effectiveness of the proposed approach in all these three natural language processing tasks.

scholarly journals Sentiments classification in stock network public opinion space based on long-short memory convolution neural network

2018 ◽  
Vol 189 ◽  
pp. 10010
Author(s):  
Gaowei Zhang ◽  
Lingyu Xu ◽  
Lei Wang
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Public Opinion ◽  
Convolutional Neural Networks ◽  
Language Processing ◽  
Short Term Memory ◽  
Investment Decision ◽  
Investor Sentiment ◽  
Lookup Table ◽  
Network Public Opinion

Deep learning is used to deal with natural language processing problems. Some are based on phrases and some are based on words. This article is inspired by the pixel level in the CV world and therefore retrains the neural network from a character perspective. Neural networks do not need to know about word lookup table or word2vec in advance, and the knowledge of these words is often high-dimensional and it is difficult to apply to convolutional neural networks. In addition, our long-short term memory convolutional neural networks no longer need to know the syntax and semantics in advance. The purpose of this paper is to analyse the investor's psychological characteristics and investment decision-making behaviour characteristics, to study the investor sentiment in the network public opinion space.

Sign in / Sign up

Export Citation Format

Share Document