scholarly journals Interactive Dual Attention Network for Text Sentiment Classification

2020 ◽  
Vol 2020 ◽  
pp. 1-11
Author(s):  
Yinglin Zhu ◽  
Wenbin Zheng ◽  
Hong Tang
Keyword(s):  
Language Processing ◽  
Classification Performance ◽  
Research Field ◽  
Sentiment Classification ◽  
Attention Network ◽  
Linguistic Resources ◽  
Benchmark Datasets ◽  
Interactive Relationship ◽  
Conventional Machine ◽  
Result Analysis

Text sentiment classification is an essential research field of natural language processing. Recently, numerous deep learning-based methods for sentiment classification have been proposed and achieved better performances compared with conventional machine learning methods. However, most of the proposed methods ignore the interactive relationship between contextual semantics and sentimental tendency while modeling their text representation. In this paper, we propose a novel Interactive Dual Attention Network (IDAN) model that aims to interactively learn the representation between contextual semantics and sentimental tendency information. Firstly, we design an algorithm that utilizes linguistic resources to obtain sentimental tendency information from text and then extract word embeddings from the BERT (Bidirectional Encoder Representations from Transformers) pretraining model as the embedding layer of IDAN. Next, we use two Bidirectional LSTM (BiLSTM) networks to learn the long-range dependencies of contextual semantics and sentimental tendency information, respectively. Finally, two types of attention mechanisms are implemented in IDAN. One is multihead attention, which is the next layer of BiLSTM and is used to learn the interactive relationship between contextual semantics and sentimental tendency information. The other is global attention that aims to make the model focus on the important parts of the sequence and generate the final representation for classification. These two attention mechanisms enable IDAN to interactively learn the relationship between semantics and sentimental tendency information and improve the classification performance. A large number of experiments on four benchmark datasets show that our IDAN model is superior to competitive methods. Moreover, both the result analysis and the attention weight visualization further demonstrate the effectiveness of our proposed method.

scholarly journals Efficient processing of GRU based on word embedding for text classification

2019 ◽  
Vol 3 (4) ◽  
Author(s):  
Muhammad Zulqarnain ◽  
Rozaida Ghazali ◽  
Muhammad Ghulam Ghouse ◽  
Muhammad Faheem Mushtaq
Keyword(s):  
Language Processing ◽  
Text Classification ◽  
Classification Performance ◽  
Word Embedding ◽  
Training Data ◽  
Superior Performance ◽  
Sequential Data ◽  
Online Data ◽  
Benchmark Datasets ◽  
Recurrent Architecture

Text classification has become very serious problem for big organization to manage the large amount of online data and has been extensively applied in the tasks of Natural Language Processing (NLP). Text classification can support users to excellently manage and exploit meaningful information require to be classified into various categories for further use. In order to best classify texts, our research efforts to develop a deep learning approach which obtains superior performance in text classification than other RNNs approaches. However, the main problem in text classification is how to enhance the classification accuracy and the sparsity of the data semantics sensitivity to context often hinders the classification performance of texts. In order to overcome the weakness, in this paper we proposed unified structure to investigate the effects of word embedding and Gated Recurrent Unit (GRU) for text classification on two benchmark datasets included (Google snippets and TREC). GRU is a well-known type of recurrent neural network (RNN), which is ability of computing sequential data over its recurrent architecture. Experimentally, the semantically connected words are commonly near to each other in embedding spaces. First, words in posts are changed into vectors via word embedding technique. Then, the words sequential in sentences are fed to GRU to extract the contextual semantics between words. The experimental results showed that proposed GRU model can effectively learn the word usage in context of texts provided training data. The quantity and quality of training data significantly affected the performance. We evaluated the performance of proposed approach with traditional recurrent approaches, RNN, MV-RNN and LSTM, the proposed approach is obtained better results on two benchmark datasets in the term of accuracy and error rate.

scholarly journals A Survey on Machine Reading Comprehension—Tasks, Evaluation Metrics and Benchmark Datasets

2020 ◽  
Vol 10 (21) ◽  
pp. 7640
Author(s):  
Changchang Zeng ◽  
Shaobo Li ◽  
Qin Li ◽  
Jie Hu ◽  
Jianjun Hu
Keyword(s):  
Reading Comprehension ◽  
Language Processing ◽  
Large Scale ◽  
Human Performance ◽  
Research Field ◽  
Evaluation Metrics ◽  
Future Research ◽  
Benchmark Datasets ◽  
Comprehensive Survey ◽  
Machine Reading

Machine Reading Comprehension (MRC) is a challenging Natural Language Processing (NLP) research field with wide real-world applications. The great progress of this field in recent years is mainly due to the emergence of large-scale datasets and deep learning. At present, a lot of MRC models have already surpassed human performance on various benchmark datasets despite the obvious giant gap between existing MRC models and genuine human-level reading comprehension. This shows the need for improving existing datasets, evaluation metrics, and models to move current MRC models toward “real” understanding. To address the current lack of comprehensive survey of existing MRC tasks, evaluation metrics, and datasets, herein, (1) we analyze 57 MRC tasks and datasets and propose a more precise classification method of MRC tasks with 4 different attributes; (2) we summarized 9 evaluation metrics of MRC tasks, 7 attributes and 10 characteristics of MRC datasets; (3) We also discuss key open issues in MRC research and highlighted future research directions. In addition, we have collected, organized, and published our data on the companion website where MRC researchers could directly access each MRC dataset, papers, baseline projects, and the leaderboard.

A Convolutional Neural Network for Aspect-Level Sentiment Classification

2019 ◽  
Vol 33 (14) ◽  
pp. 1959046 ◽  
Author(s):  
Yongping Xing ◽  
Chuangbai Xiao ◽  
Yifei Wu ◽  
Ziming Ding
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Natural Language Processing ◽  
Convolutional Neural Network ◽  
Language Processing ◽  
Recurrent Neural Networks ◽  
Classification Performance ◽  
Sentiment Classification ◽  
Extensive Literature ◽  
Effective Models

Sentiment analysis, including aspect-level sentiment classification, is an important basic natural language processing (NLP) task. Aspect-level sentiment can provide complete and in-depth results. Words with different contexts variably influence the aspect-level sentiment polarity of sentences, and polarity varies based on different aspects of a sentence. Recurrent neural networks (RNNs) are regarded as effective models for handling NLP and have performed well in aspect-level sentiment classification. Extensive literature exists on sentiment classification that utilizes convolutional neural networks (CNNs); however, no literature on aspect-level sentiment classification that uses CNNs is available. In the present study, we develop a CNN model for handling aspect-level sentiment classification. In our model, attention-based input layers are incorporated into CNN to introduce aspect information. In our experiment, in which a benchmark dataset from Twitter is compared with other models, incorporating aspect information into CNN improves aspect-level sentiment classification performance without using syntactic parser or other language features.

scholarly journals Hierarchical Phoneme Classification for Improved Speech Recognition

2021 ◽  
Vol 11 (1) ◽  
pp. 428
Author(s):  
Donghoon Oh ◽  
Jeong-Sik Park ◽  
Ji-Hwan Kim ◽  
Gil-Jin Jang
Keyword(s):  
Speech Recognition ◽  
Language Processing ◽  
Confusion Matrix ◽  
Critical Factor ◽  
Recognition System ◽  
Classification Performance ◽  
Language Models ◽  
Successful Implementation ◽  
Phoneme Classification ◽  
Improved Performance

Speech recognition consists of converting input sound into a sequence of phonemes, then finding text for the input using language models. Therefore, phoneme classification performance is a critical factor for the successful implementation of a speech recognition system. However, correctly distinguishing phonemes with similar characteristics is still a challenging problem even for state-of-the-art classification methods, and the classification errors are hard to be recovered in the subsequent language processing steps. This paper proposes a hierarchical phoneme clustering method to exploit more suitable recognition models to different phonemes. The phonemes of the TIMIT database are carefully analyzed using a confusion matrix from a baseline speech recognition model. Using automatic phoneme clustering results, a set of phoneme classification models optimized for the generated phoneme groups is constructed and integrated into a hierarchical phoneme classification method. According to the results of a number of phoneme classification experiments, the proposed hierarchical phoneme group models improved performance over the baseline by 3%, 2.1%, 6.0%, and 2.2% for fricative, affricate, stop, and nasal sounds, respectively. The average accuracy was 69.5% and 71.7% for the baseline and proposed hierarchical models, showing a 2.2% overall improvement.

scholarly journals Joint Multimodal Embedding and Backtracking Search in Vision-and-Language Navigation

Sensors ◽  
2021 ◽  
Vol 21 (3) ◽  
pp. 1012
Author(s):  
Jisu Hwang ◽  
Incheol Kim
Keyword(s):  
Natural Language ◽  
Language Processing ◽  
Input Data ◽  
Language Instruction ◽  
Scoring Method ◽  
Processing Technologies ◽  
Backtracking Search ◽  
Panoramic Images ◽  
Benchmark Datasets ◽  
Vision And Language

Due to the development of computer vision and natural language processing technologies in recent years, there has been a growing interest in multimodal intelligent tasks that require the ability to concurrently understand various forms of input data such as images and text. Vision-and-language navigation (VLN) require the alignment and grounding of multimodal input data to enable real-time perception of the task status on panoramic images and natural language instruction. This study proposes a novel deep neural network model (JMEBS), with joint multimodal embedding and backtracking search for VLN tasks. The proposed JMEBS model uses a transformer-based joint multimodal embedding module. JMEBS uses both multimodal context and temporal context. It also employs backtracking-enabled greedy local search (BGLS), a novel algorithm with a backtracking feature designed to improve the task success rate and optimize the navigation path, based on the local and global scores related to candidate actions. A novel global scoring method is also used for performance improvement by comparing the partial trajectories searched thus far with a plurality of natural language instructions. The performance of the proposed model on various operations was then experimentally demonstrated and compared with other models using the Matterport3D Simulator and room-to-room (R2R) benchmark datasets.

scholarly journals Building natural language processing tools for Runyakitara

2020 ◽  
Vol 0 (0) ◽  
Author(s):  
Fridah Katushemererwe ◽  
Andrew Caines ◽  
Paula Buttery
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Learning ◽  
Language Processing ◽  
Primary Data ◽  
Computer Assisted ◽  
Endangered Languages ◽  
Test Case ◽  
Short Supply ◽  
Linguistic Resources

AbstractThis paper describes an endeavour to build natural language processing (NLP) tools for Runyakitara, a group of four closely related Bantu languages spoken in western Uganda. In contrast with major world languages such as English, for which corpora are comparatively abundant and NLP tools are well developed, computational linguistic resources for Runyakitara are in short supply. First therefore, we need to collect corpora for these languages, before we can proceed to the design of a spell-checker, grammar-checker and applications for computer-assisted language learning (CALL). We explain how we are collecting primary data for a new Runya Corpus of speech and writing, we outline the design of a morphological analyser, and discuss how we can use these new resources to build NLP tools. We are initially working with Runyankore–Rukiga, a closely-related pair of Runyakitara languages, and we frame our project in the context of NLP for low-resource languages, as well as CALL for the preservation of endangered languages. We put our project forward as a test case for the revitalization of endangered languages through education and technology.

Towards corpus and model: Hierarchical structured-attention-based features for Indonesian named entity recognition

2021 ◽  
pp. 1-12
Author(s):  
Yingwen Fu ◽  
Nankai Lin ◽  
Xiaotian Lin ◽  
Shengyi Jiang
Keyword(s):  
Language Processing ◽  
State Of The Art ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Language Models ◽  
Neural Models ◽  
Performance Models ◽  
Named Entity ◽  
High Resource ◽  
Benchmark Datasets

Named entity recognition (NER) is fundamental to natural language processing (NLP). Most state-of-the-art researches on NER are based on pre-trained language models (PLMs) or classic neural models. However, these researches are mainly oriented to high-resource languages such as English. While for Indonesian, related resources (both in dataset and technology) are not yet well-developed. Besides, affix is an important word composition for Indonesian language, indicating the essentiality of character and token features for token-wise Indonesian NLP tasks. However, features extracted by currently top-performance models are insufficient. Aiming at Indonesian NER task, in this paper, we build an Indonesian NER dataset (IDNER) comprising over 50 thousand sentences (over 670 thousand tokens) to alleviate the shortage of labeled resources in Indonesian. Furthermore, we construct a hierarchical structured-attention-based model (HSA) for Indonesian NER to extract sequence features from different perspectives. Specifically, we use an enhanced convolutional structure as well as an enhanced attention structure to extract deeper features from characters and tokens. Experimental results show that HSA establishes competitive performance on IDNER and three benchmark datasets.

A Survey on Document-level Neural Machine Translation

2021 ◽  
Vol 54 (2) ◽  
pp. 1-36
Author(s):  
Sameen Maruf ◽  
Fahimeh Saleh ◽  
Gholamreza Haffari
Keyword(s):  
Machine Translation ◽  
Language Processing ◽  
Research Field ◽  
Translation Process ◽  
Future Directions ◽  
Translation Quality ◽  
Current State ◽  
Evaluation Strategies ◽  
Almost All ◽  
Document Level

Machine translation (MT) is an important task in natural language processing (NLP), as it automates the translation process and reduces the reliance on human translators. With the resurgence of neural networks, the translation quality surpasses that of the translations obtained using statistical techniques for most language-pairs. Up until a few years ago, almost all of the neural translation models translated sentences independently , without incorporating the wider document-context and inter-dependencies among the sentences. The aim of this survey article is to highlight the major works that have been undertaken in the space of document-level machine translation after the neural revolution, so researchers can recognize the current state and future directions of this field. We provide an organization of the literature based on novelties in modelling and architectures as well as training and decoding strategies. In addition, we cover evaluation strategies that have been introduced to account for the improvements in document MT, including automatic metrics and discourse-targeted test sets. We conclude by presenting possible avenues for future exploration in this research field.

scholarly journals Multi-granularity Deep Local Representations for Irregular Scene Text Recognition

2021 ◽  
Vol 2 (2) ◽  
pp. 1-18
Author(s):  
Hongchao Gao ◽  
Yujia Li ◽  
Jiao Dai ◽  
Xi Wang ◽  
Jizhong Han ◽  
...  
Keyword(s):  
State Of The Art ◽  
Visual Representation ◽  
Text Recognition ◽  
Natural Scene ◽  
Attention Network ◽  
Training Time ◽  
Scene Text ◽  
Benchmark Datasets ◽  
Local Representations ◽  
Scene Text Recognition

Recognizing irregular text from natural scene images is challenging due to the unconstrained appearance of text, such as curvature, orientation, and distortion. Recent recognition networks regard this task as a text sequence labeling problem and most networks capture the sequence only from a single-granularity visual representation, which to some extent limits the performance of recognition. In this article, we propose a hierarchical attention network to capture multi-granularity deep local representations for recognizing irregular scene text. It consists of several hierarchical attention blocks, and each block contains a Local Visual Representation Module (LVRM) and a Decoder Module (DM). Based on the hierarchical attention network, we propose a scene text recognition network. The extensive experiments show that our proposed network achieves the state-of-the-art performance on several benchmark datasets including IIIT-5K, SVT, CUTE, SVT-Perspective, and ICDAR datasets under shorter training time.

Sign in / Sign up

Export Citation Format

Share Document