scholarly journals Fusing Part-of-Speech Information in Low-Resource Neural Paraphrase Generation

2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
Xiaoqiang Chi ◽  
Yang Xiang
Keyword(s):  
Language Processing ◽  
Sequence Learning ◽  
Neural Nets ◽  
Linguistic Knowledge ◽  
Underlying Assumption ◽  
Low Resource ◽  
Part Of Speech ◽  
Multiple Datasets ◽  
Speech Information ◽  
Paraphrase Generation

Paraphrase generation is an essential yet challenging task in natural language processing. Neural-network-based approaches towards paraphrase generation have achieved remarkable success in recent years. Previous neural paraphrase generation approaches ignore linguistic knowledge, such as part-of-speech information regardless of its availability. The underlying assumption is that neural nets could learn such information implicitly when given sufficient data. However, it would be difficult for neural nets to learn such information properly when data are scarce. In this work, we endeavor to probe into the efficacy of explicit part-of-speech information for the task of paraphrase generation in low-resource scenarios. To this end, we devise three mechanisms to fuse part-of-speech information under the framework of sequence-to-sequence learning. We demonstrate the utility of part-of-speech information in low-resource paraphrase generation through extensive experiments on multiple datasets of varying sizes and genres.

scholarly journals Augmenting Paraphrase Generation with Syntax Information Using Graph Convolutional Networks

Entropy ◽  
2021 ◽  
Vol 23 (5) ◽  
pp. 566
Author(s):  
Xiaoqiang Chi ◽  
Yang Xiang
Keyword(s):  
Language Processing ◽  
Neural Nets ◽  
Linguistic Knowledge ◽  
Convolutional Network ◽  
Tree Structures ◽  
Convolutional Networks ◽  
Sequence Modeling ◽  
Syntactic Information ◽  
Dependency Trees ◽  
Paraphrase Generation

Paraphrase generation is an important yet challenging task in natural language processing. Neural network-based approaches have achieved remarkable success in sequence-to-sequence learning. Previous paraphrase generation work generally ignores syntactic information regardless of its availability, with the assumption that neural nets could learn such linguistic knowledge implicitly. In this work, we make an endeavor to probe into the efficacy of explicit syntactic information for the task of paraphrase generation. Syntactic information can appear in the form of dependency trees, which could be easily acquired from off-the-shelf syntactic parsers. Such tree structures could be conveniently encoded via graph convolutional networks to obtain more meaningful sentence representations, which could improve generated paraphrases. Through extensive experiments on four paraphrase datasets with different sizes and genres, we demonstrate the utility of syntactic information in neural paraphrase generation under the framework of sequence-to-sequence modeling. Specifically, our graph convolutional network-enhanced models consistently outperform their syntax-agnostic counterparts using multiple evaluation metrics.

scholarly journals An Algorithm for Morphological Segmentation of Esperanto Words

2016 ◽  
Vol 105 (1) ◽  
pp. 63-76
Author(s):  
Theresa Guinard
Keyword(s):  
Natural Language Processing ◽  
Language Processing ◽  
Morphological Analysis ◽  
Compound Words ◽  
Part Of Speech ◽  
Semantic Classes ◽  
Morphological Segmentation ◽  
Segmentation Accuracy ◽  
N Gram ◽  
Speech Information

Abstract Morphological analysis (finding the component morphemes of a word and tagging morphemes with part-of-speech information) is a useful preprocessing step in many natural language processing applications, especially for synthetic languages. Compound words from the constructed language Esperanto are formed by straightforward agglutination, but for many words, there is more than one possible sequence of component morphemes. However, one segmentation is usually more semantically probable than the others. This paper presents a modified n-gram Markov model that finds the most probable segmentation of any Esperanto word, where the model’s states represent morpheme part-of-speech and semantic classes. The overall segmentation accuracy was over 98% for a set of presegmented dictionary words.

scholarly journals To Batch or Not to Batch? Comparing Batching and Curriculum Learning Strategies across Tasks and Datasets

Mathematics ◽  
2021 ◽  
Vol 9 (18) ◽  
pp. 2234
Author(s):  
Laura Burdick ◽  
Jonathan K. Kummerfeld ◽  
Rada Mihalcea
Keyword(s):  
Learning Strategies ◽  
Language Processing ◽  
Training Data ◽  
Systematic Analysis ◽  
Part Of Speech Tagging ◽  
Part Of Speech ◽  
Multiple Datasets ◽  
Processing Architectures ◽  
The Impact ◽  
Speech Tagging

Many natural language processing architectures are greatly affected by seemingly small design decisions, such as batching and curriculum learning (how the training data are ordered during training). In order to better understand the impact of these decisions, we present a systematic analysis of different curriculum learning strategies and different batching strategies. We consider multiple datasets for three tasks: text classification, sentence and phrase similarity, and part-of-speech tagging. Our experiments demonstrate that certain curriculum learning and batching decisions do increase performance substantially for some tasks.

A Framework for Indonesian Grammar Error Correction

2021 ◽  
Vol 20 (4) ◽  
pp. 1-12
Author(s):  
Nankai Lin ◽  
Boyu Chen ◽  
Xiaotian Lin ◽  
Kanoksak Wattanachote ◽  
Shengyi Jiang
Keyword(s):  
Error Correction ◽  
Language Processing ◽  
Short Term Memory ◽  
Short Term ◽  
Low Resource ◽  
Part Of Speech ◽  
Grammatical Error ◽  
Evaluation Dataset ◽  
Long Short Term Memory ◽  
Multi Classification

Grammatical Error Correction (GEC) is a challenge in Natural Language Processing research. Although many researchers have been focusing on GEC in universal languages such as English or Chinese, few studies focus on Indonesian, which is a low-resource language. In this article, we proposed a GEC framework that has the potential to be a baseline method for Indonesian GEC tasks. This framework treats GEC as a multi-classification task. It integrates different language embedding models and deep learning models to correct 10 types of Part of Speech (POS) error in Indonesian text. In addition, we constructed an Indonesian corpus that can be utilized as an evaluation dataset for Indonesian GEC research. Our framework was evaluated on this dataset. Results showed that the Long Short-Term Memory model based on word-embedding achieved the best performance. Its overall macro-average F 0.5 in correcting 10 POS error types reached 0.551. Results also showed that the framework can be trained on a low-resource dataset.

scholarly journals BengSentiLex and BengSwearLex: creating lexicons for sentiment analysis and profanity detection in low-resource Bengali language

2021 ◽  
Vol 7 ◽  
pp. e681
Author(s):  
Salim Sazzed
Keyword(s):  
Social Media ◽  
Sentiment Analysis ◽  
Language Processing ◽  
Supervised Machine Learning ◽  
Translation System ◽  
Low Resource ◽  
Part Of Speech ◽  
Sentiment Lexicon ◽  
Cross Lingual ◽  
Bengali Language

Bengali is a low-resource language that lacks tools and resources for various natural language processing (NLP) tasks, such as sentiment analysis or profanity identification. In Bengali, only the translated versions of English sentiment lexicons are available. Moreover, no dictionary exists for detecting profanity in Bengali social media text. This study introduces a Bengali sentiment lexicon, BengSentiLex, and a Bengali swear lexicon, BengSwearLex. For creating BengSentiLex, a cross-lingual methodology is proposed that utilizes a machine translation system, a review corpus, two English sentiment lexicons, pointwise mutual information (PMI), and supervised machine learning (ML) classifiers in various stages. A semi-automatic methodology is presented to develop BengSwearLex that leverages an obscene corpus, word embedding, and part-of-speech (POS) taggers. The performance of BengSentiLex compared with the translated English lexicons in three evaluation datasets. BengSentiLex achieves 5%–50% improvement over the translated lexicons. For identifying profanity, BengSwearLex achieves documentlevel coverage of around 85% in an document-level in the evaluation dataset. The experimental results imply that BengSentiLex and BengSwearLex are effective resources for classifying sentiment and identifying profanity in Bengali social media content, respectively.

scholarly journals Semi-Supervised Learning on Meta Structure: Multi-Task Tagging and Parsing in Low-Resource Scenarios

2020 ◽  
Vol 34 (05) ◽  
pp. 8344-8351 ◽  
Author(s):  
KyungTae Lim ◽  
Jay Yoon Lee ◽  
Jaime Carbonell ◽  
Thierry Poibeau
Keyword(s):  
Supervised Learning ◽  
Language Processing ◽  
Unlabeled Data ◽  
Training Data ◽  
Training Methods ◽  
Multiple Sources ◽  
Single View ◽  
Low Resource ◽  
Part Of Speech ◽  
Proposed Model

Multi-view learning makes use of diverse models arising from multiple sources of input or different feature subsets for the same task. For example, a given natural language processing task can combine evidence from models arising from character, morpheme, lexical, or phrasal views. The most common strategy with multi-view learning, especially popular in the neural network community, is to unify multiple representations into one unified vector through concatenation, averaging, or pooling, and then build a single-view model on top of the unified representation. As an alternative, we examine whether building one model per view and then unifying the different models can lead to improvements, especially in low-resource scenarios. More specifically, taking inspiration from co-training methods, we propose a semi-supervised learning approach based on multi-view models through consensus promotion, and investigate whether this improves overall performance. To test the multi-view hypothesis, we use moderately low-resource scenarios for nine languages and test the performance of the joint model for part-of-speech tagging and dependency parsing. The proposed model shows significant improvements across the test cases, with average gains of -0.9 ∼ +9.3 labeled attachment score (LAS) points. We also investigate the effect of unlabeled data on the proposed model by varying the amount of training data and by using different domains of unlabeled data.

scholarly journals Electronic Patient Records: Some Answers to the Data Representation and Reuse Challenges

2007 ◽  
Vol 16 (01) ◽  
pp. 47-48
Author(s):  
S. Meystre ◽  
Keyword(s):  
Language Processing ◽  
Electronic Patient Record ◽  
Data Entry ◽  
Data Representation ◽  
Patient Records ◽  
Electronic Patient Records ◽  
Clinical Text ◽  
Part Of Speech ◽  
Excellent Research ◽  
Speech Information

SummaryTo summarize current excellent research in the field of patient records.Synopsis of the papers selected for the IMIA Yearbook 2007.The Electronic Patient Record encompasses a broad field of research and development. Some current research topics were selected for this IMIA Yearbook: EHR representation and communication standards, and secondary uses of clinical data for research and decision support. Four excellent papers representing the research in those fields were selected for the Patient Records section.The best papers selected for this section focus on the analysis and comparison of two important clinical documents representation standards, on direct structured data entry, on the use of Natural Language Processing to detect adverse events, and on the development and evaluation of a clinical text corpus annotated for part-of-speech information.

scholarly journals Augmenting Paraphrase Generation with Syntax Information using Graph Convolutional Networks

2021 ◽  
Author(s):  
Xiaoqiang Chi ◽  
Yang Xiang
Keyword(s):  
Neural Network ◽  
Neural Nets ◽  
Linguistic Knowledge ◽  
Evaluation Metrics ◽  
Tree Structures ◽  
Convolutional Networks ◽  
Syntactic Information ◽  
Dependency Trees ◽  
Meaningful Sentence ◽  
Paraphrase Generation

Paraphrase generation is an important yet challenging task in NLP. Neural network-based approaches have achieved remarkable success in sequence-to-sequence(seq2seq) learning. Previous paraphrase generation work generally ignores syntactic information regardless of its availability, with the assumption that neural nets could learn such linguistic knowledge implicitly. In this work we make an endeavor to probe into the efficacy of explicit syntactic information for the task of paraphrase generation. Syntactic information can appear in the form of dependency trees which could be easily acquired from off-the-shelf syntactic parsers. Such tree structures could be conveniently encoded via graph convolutional networks(GCNs) to obtain more meaningful sentence representations, which could improve generated paraphrases. Through extensive experiments on four paraphrase datasets with different sizes and genres, we demonstrate the utility of syntactic information in neural paraphrase generation under the framework of seq2seq modeling. Specifically, our GCN-enhanced models consistently outperform their syntax-agnostic counterparts in multiple evaluation metrics.

scholarly journals Improving Loanword Identification in Low-Resource Language with Data Augmentation and Multiple Feature Fusion

2021 ◽  
Vol 2021 ◽  
pp. 1-9
Author(s):  
Chenggang Mi ◽  
Shaolin Zhu ◽  
Rui Nie
Keyword(s):  
Language Processing ◽  
Data Augmentation ◽  
Feature Fusion ◽  
Training Data ◽  
Low Resource ◽  
High Resource ◽  
Part Of Speech ◽  
Word Level ◽  
Cross Lingual ◽  
Log Linear

Loanword identification is studied in recent years to alleviate data sparseness in several natural language processing (NLP) tasks, such as machine translation, cross-lingual information retrieval, and so on. However, recent studies on this topic usually put efforts on high-resource languages (such as Chinese, English, and Russian); for low-resource languages, such as Uyghur and Mongolian, due to the limitation of resources and lack of annotated data, loanword identification on these languages tends to have lower performance. To overcome this problem, we first propose a lexical constraint-based data augmentation method to generate training data for low-resource language loanword identification; then, a loanword identification model based on a log-linear RNN is introduced to improve the performance of low-resource loanword identification by incorporating features such as word-level embeddings, character-level embeddings, pronunciation similarity, and part-of-speech (POS) into one model. Experimental results on loanword identification in Uyghur (in this study, we mainly focus on Arabic, Chinese, Russian, and Turkish loanwords in Uyghur) showed that our proposed method achieves best performance compared with several strong baseline systems.

An Automatic Question Generation System using Rule-Based Approach in Bloom’s Taxonomy

2019 ◽  
Vol 13 ◽  
Author(s):  
G Deena ◽  
K Raja ◽  
K Kannan
Keyword(s):  
Language Processing ◽  
Learning Process ◽  
Question Generation ◽  
Test Question ◽  
Rule Based ◽  
Part Of Speech ◽  
Core Idea ◽  
Rule Based Approach ◽  
Teaching Learning ◽  
Automatic Question Generation

: In this competing world, education has become part of everyday life. The process of imparting the knowledge to the learner through education is the core idea in the Teaching-Learning Process (TLP). An assessment is one way to identify the learner’s weak spot of the area under discussion. An assessment question has higher preferences in judging the learner's skill. In manual preparation, the questions are not assured in excellence and fairness to assess the learner’s cognitive skill. Question generation is the most important part of the teaching-learning process. It is clearly understood that generating the test question is the toughest part. Methods: Proposed an Automatic Question Generation (AQG) system which automatically generates the assessment questions dynamically from the input file. Objective: The Proposed system is to generate the test questions that are mapped with blooms taxonomy to determine the learner’s cognitive level. The cloze type questions are generated using the tag part-of-speech and random function. Rule-based approaches and Natural Language Processing (NLP) techniques are implemented to generate the procedural question of the lowest blooms cognitive levels. Analysis: The outputs are dynamic in nature to create a different set of questions at each execution. Here, input paragraph is selected from computer science domain and their output efficiency are measured using the precision and recall.

Sign in / Sign up

Export Citation Format

Share Document