scholarly journals Natural language inference for Malayalam language using language agnostic sentence representation

2021 ◽  
Vol 7 ◽  
pp. e508
Author(s):  
Sara Renjit ◽  
Sumam Idicula
Keyword(s):  
Natural Language ◽  
Language Processing ◽  
Question Answering ◽  
Binary Classification ◽  
Multiclass Classification ◽  
The Other ◽  
Indian Language ◽  
Information Retrieval Systems ◽  
Textual Entailment ◽  
Malayalam Language

Natural language inference (NLI) is an essential subtask in many natural language processing applications. It is a directional relationship from premise to hypothesis. A pair of texts is defined as entailed if a text infers its meaning from the other text. The NLI is also known as textual entailment recognition, and it recognizes entailed and contradictory sentences in various NLP systems like Question Answering, Summarization and Information retrieval systems. This paper describes the NLI problem attempted for a low resource Indian language Malayalam, the regional language of Kerala. More than 30 million people speak this language. The paper is about the Malayalam NLI dataset, named MaNLI dataset, and its application of NLI in Malayalam language using different models, namely Doc2Vec (paragraph vector), fastText, BERT (Bidirectional Encoder Representation from Transformers), and LASER (Language Agnostic Sentence Representation). Our work attempts NLI in two ways, as binary classification and as multiclass classification. For both the classifications, LASER outperformed the other techniques. For multiclass classification, NLI using LASER based sentence embedding technique outperformed the other techniques by a significant margin of 12% accuracy. There was also an accuracy improvement of 9% for LASER based NLI system for binary classification over the other techniques.

scholarly journals A Survey of Paraphrasing and Textual Entailment Methods

2010 ◽  
Vol 38 ◽  
pp. 135-187 ◽  
Author(s):  
I. Androutsopoulos ◽  
P. Malakasiotis
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Machine Translation ◽  
Language Processing ◽  
Question Answering ◽  
Extraction Methods ◽  
The Other ◽  
Text Generation ◽  
Wide Range ◽  
Textual Entailment

Paraphrasing methods recognize, generate, or extract phrases, sentences, or longer natural language expressions that convey almost the same information. Textual entailment methods, on the other hand, recognize, generate, or extract pairs of natural language expressions, such that a human who reads (and trusts) the first element of a pair would most likely infer that the other element is also true. Paraphrasing can be seen as bidirectional textual entailment and methods from the two areas are often similar. Both kinds of methods are useful, at least in principle, in a wide range of natural language processing applications, including question answering, summarization, text generation, and machine translation. We summarize key ideas from the two areas by considering in turn recognition, generation, and extraction methods, also pointing to prominent articles and resources.

scholarly journals List-wise learning to rank biomedical question-answer pairs with deep ranking recursive autoencoders

PLoS ONE ◽  
2020 ◽  
Vol 15 (11) ◽  
pp. e0242061
Author(s):  
Yan Yan ◽  
Bo-Wen Zhang ◽  
Xu-Feng Li ◽  
Zhenhan Liu
Keyword(s):  
Natural Language ◽  
Language Processing ◽  
Question Answering ◽  
Binary Classification ◽  
Learning To Rank ◽  
Relevant Literature ◽  
Semantic Relatedness ◽  
Learning Approaches ◽  
Feature Representations ◽  
Ranking Error

Biomedical question answering (QA) represents a growing concern among industry and academia due to the crucial impact of biomedical information. When mapping and ranking candidate snippet answers within relevant literature, current QA systems typically refer to information retrieval (IR) techniques: specifically, query processing approaches and ranking models. However, these IR-based approaches are insufficient to consider both syntactic and semantic relatedness and thus cannot formulate accurate natural language answers. Recently, deep learning approaches have become well-known for learning optimal semantic feature representations in natural language processing tasks. In this paper, we present a deep ranking recursive autoencoders (rankingRAE) architecture for ranking question-candidate snippet answer pairs (Q-S) to obtain the most relevant candidate answers for biomedical questions extracted from the potentially relevant documents. In particular, we convert the task of ranking candidate answers to several simultaneous binary classification tasks for determining whether a question and a candidate answer are relevant. The compositional words and their random initialized vectors of concatenated Q-S pairs are fed into recursive autoencoders to learn the optimal semantic representations in an unsupervised way, and their semantic relatedness is classified through supervised learning. Unlike several existing methods to directly choose the top-K candidates with highest probabilities, we take the influence of different ranking results into consideration. Consequently, we define a listwise “ranking error” for loss function computation to penalize inappropriate answer ranking for each question and to eliminate their influence. The proposed architecture is evaluated with respect to the BioASQ 2013-2018 Six-year Biomedical Question Answering benchmarks. Compared with classical IR models, other deep representation models, as well as some state-of-the-art systems for these tasks, the experimental results demonstrate the robustness and effectiveness of rankingRAE.

ALBERT-QM: An ALBERT Based Method for Chinese Health Related Question Matching (Preprint)

2020 ◽  
Author(s):  
Feihong Yang ◽  
Jiao Li
Keyword(s):  
Natural Language ◽  
Language Processing ◽  
Question Answering ◽  
Binary Classification ◽  
Medical Knowledge ◽  
Related Question ◽  
Text Similarity ◽  
Web Based ◽  
Health Related ◽  
Health Question

BACKGROUND Question answering (QA) system is widely used in web-based health-care applications. Health consumers likely asked similar questions in various natural language expression due to the lack of medical knowledge. It’s challenging to match a new question to previous similar questions for answering. In health QA system development, question matching (QM) is a task to judge whether a pair of questions express the same meaning and is used to map the answer of matched question in the given question-answering database. BERT (i.e. Bidirectional Encoder Representations from Transformers) is proved to be state-of- the-art model in natural language processing (NLP) tasks, such as binary classification and sentence matching. As a light model of BERT, ALBERT is proposed to address the huge parameters and low training speed problems of BERT. Both of BERT and ALBERT can be used to address the QM problem. OBJECTIVE In this study, we aim to develop an ALBERT based method for Chinese health related question matching. METHODS Our proposed method, named as ALBERT-QM, consists of three components. (1)Data augmenting. Similar health question pairs were augmented for training preparation. (2)ALBERT model training. Given the augmented training pairs, three ALBERT models were trained and fine-tuned. (3)Similarity combining. Health question similarity score were calculated by combining ALBRT model outputs with text similarity. To evaluate our ALBERT-QM performance on similar question identification, we used an open dataset with 20,000 labeled Chinese health question pairs. RESULTS Our ALBERT-QM is able to identify similar Chinese health questions, achieving the precision of 86.69%, recall of 86.70% and F1 of 86.69%. Comparing with baseline method (text similarity algorithm), ALBERT-QM enhanced the F1-score by 20.73%. Comparing with other BERT series models, our ALBERT-QM is much lighter with the files size of 64.8MB which is 1/6 times that other BERT models. We made our ALBERT-QM open accessible at https://github.com/trueto/albert_question_match. CONCLUSIONS In this study, we developed an open source algorithm, ALBERT-QM, contributing to similar Chinese health questions identification in a health QA system. Our ALBERT-QM achieved better performance in question matching with lower memory usage, which is beneficial to the web-based or mobile-based QA applications.

A survey of question answering in natural language processing

Poetics ◽  
1990 ◽  
Vol 19 (1-2) ◽  
pp. 99-120
Author(s):  
Stefan Wermter ◽  
Wendy G. Lehnert
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Question Answering

scholarly journals Enhancing Natural Language Inference Using New and Expanded Training Data Sets and New Learning Models

2020 ◽  
Vol 34 (05) ◽  
pp. 8504-8511
Author(s):  
Arindam Mitra ◽  
Ishan Shrivastava ◽  
Chitta Baral
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Question Answering ◽  
Training Data ◽  
Data Sets ◽  
Learning Models ◽  
New Learning ◽  
Word Attention ◽  
Attention Function

Natural Language Inference (NLI) plays an important role in many natural language processing tasks such as question answering. However, existing NLI modules that are trained on existing NLI datasets have several drawbacks. For example, they do not capture the notion of entity and role well and often end up making mistakes such as “Peter signed a deal” can be inferred from “John signed a deal”. As part of this work, we have developed two datasets that help mitigate such issues and make the systems better at understanding the notion of “entities” and “roles”. After training the existing models on the new dataset we observe that the existing models do not perform well on one of the new benchmark. We then propose a modification to the “word-to-word” attention function which has been uniformly reused across several popular NLI architectures. The resulting models perform as well as their unmodified counterparts on the existing benchmarks and perform significantly well on the new benchmarks that emphasize “roles” and “entities”.

scholarly journals Emerging trends: A gentle introduction to fine-tuning

2021 ◽  
Vol 27 (6) ◽  
pp. 763-778
Author(s):  
Kenneth Ward Church ◽  
Zeyu Chen ◽  
Yanjun Ma
Keyword(s):  
Natural Language ◽  
Language Processing ◽  
Question Answering ◽  
General Purpose ◽  
Fine Tuning ◽  
Language Engineering ◽  
Training Models ◽  
Emerging Trends ◽  
Foundation Model ◽  
Programming Skills

AbstractThe previous Emerging Trends article (Church et al., 2021. Natural Language Engineering27(5), 631–645.) introduced deep nets to poets. Poets is an imperfect metaphor, intended as a gesture toward inclusion. The future for deep nets will benefit by reaching out to a broad audience of potential users, including people with little or no programming skills, and little interest in training models. That paper focused on inference, the use of pre-trained models, as is, without fine-tuning. The goal of this paper is to make fine-tuning more accessible to a broader audience. Since fine-tuning is more challenging than inference, the examples in this paper will require modest programming skills, as well as access to a GPU. Fine-tuning starts with a general purpose base (foundation) model and uses a small training set of labeled data to produce a model for a specific downstream application. There are many examples of fine-tuning in natural language processing (question answering (SQuAD) and GLUE benchmark), as well as vision and speech.

BUILD KNOWLEDGE GRAPH FROM HETEROGENEOUS DOCUMENTS

2021 ◽  
Vol 47 (05) ◽  
Author(s):  
NGUYỄN CHÍ HIẾU
Keyword(s):  
Information Retrieval ◽  
Deep Learning ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Question Answering ◽  
Semantic Analysis ◽  
Knowledge Graph ◽  
Question Answering Systems ◽  
Knowledge Graphs

Knowledge Graphs are applied in many fields such as search engines, semantic analysis, and question answering in recent years. However, there are many obstacles for building knowledge graphs as methodologies, data and tools. This paper introduces a novel methodology to build knowledge graph from heterogeneous documents.  We use the methodologies of Natural Language Processing and deep learning to build this graph. The knowledge graph can use in Question answering systems and Information retrieval especially in Computing domain

LIS4: Lesk Inspired Sense Specific Semantic Similarity using WordNet

2021 ◽  
pp. 2150006
Author(s):  
Saravanakumar Kandasamy ◽  
Aswani Kumar Cherukuri
Keyword(s):  
Information Retrieval ◽  
Natural Language Processing ◽  
Natural Language ◽  
Semantic Similarity ◽  
Language Processing ◽  
Gold Standard ◽  
Question Answering ◽  
Knowledge Based ◽  
Benchmark Datasets ◽  
Processing Information

Semantic similarity quantification between concepts is one of the inevitable parts in domains like Natural Language Processing, Information Retrieval, Question Answering, etc. to understand the text and their relationships better. Last few decades, many measures have been proposed by incorporating various corpus-based and knowledge-based resources. WordNet and Wikipedia are two of the Knowledge-based resources. The contribution of WordNet in the above said domain is enormous due to its richness in defining a word and all of its relationship with others. In this paper, we proposed an approach to quantify the similarity between concepts that exploits the synsets and the gloss definitions of different concepts using WordNet. Our method considers the gloss definitions, contextual words that are helping in defining a word, synsets of contextual word and the confidence of occurrence of a word in other word’s definition for calculating the similarity. The evaluation based on different gold standard benchmark datasets shows the efficiency of our system in comparison with other existing taxonomical and definitional measures.

Question Answering and Generation

2012 ◽  
pp. 1-16 ◽  
Author(s):  
Arthur C. Graesser ◽  
Vasile Rus ◽  
Zhiqiang Cai ◽  
Xiangen Hu
Keyword(s):  
Natural Language ◽  
Language Processing ◽  
Information Needs ◽  
Question Answering ◽  
Learning Technologies ◽  
Question Asking ◽  
Tutoring Systems ◽  
Computer Based Training ◽  
Recent Developments ◽  
Text Meaning Representation

Automated Question Answering and Asking are two active areas of Natural Language Processing with the former dominating the past decade and the latter most likely to dominate the next one. Due to the vast amounts of information available electronically in the Internet era, automated Question Answering is needed to fulfill information needs in an efficient and effective manner. Automated Question Answering is the task of providing answers automatically to questions asked in natural language. Typically, the answers are retrieved from large collections of documents. While answering any question is difficult, successful automated solutions to answer some type of questions, so-called factoid questions, have been developed recently, culminating with the just announced Watson Question Answering system developed by I.B.M. to compete in Jeopardy-like games. The flip process, automated Question Asking or Generation, is about generating questions from some form of input such as a text, meaning representation, or database. Question Asking/Generation is an important component in the full gamut of learning technologies, from conventional computer-based training to tutoring systems. Advances in Question Asking/Generation are projected to revolutionize learning and dialogue systems. This chapter presents an overview of recent developments in Question Answering and Generation starting with the landscape of questions that people ask.

Sign in / Sign up

Export Citation Format

Share Document