ALBERT-QM: An ALBERT Based Method for Chinese Health Related Question Matching (Preprint)

2020 ◽  
Author(s):  
Feihong Yang ◽  
Jiao Li
Keyword(s):  
Natural Language ◽  
Language Processing ◽  
Question Answering ◽  
Binary Classification ◽  
Medical Knowledge ◽  
Related Question ◽  
Text Similarity ◽  
Web Based ◽  
Health Related ◽  
Health Question

BACKGROUND Question answering (QA) system is widely used in web-based health-care applications. Health consumers likely asked similar questions in various natural language expression due to the lack of medical knowledge. It’s challenging to match a new question to previous similar questions for answering. In health QA system development, question matching (QM) is a task to judge whether a pair of questions express the same meaning and is used to map the answer of matched question in the given question-answering database. BERT (i.e. Bidirectional Encoder Representations from Transformers) is proved to be state-of- the-art model in natural language processing (NLP) tasks, such as binary classification and sentence matching. As a light model of BERT, ALBERT is proposed to address the huge parameters and low training speed problems of BERT. Both of BERT and ALBERT can be used to address the QM problem. OBJECTIVE In this study, we aim to develop an ALBERT based method for Chinese health related question matching. METHODS Our proposed method, named as ALBERT-QM, consists of three components. (1)Data augmenting. Similar health question pairs were augmented for training preparation. (2)ALBERT model training. Given the augmented training pairs, three ALBERT models were trained and fine-tuned. (3)Similarity combining. Health question similarity score were calculated by combining ALBRT model outputs with text similarity. To evaluate our ALBERT-QM performance on similar question identification, we used an open dataset with 20,000 labeled Chinese health question pairs. RESULTS Our ALBERT-QM is able to identify similar Chinese health questions, achieving the precision of 86.69%, recall of 86.70% and F1 of 86.69%. Comparing with baseline method (text similarity algorithm), ALBERT-QM enhanced the F1-score by 20.73%. Comparing with other BERT series models, our ALBERT-QM is much lighter with the files size of 64.8MB which is 1/6 times that other BERT models. We made our ALBERT-QM open accessible at https://github.com/trueto/albert_question_match. CONCLUSIONS In this study, we developed an open source algorithm, ALBERT-QM, contributing to similar Chinese health questions identification in a health QA system. Our ALBERT-QM achieved better performance in question matching with lower memory usage, which is beneficial to the web-based or mobile-based QA applications.

scholarly journals Natural language inference for Malayalam language using language agnostic sentence representation

2021 ◽  
Vol 7 ◽  
pp. e508
Author(s):  
Sara Renjit ◽  
Sumam Idicula
Keyword(s):  
Natural Language ◽  
Language Processing ◽  
Question Answering ◽  
Binary Classification ◽  
Multiclass Classification ◽  
The Other ◽  
Indian Language ◽  
Information Retrieval Systems ◽  
Textual Entailment ◽  
Malayalam Language

Natural language inference (NLI) is an essential subtask in many natural language processing applications. It is a directional relationship from premise to hypothesis. A pair of texts is defined as entailed if a text infers its meaning from the other text. The NLI is also known as textual entailment recognition, and it recognizes entailed and contradictory sentences in various NLP systems like Question Answering, Summarization and Information retrieval systems. This paper describes the NLI problem attempted for a low resource Indian language Malayalam, the regional language of Kerala. More than 30 million people speak this language. The paper is about the Malayalam NLI dataset, named MaNLI dataset, and its application of NLI in Malayalam language using different models, namely Doc2Vec (paragraph vector), fastText, BERT (Bidirectional Encoder Representation from Transformers), and LASER (Language Agnostic Sentence Representation). Our work attempts NLI in two ways, as binary classification and as multiclass classification. For both the classifications, LASER outperformed the other techniques. For multiclass classification, NLI using LASER based sentence embedding technique outperformed the other techniques by a significant margin of 12% accuracy. There was also an accuracy improvement of 9% for LASER based NLI system for binary classification over the other techniques.

FindThatQuote: A Question-Answering Web-based System to Locate Quotes using Deep Learning and Natural-Language Processing

2021 ◽  
Author(s):  
Nathan Ji ◽  
Yu Sun
Keyword(s):  
Deep Learning ◽  
Natural Language Processing ◽  
Natural Language ◽  
Success Rate ◽  
Language Processing ◽  
Question Answering ◽  
Specific Information ◽  
Web Based ◽  
User Friendly ◽  
Web Based System

The digital age gives us access to a multitude of both information and mediums in which we can interpret information. A majority of the time, many people find interpreting such information difficult as the medium may not be as user friendly as possible. This project has examined the inquiry of how one can identify specific information in a given text based on a question. This inquiry is intended to streamline one's ability to determine the relevance of a given text relative to his objective. The project has an overall 80% success rate given 10 articles with three questions asked per article. This success rate indicates that this project is likely applicable to those who are asking for content level questions within an article.

scholarly journals List-wise learning to rank biomedical question-answer pairs with deep ranking recursive autoencoders

PLoS ONE ◽  
2020 ◽  
Vol 15 (11) ◽  
pp. e0242061
Author(s):  
Yan Yan ◽  
Bo-Wen Zhang ◽  
Xu-Feng Li ◽  
Zhenhan Liu
Keyword(s):  
Natural Language ◽  
Language Processing ◽  
Question Answering ◽  
Binary Classification ◽  
Learning To Rank ◽  
Relevant Literature ◽  
Semantic Relatedness ◽  
Learning Approaches ◽  
Feature Representations ◽  
Ranking Error

Biomedical question answering (QA) represents a growing concern among industry and academia due to the crucial impact of biomedical information. When mapping and ranking candidate snippet answers within relevant literature, current QA systems typically refer to information retrieval (IR) techniques: specifically, query processing approaches and ranking models. However, these IR-based approaches are insufficient to consider both syntactic and semantic relatedness and thus cannot formulate accurate natural language answers. Recently, deep learning approaches have become well-known for learning optimal semantic feature representations in natural language processing tasks. In this paper, we present a deep ranking recursive autoencoders (rankingRAE) architecture for ranking question-candidate snippet answer pairs (Q-S) to obtain the most relevant candidate answers for biomedical questions extracted from the potentially relevant documents. In particular, we convert the task of ranking candidate answers to several simultaneous binary classification tasks for determining whether a question and a candidate answer are relevant. The compositional words and their random initialized vectors of concatenated Q-S pairs are fed into recursive autoencoders to learn the optimal semantic representations in an unsupervised way, and their semantic relatedness is classified through supervised learning. Unlike several existing methods to directly choose the top-K candidates with highest probabilities, we take the influence of different ranking results into consideration. Consequently, we define a listwise “ranking error” for loss function computation to penalize inappropriate answer ranking for each question and to eliminate their influence. The proposed architecture is evaluated with respect to the BioASQ 2013-2018 Six-year Biomedical Question Answering benchmarks. Compared with classical IR models, other deep representation models, as well as some state-of-the-art systems for these tasks, the experimental results demonstrate the robustness and effectiveness of rankingRAE.

SWFQA Semantic Web Based Framework for Question Answering

2019 ◽  
Vol 9 (1) ◽  
pp. 88-106
Author(s):  
Irphan Ali ◽  
Divakar Yadav ◽  
Ashok Kumar Sharma
Keyword(s):  
Semantic Web ◽  
Natural Language ◽  
Knowledge Base ◽  
Language Processing ◽  
Question Answering ◽  
Knowledge Bases ◽  
Digital Information ◽  
Web Based ◽  
Question Answering System ◽  
User Query

A question answering system aims to provide the correct and quick answer to users' query from a knowledge base. Due to the growth of digital information on the web, information retrieval system is the need of the day. Most recent question answering systems consult knowledge bases to answer a question, after parsing and transforming natural language queries to knowledge base-executable forms. In this article, the authors propose a semantic web-based approach for question answering system that uses natural language processing for analysis and understanding the user query. It employs a “Total Answer Relevance Score” to find the relevance of each answer returned by the system. The results obtained thereof are quite promising. The real-time performance of the system has been evaluated on the answers, extracted from the knowledge base.

scholarly journals AN AUTOMATIC WEB-BASED QUESTION ANSWERING SYSTEM FOR E-LEARNING

2017 ◽  
Vol 58 (2) ◽  
pp. 1
Author(s):  
Waheeb Ahmed ◽  
Babu Anto
Keyword(s):  
Natural Language ◽  
Language Processing ◽  
Question Answering ◽  
Web Based ◽  
Question Answering System ◽  
Natural Language Text ◽  
Exact Answer ◽  
E Learning ◽  
Language Text ◽  
Develop Technique

An automatic web based Question Answering (QA) system is a valuable tool for improving e-learning and education. Several approaches employ natural language processing technology to understand questions given in natural language text, which is incomplete and error-prone. In addition, instead of extracting exact answer, many approaches simply return hyperlinks to documents containing the answers, which is inconvenient for the students or learners. In this paper we develop technique to detect the type of a question, based on which the proper technique for extracting the answer is used. The system returns only blocks or phrases of data containing the answer rather than full documents. Therefore, we can highly improve the efficiency of Web QA systems for e-learning.

UGLEO: A WEB BASED INTELLIGENCE CHATBOT FOR STUDENT ADMISSION PORTAL USING MEGAHAL STYLE

2018 ◽  
Vol 23 (3) ◽  
pp. 175-191
Author(s):  
Anneke Annassia Putri Siswadi ◽  
Avinanta Tarigan
Keyword(s):  
Markov Chain ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Information Need ◽  
Web Based ◽  
Markov Chain Method ◽  
Information Center

To fulfill the prospective student's information need about student admission, Gunadarma University has already many kinds of services which are time limited, such as website, book, registration place, Media Information Center, and Question Answering’s website (UG-Pedia). It needs a service that can serve them anytime and anywhere. Therefore, this research is developing the UGLeo as a web based QA intelligence chatbot application for Gunadarma University's student admission portal. UGLeo is developed by MegaHal style which implements the Markov Chain method. In this research, there are some modifications in MegaHal style, those modifications are the structure of natural language processing and the structure of database. The accuracy of UGLeo reply is 65%. However, to increase the accuracy there are some improvements to be applied in UGLeo system, both improvement in natural language processing and improvement in MegaHal style.

A survey of question answering in natural language processing

Poetics ◽  
1990 ◽  
Vol 19 (1-2) ◽  
pp. 99-120
Author(s):  
Stefan Wermter ◽  
Wendy G. Lehnert
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Question Answering

scholarly journals Enhancing Natural Language Inference Using New and Expanded Training Data Sets and New Learning Models

2020 ◽  
Vol 34 (05) ◽  
pp. 8504-8511
Author(s):  
Arindam Mitra ◽  
Ishan Shrivastava ◽  
Chitta Baral
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Question Answering ◽  
Training Data ◽  
Data Sets ◽  
Learning Models ◽  
New Learning ◽  
Word Attention ◽  
Attention Function

Natural Language Inference (NLI) plays an important role in many natural language processing tasks such as question answering. However, existing NLI modules that are trained on existing NLI datasets have several drawbacks. For example, they do not capture the notion of entity and role well and often end up making mistakes such as “Peter signed a deal” can be inferred from “John signed a deal”. As part of this work, we have developed two datasets that help mitigate such issues and make the systems better at understanding the notion of “entities” and “roles”. After training the existing models on the new dataset we observe that the existing models do not perform well on one of the new benchmark. We then propose a modification to the “word-to-word” attention function which has been uniformly reused across several popular NLI architectures. The resulting models perform as well as their unmodified counterparts on the existing benchmarks and perform significantly well on the new benchmarks that emphasize “roles” and “entities”.

scholarly journals Emerging trends: A gentle introduction to fine-tuning

2021 ◽  
Vol 27 (6) ◽  
pp. 763-778
Author(s):  
Kenneth Ward Church ◽  
Zeyu Chen ◽  
Yanjun Ma
Keyword(s):  
Natural Language ◽  
Language Processing ◽  
Question Answering ◽  
General Purpose ◽  
Fine Tuning ◽  
Language Engineering ◽  
Training Models ◽  
Emerging Trends ◽  
Foundation Model ◽  
Programming Skills

AbstractThe previous Emerging Trends article (Church et al., 2021. Natural Language Engineering27(5), 631–645.) introduced deep nets to poets. Poets is an imperfect metaphor, intended as a gesture toward inclusion. The future for deep nets will benefit by reaching out to a broad audience of potential users, including people with little or no programming skills, and little interest in training models. That paper focused on inference, the use of pre-trained models, as is, without fine-tuning. The goal of this paper is to make fine-tuning more accessible to a broader audience. Since fine-tuning is more challenging than inference, the examples in this paper will require modest programming skills, as well as access to a GPU. Fine-tuning starts with a general purpose base (foundation) model and uses a small training set of labeled data to produce a model for a specific downstream application. There are many examples of fine-tuning in natural language processing (question answering (SQuAD) and GLUE benchmark), as well as vision and speech.

Sign in / Sign up

Export Citation Format

Share Document