Retrieving Relevant Passages Using N-grams for Open-Domain Question Answering

2019 ◽  
Vol 28 (07) ◽  
pp. 1950021
Author(s):  
Rim Faiz ◽  
Nouha Othman

Question Answering is most likely one of the toughest tasks in the field of Natural Language Processing. It aims at directly returning accurate and short answers to questions asked by users in human language over a huge collection of documents or database. Recently, the continuously exponential rise of digital information has imposed the need for more direct access to relevant answers. Thus, question answering has been the subject of a widespread attention and has been extensively explored over the last few years. Retrieving passages remains a crucial but also a challenging task in question answering. Although there has been an abundance of work on this task, this latter still implies non-trivial endeavor. In this paper, we propose an ad-hoc passage retrieval approach for Question Answering using n-grams. This approach relies on a new measure of similarity between a passage and a question for the extraction and ranking of the different passages based on n-gram overlapping. More concretely, our measure is based on the dependency degree of n-gram words of the question in the passage. We validate our approach by the development of the “SysPex” system that automatically returns the most relevant passages to a given question.

2020 ◽  
pp. 1686-1704
Author(s):  
Emna Hkiri ◽  
Souheyl Mallat ◽  
Mounir Zrigui

The event extraction task consists in determining and classifying events within an open-domain text. It is very new for the Arabic language, whereas it attained its maturity for some languages such as English and French. Events extraction was also proved to help Natural Language Processing tasks such as Information Retrieval and Question Answering, text mining, machine translation etc… to obtain a higher performance. In this article, we present an ongoing effort to build a system for event extraction from Arabic texts using Gate platform and other tools.


2020 ◽  
Author(s):  
Lana Alsabbagh ◽  
Oumayma AlDakkak ◽  
Nada Ghneim

Abstract In this paper, we present our approach to improve the performance of open-domain Arabic Question Answering systems. We focus on the passage retrieval phase which aims to retrieve the most related passages to the correct answer. To extract passages that are related to the question, the system passes through three phases: Question Analysis, Document Retrieval and Passage Retrieval. We define the passage as the sentence that ends with a dot ".". In the Question Processing phase, we applied the traditional NLP steps of tokenization, stopwords and unrelated symbols removal, and replacing the question words with their stems. We also applied Query Expansion by adding synonyms to the question words. In the Document Retrieval phase, we used the Vector Space Model (VSM) with TF-IDF vectorizer and cosine similarity. For the Passage Retrieval phase, which is the core of our system, we measured the similarity between passages and the question by a combination of the BM25 ranker and Word Embedding approach. We tested our system on ACRD dataset, which contains 1395 questions in different domains, and the system was able to achieve correct results with a precision of 92.2% and recall of 79.9% in finding the top-3 related passages for the query.


2021 ◽  
Author(s):  
Yingqi Qu ◽  
Yuchen Ding ◽  
Jing Liu ◽  
Kai Liu ◽  
Ruiyang Ren ◽  
...  

Author(s):  
Vladimir Karpukhin ◽  
Barlas Oguz ◽  
Sewon Min ◽  
Patrick Lewis ◽  
Ledell Wu ◽  
...  

2019 ◽  
Vol 9 (1) ◽  
pp. 88-106
Author(s):  
Irphan Ali ◽  
Divakar Yadav ◽  
Ashok Kumar Sharma

A question answering system aims to provide the correct and quick answer to users' query from a knowledge base. Due to the growth of digital information on the web, information retrieval system is the need of the day. Most recent question answering systems consult knowledge bases to answer a question, after parsing and transforming natural language queries to knowledge base-executable forms. In this article, the authors propose a semantic web-based approach for question answering system that uses natural language processing for analysis and understanding the user query. It employs a “Total Answer Relevance Score” to find the relevance of each answer returned by the system. The results obtained thereof are quite promising. The real-time performance of the system has been evaluated on the answers, extracted from the knowledge base.


2016 ◽  
Vol 6 (1) ◽  
pp. 36-51 ◽  
Author(s):  
Emna Hkiri ◽  
Souheyl Mallat ◽  
Mounir Zrigui

The event extraction task consists in determining and classifying events within an open-domain text. It is very new for the Arabic language, whereas it attained its maturity for some languages such as English and French. Events extraction was also proved to help Natural Language Processing tasks such as Information Retrieval and Question Answering, text mining, machine translation etc… to obtain a higher performance. In this article, we present an ongoing effort to build a system for event extraction from Arabic texts using Gate platform and other tools.


Author(s):  
Di Jin ◽  
Eileen Pan ◽  
Nassim Oufattole ◽  
Wei-Hung Weng ◽  
Hanyi Fang ◽  
...  

Open domain question answering (OpenQA) tasks have been recently attracting more and more attention from the natural language processing (NLP) community. In this work, we present the first free-form multiple-choice OpenQA dataset for solving medical problems, MedQA, collected from the professional medical board exams. It covers three languages: English, simplified Chinese, and traditional Chinese, and contains 12,723, 34,251, and 14,123 questions for the three languages, respectively. We implement both rule-based and popular neural methods by sequentially combining a document retriever and a machine comprehension model. Through experiments, we find that even the current best method can only achieve 36.7%, 42.0%, and 70.1% of test accuracy on the English, traditional Chinese, and simplified Chinese questions, respectively. We expect MedQA to present great challenges to existing OpenQA systems and hope that it can serve as a platform to promote much stronger OpenQA models from the NLP community in the future.


2021 ◽  
Vol 11 (14) ◽  
pp. 6421
Author(s):  
Di Jin ◽  
Eileen Pan ◽  
Nassim Oufattole ◽  
Wei-Hung Weng ◽  
Hanyi Fang ◽  
...  

Open domain question answering (OpenQA) tasks have been recently attracting more and more attention from the natural language processing (NLP) community. In this work, we present the first free-form multiple-choice OpenQA dataset for solving medical problems, MedQA, collected from the professional medical board exams. It covers three languages: English, simplified Chinese, and traditional Chinese, and contains 12,723, 34,251, and 14,123 questions for the three languages, respectively. We implement both rule-based and popular neural methods by sequentially combining a document retriever and a machine comprehension model. Through experiments, we find that even the current best method can only achieve 36.7%, 42.0%, and 70.1% of test accuracy on the English, traditional Chinese, and simplified Chinese questions, respectively. We expect MedQA to present great challenges to existing OpenQA systems and hope that it can serve as a platform to promote much stronger OpenQA models from the NLP community in the future.


2013 ◽  
Vol 19 (3) ◽  
pp. 289-290 ◽  
Author(s):  
STAN SZPAKOWICZ ◽  
FRANCIS BOND ◽  
PRESLAV NAKOV ◽  
SU NAM KIM

The noun compound – a sequence of nouns which functions as a single noun – is very common in English texts. No language processing system should ignore expressions like steel soup pot cover if it wants to be serious about such high-end applications of computational linguistics as question answering, information extraction, text summarization, machine translation – the list goes on. Processing noun compounds, however, is far from trouble-free. For one thing, they can be bracketed in various ways: is it steel soup, steel pot, or steel cover? Then there are relations inside a compound, annoyingly not signalled by any words: does potcontainsoup or is it for cookingsoup? These and many other research challenges are the subject of this special issue.


Sign in / Sign up

Export Citation Format

Share Document