What Disease Does This Patient Have? A Large-Scale Open Domain Question Answering Dataset from Medical Exams

Open domain question answering (OpenQA) tasks have been recently attracting more and more attention from the natural language processing (NLP) community. In this work, we present the first free-form multiple-choice OpenQA dataset for solving medical problems, MedQA, collected from the professional medical board exams. It covers three languages: English, simplified Chinese, and traditional Chinese, and contains 12,723, 34,251, and 14,123 questions for the three languages, respectively. We implement both rule-based and popular neural methods by sequentially combining a document retriever and a machine comprehension model. Through experiments, we find that even the current best method can only achieve 36.7%, 42.0%, and 70.1% of test accuracy on the English, traditional Chinese, and simplified Chinese questions, respectively. We expect MedQA to present great challenges to existing OpenQA systems and hope that it can serve as a platform to promote much stronger OpenQA models from the NLP community in the future.

Download Full-text

Structured Two-Stream Attention Network for Video Question Answering

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v33i01.33016391 ◽

2019 ◽

Vol 33 ◽

pp. 6391-6398 ◽

Cited By ~ 3

Author(s):

Lianli Gao ◽

Pengpeng Zeng ◽

Jingkuan Song ◽

Yuan-Fang Li ◽

Wu Liu ◽

...

Keyword(s):

Large Scale ◽

Question Answering ◽

Free Form ◽

Image Region ◽

Attention Network ◽

Natural Language Question ◽

Temporal Structures ◽

Text Features ◽

Video Input ◽

Language Question

To date, visual question answering (VQA) (i.e., image QA and video QA) is still a holy grail in vision and language understanding, especially for video QA. Compared with image QA that focuses primarily on understanding the associations between image region-level details and corresponding questions, video QA requires a model to jointly reason across both spatial and long-range temporal structures of a video as well as text to provide an accurate answer. In this paper, we specifically tackle the problem of video QA by proposing a Structured Two-stream Attention network, namely STA, to answer a free-form or open-ended natural language question about the content of a given video. First, we infer rich longrange temporal structures in videos using our structured segment component and encode text features. Then, our structured two-stream attention component simultaneously localizes important visual instance, reduces the influence of background video and focuses on the relevant text. Finally, the structured two-stream fusion component incorporates different segments of query and video aware context representation and infers the answers. Experiments on the large-scale video QA dataset TGIF-QA show that our proposed method significantly surpasses the best counterpart (i.e., with one representation for the video input) by 13.0%, 13.5%, 11.0% and 0.3 for Action, Trans., TrameQA and Count tasks. It also outperforms the best competitor (i.e., with two representations) on the Action, Trans., TrameQA tasks by 4.1%, 4.7%, and 5.1%.

Download Full-text

Events Automatic Extraction from Arabic Texts

Natural Language Processing ◽

10.4018/978-1-7998-0951-7.ch078 ◽

2020 ◽

pp. 1686-1704

Author(s):

Emna Hkiri ◽

Souheyl Mallat ◽

Mounir Zrigui

Keyword(s):

Information Retrieval ◽

Natural Language Processing ◽

Text Mining ◽

Machine Translation ◽

Language Processing ◽

Question Answering ◽

Arabic Language ◽

Event Extraction ◽

Mining Machine ◽

Open Domain

The event extraction task consists in determining and classifying events within an open-domain text. It is very new for the Arabic language, whereas it attained its maturity for some languages such as English and French. Events extraction was also proved to help Natural Language Processing tasks such as Information Retrieval and Question Answering, text mining, machine translation etc… to obtain a higher performance. In this article, we present an ongoing effort to build a system for event extraction from Arabic texts using Gate platform and other tools.

Download Full-text

Predicting Lexical Answer Types in Open Domain QA

International Journal on Semantic Web and Information Systems ◽

10.4018/jswis.2012070104 ◽

2012 ◽

Vol 8 (3) ◽

pp. 74-88 ◽

Cited By ~ 1

Author(s):

Alfio Massimiliano Gliozzo ◽

Aditya Kalyanpur

Keyword(s):

Knowledge Acquisition ◽

Knowledge Base ◽

Large Scale ◽

Question Answering ◽

General Purpose ◽

Open Domain ◽

Lexical Knowledge ◽

Research Challenge ◽

Lexical Knowledge Base

Automatic open-domain Question Answering has been a long standing research challenge in the AI community. IBM Research undertook this challenge with the design of the DeepQA architecture and the implementation of Watson. This paper addresses a specific subtask of Deep QA, consisting of predicting the Lexical Answer Type (LAT) of a question. Our approach is completely unsupervised and is based on PRISMATIC, a large-scale lexical knowledge base automatically extracted from a Web corpus. Experiments on the Jeopardy! data shows that it is possible to correctly predict the LAT in a substantial number of questions. This approach can be used for general purpose knowledge acquisition tasks such as frame induction from text.

Download Full-text

Research and Implementation of Intelligent Question Answering System in MOOC

Applied Mechanics and Materials ◽

10.4028/www.scientific.net/amm.678.639 ◽

2014 ◽

Vol 678 ◽

pp. 639-643

Author(s):

Wei Jun Dong ◽

Guo Hua Geng

Keyword(s):

Language Processing ◽

Large Scale ◽

Question Answering ◽

Open Educational Resource ◽

Agent Model ◽

Question Answering System ◽

Passive Learning ◽

Self Development ◽

The Status ◽

Massive Online Open Course

Massive Online Open Course which based on Open Educational Resource might be the most effective method to large-scale quality education, which can realize passive learning to active learning. Analyzing the status and shortages of Intelligent Answering System, propose and design an intelligent question answering system based on agent-model. System use software agents to implement and improve MOOC system’s Intelligent Answering System performance, which has capacity of natural language processing, and good versatility. It can provide an efficient online problem answer environment for thousands of learners, and can effectively promote students' autonomous learning and self-development.

Download Full-text

UQuAD1.0: Development of an Urdu Question Answering Training Data for Machine Reading Comprehension

10.36227/techrxiv.16924255 ◽

2021 ◽

Author(s):

Samreen Ahmed ◽

shakeel khoja

Keyword(s):

Reading Comprehension ◽

Machine Translation ◽

Large Scale ◽

Question Answering ◽

Training Data ◽

Significant Progress ◽

Rule Based ◽

Low Resource ◽

Machine Reading ◽

Answer Format

In recent years, low-resource Machine Reading Comprehension (MRC) has made significant progress, with models getting remarkable performance on various language datasets. However, none of these models have been customized for the Urdu language. This work explores the semi-automated creation of the Urdu Question Answering Dataset (UQuAD1.0) by combining machine-translated SQuAD with human-generated samples derived from Wikipedia articles and Urdu RC worksheets from Cambridge O-level books. UQuAD1.0 is a large-scale Urdu dataset intended for extractive machine reading comprehension tasks consisting of 49k question Answers pairs in question, passage, and answer format. In UQuAD1.0, 45000 pairs of QA were generated by machine translation of the original SQuAD1.0 and approximately 4000 pairs via crowdsourcing. In this study, we used two types of MRC models: rule-based baseline and advanced Transformer-based models. However, we have discovered that the latter outperforms the others; thus, we have decided to concentrate solely on Transformer-based architectures. Using XLMRoBERTa and multi-lingual BERT, we acquire an F1 score of 0.66 and 0.63, respectively.

Download Full-text

A Composite Natural Language Processing and Information Retrieval Approach to Question Answering Using a Structured Knowledge Base

International Journal of Semantic Computing ◽

10.1142/s1793351x17400141 ◽

2017 ◽

Vol 11 (03) ◽

pp. 345-371

Author(s):

Avani Chandurkar ◽

Ajay Bansal

Keyword(s):

Information Retrieval ◽

Natural Language Processing ◽

Natural Language ◽

Knowledge Base ◽

Language Processing ◽

Question Answering ◽

Automated System ◽

Free Form ◽

Question Answering System ◽

Novel Approach

With the inception of the World Wide Web, the amount of data present on the Internet is tremendous. This makes the task of navigating through this enormous amount of data quite difficult for the user. As users struggle to navigate through this wealth of information, the need for the development of an automated system that can extract the required information becomes urgent. This paper presents a Question Answering system to ease the process of information retrieval. Question Answering systems have been around for quite some time and are a sub-field of information retrieval and natural language processing. The task of any Question Answering system is to seek an answer to a free form factual question. The difficulty of pinpointing and verifying the precise answer makes question answering more challenging than simple information retrieval done by search engines. The research objective of this paper is to develop a novel approach to Question Answering based on a composition of conventional approaches of Information Retrieval (IR) and Natural Language processing (NLP). The focus is on using a structured and annotated knowledge base instead of an unstructured one. The knowledge base used here is DBpedia and the final system is evaluated on the Text REtrieval Conference (TREC) 2004 questions dataset.

Download Full-text

Learning to Rank Answers to Non-Factoid Questions from Web Collections

Computational Linguistics ◽

10.1162/coli_a_00051 ◽

2011 ◽

Vol 37 (2) ◽

pp. 351-383 ◽

Cited By ~ 65

Author(s):

Mihai Surdeanu ◽

Massimiliano Ciaramita ◽

Hugo Zaragoza

Keyword(s):

Language Processing ◽

Large Scale ◽

Question Answering ◽

Learning To Rank ◽

Word Sense Disambiguation ◽

Word Sense ◽

Semantic Role Labeling ◽

Linguistic Features ◽

Wide Range ◽

Entity Identification

This work investigates the use of linguistically motivated features to improve search, in particular for ranking answers to non-factoid questions. We show that it is possible to exploit existing large collections of question–answer pairs (from online social Question Answering sites) to extract such features and train ranking models which combine them effectively. We investigate a wide range of feature types, some exploiting natural language processing such as coarse word sense disambiguation, named-entity identification, syntactic parsing, and semantic role labeling. Our experiments demonstrate that linguistic features, in combination, yield considerable improvements in accuracy. Depending on the system settings we measure relative improvements of 14% to 21% in Mean Reciprocal Rank and Precision@1, providing one of the most compelling evidence to date that complex linguistic features such as word senses and semantic roles can have a significant impact on large-scale information retrieval tasks.

Download Full-text

External features enriched model for biomedical question answering

BMC Bioinformatics ◽

10.1186/s12859-021-04176-7 ◽

2021 ◽

Vol 22 (1) ◽

Author(s):

Gezheng Xu ◽

Wenge Rong ◽

Yanmeng Wang ◽

Yuanxin Ouyang ◽

Zhang Xiong

Keyword(s):

Language Processing ◽

Large Scale ◽

Question Answering ◽

Language Model ◽

Named Entity Recognition ◽

Entity Recognition ◽

Small Scale ◽

Original Text ◽

Specific Domain ◽

Part Of Speech

Abstract Background Biomedical question answering (QA) is a sub-task of natural language processing in a specific domain, which aims to answer a question in the biomedical field based on one or more related passages and can provide people with accurate healthcare-related information. Recently, a lot of approaches based on the neural network and large scale pre-trained language model have largely improved its performance. However, considering the lexical characteristics of biomedical corpus and its small scale dataset, there is still much improvement room for biomedical QA tasks. Results Inspired by the importance of syntactic and lexical features in the biomedical corpus, we proposed a new framework to extract external features, such as part-of-speech and named-entity recognition, and fused them with the original text representation encoded by pre-trained language model, to enhance the biomedical question answering performance. Our model achieves an overall improvement of all three metrics on BioASQ 6b, 7b, and 8b factoid question answering tasks. Conclusions The experiments on BioASQ question answering dataset demonstrated the effectiveness of our external feature-enriched framework. It is proven by the experiments conducted that external lexical and syntactic features can improve Pre-trained Language Model’s performance in biomedical domain question answering task.

Download Full-text

UQuAD1.0: Development of an Urdu Question Answering Training Data for Machine Reading Comprehension

10.36227/techrxiv.16924255.v1 ◽

2021 ◽

Author(s):

Samreen Ahmed ◽

shakeel khoja

Keyword(s):

Reading Comprehension ◽

Machine Translation ◽

Large Scale ◽

Question Answering ◽

Training Data ◽

Significant Progress ◽

Rule Based ◽

Low Resource ◽

Machine Reading ◽

Answer Format

In recent years, low-resource Machine Reading Comprehension (MRC) has made significant progress, with models getting remarkable performance on various language datasets. However, none of these models have been customized for the Urdu language. This work explores the semi-automated creation of the Urdu Question Answering Dataset (UQuAD1.0) by combining machine-translated SQuAD with human-generated samples derived from Wikipedia articles and Urdu RC worksheets from Cambridge O-level books. UQuAD1.0 is a large-scale Urdu dataset intended for extractive machine reading comprehension tasks consisting of 49k question Answers pairs in question, passage, and answer format. In UQuAD1.0, 45000 pairs of QA were generated by machine translation of the original SQuAD1.0 and approximately 4000 pairs via crowdsourcing. In this study, we used two types of MRC models: rule-based baseline and advanced Transformer-based models. However, we have discovered that the latter outperforms the others; thus, we have decided to concentrate solely on Transformer-based architectures. Using XLMRoBERTa and multi-lingual BERT, we acquire an F1 score of 0.66 and 0.63, respectively.

Download Full-text