Medical Entity Relation Verification with Large-scale Machine Reading Comprehension

UQuAD1.0: Development of an Urdu Question Answering Training Data for Machine Reading Comprehension

10.36227/techrxiv.16924255 ◽

2021 ◽

Author(s):

Samreen Ahmed ◽

shakeel khoja

Keyword(s):

Reading Comprehension ◽

Machine Translation ◽

Large Scale ◽

Question Answering ◽

Training Data ◽

Significant Progress ◽

Rule Based ◽

Low Resource ◽

Machine Reading ◽

Answer Format

In recent years, low-resource Machine Reading Comprehension (MRC) has made significant progress, with models getting remarkable performance on various language datasets. However, none of these models have been customized for the Urdu language. This work explores the semi-automated creation of the Urdu Question Answering Dataset (UQuAD1.0) by combining machine-translated SQuAD with human-generated samples derived from Wikipedia articles and Urdu RC worksheets from Cambridge O-level books. UQuAD1.0 is a large-scale Urdu dataset intended for extractive machine reading comprehension tasks consisting of 49k question Answers pairs in question, passage, and answer format. In UQuAD1.0, 45000 pairs of QA were generated by machine translation of the original SQuAD1.0 and approximately 4000 pairs via crowdsourcing. In this study, we used two types of MRC models: rule-based baseline and advanced Transformer-based models. However, we have discovered that the latter outperforms the others; thus, we have decided to concentrate solely on Transformer-based architectures. Using XLMRoBERTa and multi-lingual BERT, we acquire an F1 score of 0.66 and 0.63, respectively.

Download Full-text

A Deep Cascade Model for Multi-Document Reading Comprehension

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v33i01.33017354 ◽

2019 ◽

Vol 33 ◽

pp. 7354-7361 ◽

Cited By ~ 3

Author(s):

Ming Yan ◽

Jiangnan Xia ◽

Chen Wu ◽

Bin Bi ◽

Zhongzhou Zhao ◽

...

Keyword(s):

Reading Comprehension ◽

Large Scale ◽

Question Answering ◽

Answer Extraction ◽

Benchmark Datasets ◽

Previous State ◽

Document Extraction ◽

Machine Reading ◽

System Effectiveness ◽

Extraction Experiment

A fundamental trade-off between effectiveness and efficiency needs to be balanced when designing an online question answering system. Effectiveness comes from sophisticated functions such as extractive machine reading comprehension (MRC), while efficiency is obtained from improvements in preliminary retrieval components such as candidate document selection and paragraph ranking. Given the complexity of the real-world multi-document MRC scenario, it is difficult to jointly optimize both in an end-to-end system. To address this problem, we develop a novel deep cascade learning model, which progressively evolves from the documentlevel and paragraph-level ranking of candidate texts to more precise answer extraction with machine reading comprehension. Specifically, irrelevant documents and paragraphs are first filtered out with simple functions for efficiency consideration. Then we jointly train three modules on the remaining texts for better tracking the answer: the document extraction, the paragraph extraction and the answer extraction. Experiment results show that the proposed method outperforms the previous state-of-the-art methods on two large-scale multidocument benchmark datasets, i.e., TriviaQA and DuReader. In addition, our online system can stably serve typical scenarios with millions of daily requests in less than 50ms.

Download Full-text

ReCO: A Large Scale Chinese Reading Comprehension Dataset on Opinion

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i05.6450 ◽

2020 ◽

Vol 34 (05) ◽

pp. 9146-9153

Author(s):

Bingning Wang ◽

Ting Yao ◽

Qi Zhang ◽

Jingfang Xu ◽

Xiaochuan Wang

Keyword(s):

Reading Comprehension ◽

Search Engine ◽

Large Scale ◽

Question Answering ◽

Quality Analysis ◽

Chinese Reading ◽

Large Margin ◽

Commercial Search Engine ◽

Machine Reading ◽

Support Evidence

This paper presents the ReCO, a human-curated Chinese Reading Comprehension dataset on Opinion. The questions in ReCO are opinion based queries issued to commercial search engine. The passages are provided by the crowdworkers who extract the support snippet from the retrieved documents. Finally, an abstractive yes/no/uncertain answer was given by the crowdworkers. The release of ReCO consists of 300k questions that to our knowledge is the largest in Chinese reading comprehension. A prominent characteristic of ReCO is that in addition to the original context paragraph, we also provided the support evidence that could be directly used to answer the question. Quality analysis demonstrates the challenge of ReCO that it requires various types of reasoning skills such as causal inference, logical reasoning, etc. Current QA models that perform very well on many question answering problems, such as BERT (Devlin et al. 2018), only achieves 77% accuracy on this dataset, a large margin behind humans nearly 92% performance, indicating ReCO present a good challenge for machine reading comprehension. The codes, dataset and leaderboard will be freely available at https://github.com/benywon/ReCO.

Download Full-text

UQuAD1.0: Development of an Urdu Question Answering Training Data for Machine Reading Comprehension

10.36227/techrxiv.16924255.v1 ◽

2021 ◽

Author(s):

Samreen Ahmed ◽

shakeel khoja

Keyword(s):

Reading Comprehension ◽

Machine Translation ◽

Large Scale ◽

Question Answering ◽

Training Data ◽

Significant Progress ◽

Rule Based ◽

Low Resource ◽

Machine Reading ◽

Answer Format

In recent years, low-resource Machine Reading Comprehension (MRC) has made significant progress, with models getting remarkable performance on various language datasets. However, none of these models have been customized for the Urdu language. This work explores the semi-automated creation of the Urdu Question Answering Dataset (UQuAD1.0) by combining machine-translated SQuAD with human-generated samples derived from Wikipedia articles and Urdu RC worksheets from Cambridge O-level books. UQuAD1.0 is a large-scale Urdu dataset intended for extractive machine reading comprehension tasks consisting of 49k question Answers pairs in question, passage, and answer format. In UQuAD1.0, 45000 pairs of QA were generated by machine translation of the original SQuAD1.0 and approximately 4000 pairs via crowdsourcing. In this study, we used two types of MRC models: rule-based baseline and advanced Transformer-based models. However, we have discovered that the latter outperforms the others; thus, we have decided to concentrate solely on Transformer-based architectures. Using XLMRoBERTa and multi-lingual BERT, we acquire an F1 score of 0.66 and 0.63, respectively.

Download Full-text

Large-Scale Multi-granular Concept Extraction Based on Machine Reading Comprehension

10.1007/978-3-030-88361-4_6 ◽

2021 ◽

pp. 93-110

Author(s):

Siyu Yuan ◽

Deqing Yang ◽

Jiaqing Liang ◽

Jilun Sun ◽

Jingyue Huang ◽

...

Keyword(s):

Reading Comprehension ◽

Large Scale ◽

Concept Extraction ◽

Machine Reading

Download Full-text

A Survey on Machine Reading Comprehension—Tasks, Evaluation Metrics and Benchmark Datasets

Applied Sciences ◽

10.3390/app10217640 ◽

2020 ◽

Vol 10 (21) ◽

pp. 7640

Author(s):

Changchang Zeng ◽

Shaobo Li ◽

Qin Li ◽

Jie Hu ◽

Jianjun Hu

Keyword(s):

Reading Comprehension ◽

Language Processing ◽

Large Scale ◽

Human Performance ◽

Research Field ◽

Evaluation Metrics ◽

Future Research ◽

Benchmark Datasets ◽

Comprehensive Survey ◽

Machine Reading

Machine Reading Comprehension (MRC) is a challenging Natural Language Processing (NLP) research field with wide real-world applications. The great progress of this field in recent years is mainly due to the emergence of large-scale datasets and deep learning. At present, a lot of MRC models have already surpassed human performance on various benchmark datasets despite the obvious giant gap between existing MRC models and genuine human-level reading comprehension. This shows the need for improving existing datasets, evaluation metrics, and models to move current MRC models toward “real” understanding. To address the current lack of comprehensive survey of existing MRC tasks, evaluation metrics, and datasets, herein, (1) we analyze 57 MRC tasks and datasets and propose a more precise classification method of MRC tasks with 4 different attributes; (2) we summarized 9 evaluation metrics of MRC tasks, 7 attributes and 10 characteristics of MRC datasets; (3) We also discuss key open issues in MRC research and highlighted future research directions. In addition, we have collected, organized, and published our data on the companion website where MRC researchers could directly access each MRC dataset, papers, baseline projects, and the leaderboard.

Download Full-text

Multi-Matching Network for Multiple Choice Reading Comprehension

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v33i01.33017088 ◽

2019 ◽

Vol 33 ◽

pp. 7088-7095 ◽

Cited By ~ 2

Author(s):

Min Tang ◽

Jiaran Cai ◽

Hankz Hankui Zhuo

Keyword(s):

Reading Comprehension ◽

Correct Answer ◽

Large Scale ◽

State Of The Art ◽

Multiple Choice ◽

Semantic Relationship ◽

Matching Network ◽

Empirical Results ◽

Proposed Model ◽

Machine Reading

Multiple-choice machine reading comprehension is an important and challenging task where the machine is required to select the correct answer from a set of candidate answers given passage and question. Existing approaches either match extracted evidence with candidate answers shallowly or model passage, question and candidate answers with a single paradigm of matching. In this paper, we propose Multi-Matching Network (MMN) which models the semantic relationship among passage, question and candidate answers from multiple different paradigms of matching. In our MMN model, each paradigm is inspired by how human think and designed under a unified compose-match framework. To demonstrate the effectiveness of our model, we evaluate MMN on a large-scale multiple choice machine reading comprehension dataset (i.e. RACE). Empirical results show that our proposed model achieves a significant improvement compared to strong baselines and obtains state-of-the-art results.

Download Full-text

ViReader: A wikipedia-based Vietnamese reading comprehension system using transfer learning

Journal of Intelligent & Fuzzy Systems ◽

10.3233/jifs-210683 ◽

2021 ◽

pp. 1-19

Author(s):

Kiet Van Nguyen ◽

Nhat Duy Nguyen ◽

Phong Nguyen-Thuan Do ◽

Anh Gia-Tuan Nguyen ◽

Ngan Luu-Thuy Nguyen

Keyword(s):

Reading Comprehension ◽

Transfer Learning ◽

Large Scale ◽

State Of The Art ◽

Application Programming Interface ◽

Multiple Datasets ◽

Application Programming ◽

Interest In Research ◽

The Given ◽

Machine Reading

Machine Reading Comprehension has attracted significant interest in research on natural language understanding, and large-scale datasets and neural network-based methods have been developed for this task. However, most developments of resources and methods in machine reading comprehension have been investigated using two resource-rich languages, English and Chinese. This article proposes a system called ViReader for open-domain machine reading comprehension in Vietnamese by using Wikipedia as the textual knowledge source, where the answer to any particular question is a textual span derived directly from texts on Vietnamese Wikipedia. Our system combines a sentence retriever component, based on techniques of information retrieval to extract the relevant sentences, with a transfer learning-based answer extractor trained to predict answers based on Wikipedia texts. Experiments on multiple datasets for machine reading comprehension in Vietnamese and other languages demonstrate that (1) our ViReader system is highly competitive with prevalent machine learning-based systems, and (2) multi-task learning by using a combination consisting of the sentence retriever and answer extractor is an end-to-end reading comprehension system. The sentence retriever component of our proposed system retrieves the sentences that are most likely to provide the answer response to the given question. The transfer learning-based answer extractor then reads the document from which the sentences have been retrieved, predicts the answer, and returns it to the user. The ViReader system achieves new state-of-the-art performances, with values of 70.83% EM (exact match) and 89.54% F1, outperforming the BERT-based system by 11.55% and 9.54% , respectively. It also obtains state-of-the-art performance on UIT-ViNewsQA (another Vietnamese dataset consisting of online health-domain news) and BiPaR (a bilingual dataset on English and Chinese novel texts). Compared with the BERT-based system, our system achieves significant improvements (in terms of F1) with 7.65% for English and 6.13% for Chinese on the BiPaR dataset. Furthermore, we build a ViReader application programming interface that programmers can employ in Artificial Intelligence applications.

Download Full-text

Cross-Domain Slot Filling as Machine Reading Comprehension

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2021/550 ◽

2021 ◽

Author(s):

Mengshi Yu ◽

Jian Liu ◽

Yufeng Chen ◽

Jinan Xu ◽

Yujie Zhang

Keyword(s):

Reading Comprehension ◽

Large Scale ◽

Training Data ◽

Dialogue Systems ◽

Domain Specific ◽

Cross Domain ◽

New Perspective ◽

Task Oriented ◽

Slot Filling ◽

Machine Reading

With task-oriented dialogue systems being widely applied in everyday life, slot filling, the essential component of task-oriented dialogue systems, is required to be quickly adapted to new domains that contain domain-specific slots with few or no training data. Previous methods for slot filling usually adopt sequence labeling framework, which, however, often has limited ability when dealing with the domain-specific slots. In this paper, we take a new perspective on cross-domain slot filling by framing it as a machine reading comprehension (MRC) problem. Our approach firstly transforms slot names into well-designed queries, which contain rich informative prior knowledge and are very helpful for the detection of domain-specific slots. In addition, we utilize the large-scale MRC dataset for pre-training, which further alleviates the data scarcity problem. Experimental results on SNIPS and ATIS datasets show that our approach consistently outperforms the existing state-of-the-art methods by a large margin.

Download Full-text