scholarly journals Investigating Prior Knowledge for Challenging Chinese Machine Reading Comprehension

2020 ◽  
Vol 8 ◽  
pp. 141-155
Author(s):  
Kai Sun ◽  
Dian Yu ◽  
Dong Yu ◽  
Claire Cardie
Keyword(s):  
Reading Comprehension ◽  
Prior Knowledge ◽  
Data Augmentation ◽  
Multiple Choice ◽  
Model Performance ◽  
Free Form ◽  
World Knowledge ◽  
Domain Specific ◽  
Significant Performance ◽  
Machine Reading

Machine reading comprehension tasks require a machine reader to answer questions relevant to the given document. In this paper, we present the first free-form multiple-Choice Chinese machine reading Comprehension dataset (C3), containing 13,369 documents (dialogues or more formally written mixed-genre texts) and their associated 19,577 multiple-choice free-form questions collected from Chinese-as-a-second-language examinations. We present a comprehensive analysis of the prior knowledge (i.e., linguistic, domain-specific, and general world knowledge) needed for these real-world problems. We implement rule-based and popular neural methods and find that there is still a significant performance gap between the best performing model (68.5%) and human readers (96.0%), especiallyon problems that require prior knowledge. We further study the effects of distractor plausibility and data augmentation based on translated relevant datasets for English on model performance. We expect C3 to present great challenges to existing systems as answering 86.8% of questions requires both knowledge within and beyond the accompanying document, and we hope that C3 can serve as a platform to study how to leverage various kinds of prior knowledge to better understand a given written or orally oriented text. C3 is available at https://dataset.org/c3/ .

scholarly journals Analyzing the Effect of Masking Length Distribution of MLM: An Evaluation Framework and Case Study on Chinese MRC Datasets

2021 ◽  
Vol 2021 ◽  
pp. 1-17
Author(s):  
Changchang Zeng ◽  
Shaobo Li
Keyword(s):  
Reading Comprehension ◽  
Language Processing ◽  
Question Answering ◽  
Multiple Choice ◽  
Length Distribution ◽  
Research Field ◽  
Evaluation Framework ◽  
Language Models ◽  
Training Objective ◽  
Machine Reading

Machine reading comprehension (MRC) is a challenging natural language processing (NLP) task. It has a wide application potential in the fields of question answering robots, human-computer interactions in mobile virtual reality systems, etc. Recently, the emergence of pretrained models (PTMs) has brought this research field into a new era, in which the training objective plays a key role. The masked language model (MLM) is a self-supervised training objective widely used in various PTMs. With the development of training objectives, many variants of MLM have been proposed, such as whole word masking, entity masking, phrase masking, and span masking. In different MLMs, the length of the masked tokens is different. Similarly, in different machine reading comprehension tasks, the length of the answer is also different, and the answer is often a word, phrase, or sentence. Thus, in MRC tasks with different answer lengths, whether the length of MLM is related to performance is a question worth studying. If this hypothesis is true, it can guide us on how to pretrain the MLM with a relatively suitable mask length distribution for MRC tasks. In this paper, we try to uncover how much of MLM’s success in the machine reading comprehension tasks comes from the correlation between masking length distribution and answer length in the MRC dataset. In order to address this issue, herein, (1) we propose four MRC tasks with different answer length distributions, namely, the short span extraction task, long span extraction task, short multiple-choice cloze task, and long multiple-choice cloze task; (2) four Chinese MRC datasets are created for these tasks; (3) we also have pretrained four masked language models according to the answer length distributions of these datasets; and (4) ablation experiments are conducted on the datasets to verify our hypothesis. The experimental results demonstrate that our hypothesis is true. On four different machine reading comprehension datasets, the performance of the model with correlation length distribution surpasses the model without correlation.

scholarly journals An Iterative Multi-Source Mutual Knowledge Transfer Framework for Machine Reading Comprehension

2020 ◽  
Author(s):  
Xin Liu ◽  
Kai Liu ◽  
Xiang Li ◽  
Jinsong Su ◽  
Yubin Ge ◽  
...  
Keyword(s):  
Reading Comprehension ◽  
Knowledge Transfer ◽  
Training Data ◽  
Target Domain ◽  
Domain Specific ◽  
Mutual Knowledge ◽  
Benchmark Datasets ◽  
Knowledge Distillation ◽  
The Many ◽  
Machine Reading

The lack of sufficient training data in many domains, poses a major challenge to the construction of domain-specific machine reading comprehension (MRC) models with satisfying performance. In this paper, we propose a novel iterative multi-source mutual knowledge transfer framework for MRC. As an extension of the conventional knowledge transfer with one-to-one correspondence, our framework focuses on the many-to-many mutual transfer, which involves synchronous executions of multiple many-to-one transfers in an iterative manner.Specifically, to update a target-domain MRC model, we first consider other domain-specific MRC models as individual teachers, and employ knowledge distillation to train a multi-domain MRC model, which is differentially required to fit the training data and match the outputs of these individual models according to their domain-level similarities to the target domain. After being initialized by the multi-domain MRC model, the target-domain MRC model is fine-tuned to match both its training data and the output of its previous best model simultaneously via knowledge distillation. Compared with previous approaches, our framework can continuously enhance all domain-specific MRC models by enabling each model to iteratively and differentially absorb the domain-shared knowledge from others. Experimental results and in-depth analyses on several benchmark datasets demonstrate the effectiveness of our framework.

scholarly journals Convolutional Spatial Attention Model for Reading Comprehension with Multiple-Choice Questions

2019 ◽  
Vol 33 ◽  
pp. 6276-6283 ◽  
Author(s):  
Zhipeng Chen ◽  
Yiming Cui ◽  
Wentao Ma ◽  
Shijin Wang ◽  
Guoping Hu
Keyword(s):  
Reading Comprehension ◽  
Mutual Information ◽  
Spatial Attention ◽  
State Of The Art ◽  
Multiple Choice ◽  
Multiple Choice Questions ◽  
Attention Model ◽  
Novel Approach ◽  
Proposed Model ◽  
Machine Reading

Machine Reading Comprehension (MRC) with multiplechoice questions requires the machine to read given passage and select the correct answer among several candidates. In this paper, we propose a novel approach called Convolutional Spatial Attention (CSA) model which can better handle the MRC with multiple-choice questions. The proposed model could fully extract the mutual information among the passage, question, and the candidates, to form the enriched representations. Furthermore, to merge various attention results, we propose to use convolutional operation to dynamically summarize the attention values within the different size of regions. Experimental results show that the proposed model could give substantial improvements over various state-of- the-art systems on both RACE and SemEval-2018 Task11 datasets.

The Effect of Perceived Interest and Prior Knowledge on L2 Reading Comprehension via Several Assessment Methods in a Higher Education Context

2019 ◽  
Vol 5 (1) ◽  
pp. 52
Author(s):  
Ana Cristina Lahuerta Martínez
Keyword(s):  
Reading Comprehension ◽  
Prior Knowledge ◽  
Multiple Choice ◽  
Assessment Method ◽  
Multiple Choice Questions ◽  
L2 Reading ◽  
Assessment Tasks ◽  
Written Recall ◽  
The Relationship ◽  
L2 Reading Comprehension

The aim of the present study is to examine the effect of perceived interest and prior knowledge on EFL reading comprehension. Participants were 227 undergraduates with advanced competence in English. With respect to the method, participants had to read a 450-word text entitled Wales. After that, they had to complete a Perceived Interest Questionnaire (PIQ), which consisted of 9 items and two assessment tasks: a written recall and a multiple choice task. The results of our study show the significant effect of perceived interest and prior knowledge on L2 reading comprehension. Thus, comprehension assessed via written recall and multiple choice questions had higher scores when readers read texts related to their interests. Besides, prior knowledge had a positive effect on the reader’s comprehension irrespective of the assessment method used. This study concludes that different assessment tasks may be crucial factors that affect the relationship between factors like interest and prior knowledge, and L2 reading comprehension.

scholarly journals Enhancing Lexical-Based Approach With External Knowledge for Vietnamese Multiple-Choice Machine Reading Comprehension

IEEE Access ◽  
2020 ◽  
Vol 8 ◽  
pp. 201404-201417
Author(s):  
Kiet Van Nguyen ◽  
Khiem Vinh Tran ◽  
Son T. Luu ◽  
Anh Gia-Tuan Nguyen ◽  
Ngan Luu-Thuy Nguyen
Keyword(s):  
Reading Comprehension ◽  
Multiple Choice ◽  
External Knowledge ◽  
Machine Reading

scholarly journals Multi-Matching Network for Multiple Choice Reading Comprehension

2019 ◽  
Vol 33 ◽  
pp. 7088-7095 ◽  
Author(s):  
Min Tang ◽  
Jiaran Cai ◽  
Hankz Hankui Zhuo
Keyword(s):  
Reading Comprehension ◽  
Correct Answer ◽  
Large Scale ◽  
State Of The Art ◽  
Multiple Choice ◽  
Semantic Relationship ◽  
Matching Network ◽  
Empirical Results ◽  
Proposed Model ◽  
Machine Reading

Multiple-choice machine reading comprehension is an important and challenging task where the machine is required to select the correct answer from a set of candidate answers given passage and question. Existing approaches either match extracted evidence with candidate answers shallowly or model passage, question and candidate answers with a single paradigm of matching. In this paper, we propose Multi-Matching Network (MMN) which models the semantic relationship among passage, question and candidate answers from multiple different paradigms of matching. In our MMN model, each paradigm is inspired by how human think and designed under a unified compose-match framework. To demonstrate the effectiveness of our model, we evaluate MMN on a large-scale multiple choice machine reading comprehension dataset (i.e. RACE). Empirical results show that our proposed model achieves a significant improvement compared to strong baselines and obtains state-of-the-art results.

scholarly journals The Influence of Prior Knowledge on Students’ Listening and Reading Comprehension

2018 ◽  
Vol 4 (1) ◽  
pp. 1-15
Author(s):  
Abdullah Hasan ◽  
Rizky Gushendra ◽  
Ferri Yonantha
Keyword(s):  
Multivariate Analysis ◽  
Reading Comprehension ◽  
Prior Knowledge ◽  
Analysis Of Variance ◽  
Listening Comprehension ◽  
English Education ◽  
Multiple Choice ◽  
Sampling Technique ◽  
Multivariate Analysis Of Variance ◽  
The Third

ABSTRACTThe research aims to investigate the influence of prior knowledge on students’ listening and reading comprehension at the tenth year of MAN 1 Pekanbaru. This study is a correlational research that involved 75 respondents as a sample from 150 students of the the tenth year of Science classes as the total population. The respondents were selected by using a simple random sampling technique. 20 items of multiple choice of listening test and 20 items of multiple choice of reading test and 15 items of the prior knowledge questionnaire were used to collect the data. Afterwards, the data were further analyzed by using Pearson product moment correlation for the first and the second hypotheses and MANOVA (Multivariate Analysis of Variance) for the third hypothesis by using SPSS 25. The research findings revealed that the mean score of students’ prior knowledge is 73.41 and is categorized as “Good”, their listening comprehension is 68.13 and is categorized as “Good”, and their reading comprehension is 70.67 and it is also categorized as “Good”. It can be seen that the value of Sig. (2-tailed) is 0.000<0.05. Lastly, for the third hypothesis, the value of significance is 0.000<0.05. It means Ha is accepted. Then, it is generated that there is a significant influence of prior knowledge on both students’ listening comprehension and reading comprehension.ABSTRAKPenelitian  ini  bertujuan untuk  mengetahui pengaruh pengetahuan yang ada pada siswa terhadap pemahaman listening dan speaking siswa kelas X MAN 1 Pekanbaru. Kajian ini merupakan penelitian korelasi yang terdiri dari 75 responden sebagai sampel dari populasi 150 orang siswa kelas X jurusan IPA dengan menggunakan tehnik sampel acak. tes Listening  dan Reading menggunakan pilihan ganda yang masing-masingnya terdiri dari 20 soal. Kuesioner digunakan untuk mengumpulkan data. Kuesioner ini terdiri dari 15 pernyataan. Selanjutnya data dianalisa dengan menggunakan korelasi Pearson Product Moment untuk hipotesis yang pertama dan kedua, sedangkan untuk menganalisa hipotesis yang ketiga menggunakan MANOVA (Multivariate Analysis of Variance) SPSS versi 25. Hasil penelitian mengungkapkan nilai rata-rata pengetahuan yang sudah ada adalah 73.41 dan dikategorikan “Baik”. Nilai rata-rata listening comprehension  68.13 dan dikategorikan “Baik” serta nilai rata-rata reading comprehension 70.67 dan juga dikategorikan “Baik”  Dapat diketahui bahwa nilai signifikan (2;tailed) adalah 0.000<0.05. Akhirnya, untuk hipotesa yang ketiga nilai signifikannya 0.000<0.05. Hasil ini menunjukkan bahwa Ha diterima. Kemudian, disimpulkan bahwa terdapat pengaruh yang signifikan prior knowledge siswa terhadap listening dan reading comprehension. How to Cite: Hasan, A. Gushendra, R. Yonantha, F. (2017). The Influence of Prior Knowledge on Students’ Listening and Reading Comprehension. IJEE (Indonesian Journal of English Education), 4(1), 1-15 doi:10.15408/ijee.v4i1.4744.DOI: http://dx.doi.org/10.15408/ijee.v4i1.4744

scholarly journals A Multiple-Choice Machine Reading Comprehension Model with Multi-Granularity Semantic Reasoning

2021 ◽  
Vol 11 (17) ◽  
pp. 7945
Author(s):  
Yu Dai ◽  
Yufan Fu ◽  
Lei Yang
Keyword(s):  
Reading Comprehension ◽  
Semantic Information ◽  
Multiple Choice ◽  
Global Information ◽  
Reasoning Ability ◽  
Semantic Reasoning ◽  
Benchmark Model ◽  
Proposed Model ◽  
Convolution Kernels ◽  
Machine Reading

To address the problem of poor semantic reasoning of models in multiple-choice Chinese machine reading comprehension (MRC), this paper proposes an MRC model incorporating multi-granularity semantic reasoning. In this work, we firstly encode articles, questions and candidates to extract global reasoning information; secondly, we use multiple convolution kernels of different sizes to convolve and maximize pooling of the BERT-encoded articles, questions and candidates to extract local semantic reasoning information of different granularities; we then fuse the global information with the local multi-granularity information and use it to make an answer selection. The proposed model can combine the learned multi-granularity semantic information for reasoning, solving the problem of poor semantic reasoning ability of the model, and thus can improve the reasoning ability of machine reading comprehension. The experiments show that the proposed model achieves better performance on the C3 dataset than the benchmark model in semantic reasoning, which verifies the effectiveness of the proposed model in semantic reasoning.

Sign in / Sign up

Export Citation Format

Share Document