scholarly journals A Multi-Task Learning Machine Reading Comprehension Model for Noisy Document (Student Abstract)

2020 ◽  
Vol 34 (10) ◽  
pp. 13963-13964
Author(s):  
Zhijing Wu ◽  
Hua Xu
Keyword(s):  
Reading Comprehension ◽  
Neural Models ◽  
Successful Performance ◽  
Task Learning ◽  
Benchmark Datasets ◽  
Robustness To Noise ◽  
Learning Machine ◽  
Machine Reading ◽  
Coarse To Fine ◽  
Noise Experiment

Current neural models for Machine Reading Comprehension (MRC) have achieved successful performance in recent years. However, the model is too fragile and lack robustness to tackle the imperceptible adversarial perturbations to the input. In this work, we propose a multi-task learning MRC model with a hierarchical knowledge enrichment to further improve the robustness for noisy document. Our model follows a typical encode-align-decode framework. Additionally, we apply a hierarchical method of adding background knowledge into the model from coarse-to-fine to enhance the language representations. Besides, we optimize our model by jointly training the answer span and unanswerability prediction, aiming to improve the robustness to noise. Experiment results on benchmark datasets confirm the superiority of our method, and our method can achieve competitive performance compared with other strong baselines.

scholarly journals An Iterative Multi-Source Mutual Knowledge Transfer Framework for Machine Reading Comprehension

2020 ◽  
Author(s):  
Xin Liu ◽  
Kai Liu ◽  
Xiang Li ◽  
Jinsong Su ◽  
Yubin Ge ◽  
...  
Keyword(s):  
Reading Comprehension ◽  
Knowledge Transfer ◽  
Training Data ◽  
Target Domain ◽  
Domain Specific ◽  
Mutual Knowledge ◽  
Benchmark Datasets ◽  
Knowledge Distillation ◽  
The Many ◽  
Machine Reading

The lack of sufficient training data in many domains, poses a major challenge to the construction of domain-specific machine reading comprehension (MRC) models with satisfying performance. In this paper, we propose a novel iterative multi-source mutual knowledge transfer framework for MRC. As an extension of the conventional knowledge transfer with one-to-one correspondence, our framework focuses on the many-to-many mutual transfer, which involves synchronous executions of multiple many-to-one transfers in an iterative manner.Specifically, to update a target-domain MRC model, we first consider other domain-specific MRC models as individual teachers, and employ knowledge distillation to train a multi-domain MRC model, which is differentially required to fit the training data and match the outputs of these individual models according to their domain-level similarities to the target domain. After being initialized by the multi-domain MRC model, the target-domain MRC model is fine-tuned to match both its training data and the output of its previous best model simultaneously via knowledge distillation. Compared with previous approaches, our framework can continuously enhance all domain-specific MRC models by enabling each model to iteratively and differentially absorb the domain-shared knowledge from others. Experimental results and in-depth analyses on several benchmark datasets demonstrate the effectiveness of our framework.

scholarly journals Multi-Task Learning with Generative Adversarial Training for Multi-Passage Machine Reading Comprehension

2020 ◽  
Vol 34 (05) ◽  
pp. 8705-8712
Author(s):  
Qiyu Ren ◽  
Xiang Cheng ◽  
Sen Su
Keyword(s):  
Reading Comprehension ◽  
Open Domain ◽  
Training Process ◽  
Task Learning ◽  
Novel Approach ◽  
Adversarial Training ◽  
Model Training ◽  
Final Answer ◽  
Machine Reading ◽  
Candidate Set

Multi-passage machine reading comprehension (MRC) aims to answer a question by multiple passages. Existing multi-passage MRC approaches have shown that employing passages with and without golden answers (i.e. labeled and unlabeled passages) for model training can improve prediction accuracy. In this paper, we present MG-MRC, a novel approach for multi-passage MRC via multi-task learning with generative adversarial training. MG-MRC adopts the extract-then-select framework, where an extractor is first used to predict answer candidates, then a selector is used to choose the final answer. In MG-MRC, we adopt multi-task learning to train the extractor by using both labeled and unlabeled passages. In particular, we use labeled passages to train the extractor by supervised learning, while using unlabeled passages to train the extractor by generative adversarial training, where the extractor is regarded as the generator and a discriminator is introduced to evaluate the generated answer candidates. Moreover, to train the extractor by backpropagation in the generative adversarial training process, we propose a hybrid method which combines boundary-based and content-based extracting methods to produce the answer candidate set and its representation. The experimental results on three open-domain QA datasets confirm the effectiveness of our approach.

scholarly journals MMM: Multi-Stage Multi-Task Learning for Multi-Choice Reading Comprehension

2020 ◽  
Vol 34 (05) ◽  
pp. 8010-8017 ◽  
Author(s):  
Di Jin ◽  
Shuyang Gao ◽  
Jiun-Yu Kao ◽  
Tagyoung Chung ◽  
Dilek Hakkani-tur
Keyword(s):  
Reading Comprehension ◽  
Question Answering ◽  
Limited Data ◽  
Learning Stage ◽  
Learning Framework ◽  
Task Learning ◽  
Multi Stage ◽  
Comprehension Skills ◽  
Machine Reading ◽  
Reading Comprehension Skills

Machine Reading Comprehension (MRC) for question answering (QA), which aims to answer a question given the relevant context passages, is an important way to test the ability of intelligence systems to understand human language. Multiple-Choice QA (MCQA) is one of the most difficult tasks in MRC because it often requires more advanced reading comprehension skills such as logical reasoning, summarization, and arithmetic operations, compared to the extractive counterpart where answers are usually spans of text within given passages. Moreover, most existing MCQA datasets are small in size, making the task even harder. We introduce MMM, a Multi-stage Multi-task learning framework for Multi-choice reading comprehension. Our method involves two sequential stages: coarse-tuning stage using out-of-domain datasets and multi-task learning stage using a larger in-domain dataset to help model generalize better with limited data. Furthermore, we propose a novel multi-step attention network (MAN) as the top-level classifier for this task. We demonstrate MMM significantly advances the state-of-the-art on four representative MCQA datasets.

scholarly journals A Deep Cascade Model for Multi-Document Reading Comprehension

2019 ◽  
Vol 33 ◽  
pp. 7354-7361 ◽  
Author(s):  
Ming Yan ◽  
Jiangnan Xia ◽  
Chen Wu ◽  
Bin Bi ◽  
Zhongzhou Zhao ◽  
...  
Keyword(s):  
Reading Comprehension ◽  
Large Scale ◽  
Question Answering ◽  
Answer Extraction ◽  
Benchmark Datasets ◽  
Previous State ◽  
Document Extraction ◽  
Machine Reading ◽  
System Effectiveness ◽  
Extraction Experiment

A fundamental trade-off between effectiveness and efficiency needs to be balanced when designing an online question answering system. Effectiveness comes from sophisticated functions such as extractive machine reading comprehension (MRC), while efficiency is obtained from improvements in preliminary retrieval components such as candidate document selection and paragraph ranking. Given the complexity of the real-world multi-document MRC scenario, it is difficult to jointly optimize both in an end-to-end system. To address this problem, we develop a novel deep cascade learning model, which progressively evolves from the documentlevel and paragraph-level ranking of candidate texts to more precise answer extraction with machine reading comprehension. Specifically, irrelevant documents and paragraphs are first filtered out with simple functions for efficiency consideration. Then we jointly train three modules on the remaining texts for better tracking the answer: the document extraction, the paragraph extraction and the answer extraction. Experiment results show that the proposed method outperforms the previous state-of-the-art methods on two large-scale multidocument benchmark datasets, i.e., TriviaQA and DuReader. In addition, our online system can stably serve typical scenarios with millions of daily requests in less than 50ms.

scholarly journals Enhancement of Target-Oriented Opinion Words Extraction with Multiview-Trained Machine Reading Comprehension Model

2021 ◽  
Vol 2021 ◽  
pp. 1-13
Author(s):  
Jingyuan Zhang ◽  
Zequn Zhang ◽  
Zhi Guo ◽  
Li Jin ◽  
Kang Liu ◽  
...  
Keyword(s):  
Reading Comprehension ◽  
Question Answering ◽  
Opinion Mining ◽  
Common Knowledge ◽  
Multiple Perspectives ◽  
Fine Grained ◽  
Proposed Model ◽  
Meta Learning ◽  
Benchmark Datasets ◽  
Machine Reading

Target-oriented opinion words extraction (TOWE) seeks to identify opinion expressions oriented to a specific target, and it is a crucial step toward fine-grained opinion mining. Recent neural networks have achieved significant success in this task by building target-aware representations. However, there are still two limitations of these methods that hinder the progress of TOWE. Mainstream approaches typically utilize position indicators to mark the given target, which is a naive strategy and lacks task-specific semantic meaning. Meanwhile, the annotated target-opinion pairs contain rich latent structural knowledge from multiple perspectives, but existing methods only exploit the TOWE view. To tackle these issues, we formulate the TOWE task as a question answering (QA) problem and leverage a machine reading comprehension (MRC) model trained with a multiview paradigm to extract targeted opinions. Specifically, we introduce a template-based pseudo-question generation method and utilize deep attention interaction to build target-aware context representations and extract related opinion words. To take advantage of latent structural correlations, we further cast the opinion-target structure into three distinct yet correlated views and leverage meta-learning to aggregate common knowledge among them to enhance the TOWE task. We evaluate the proposed model on four benchmark datasets, and our method achieves new state-of-the-art results. Extensional experiments have shown that the pipeline method with our approach could surpass existing opinion pair extraction models, including joint methods that are usually believed to work better.

scholarly journals A Survey on Machine Reading Comprehension—Tasks, Evaluation Metrics and Benchmark Datasets

2020 ◽  
Vol 10 (21) ◽  
pp. 7640
Author(s):  
Changchang Zeng ◽  
Shaobo Li ◽  
Qin Li ◽  
Jie Hu ◽  
Jianjun Hu
Keyword(s):  
Reading Comprehension ◽  
Language Processing ◽  
Large Scale ◽  
Human Performance ◽  
Research Field ◽  
Evaluation Metrics ◽  
Future Research ◽  
Benchmark Datasets ◽  
Comprehensive Survey ◽  
Machine Reading

Machine Reading Comprehension (MRC) is a challenging Natural Language Processing (NLP) research field with wide real-world applications. The great progress of this field in recent years is mainly due to the emergence of large-scale datasets and deep learning. At present, a lot of MRC models have already surpassed human performance on various benchmark datasets despite the obvious giant gap between existing MRC models and genuine human-level reading comprehension. This shows the need for improving existing datasets, evaluation metrics, and models to move current MRC models toward “real” understanding. To address the current lack of comprehensive survey of existing MRC tasks, evaluation metrics, and datasets, herein, (1) we analyze 57 MRC tasks and datasets and propose a more precise classification method of MRC tasks with 4 different attributes; (2) we summarized 9 evaluation metrics of MRC tasks, 7 attributes and 10 characteristics of MRC datasets; (3) We also discuss key open issues in MRC research and highlighted future research directions. In addition, we have collected, organized, and published our data on the companion website where MRC researchers could directly access each MRC dataset, papers, baseline projects, and the leaderboard.

scholarly journals Machine Reading Comprehension Based On Multi-headed attention Model

2018 ◽  
Vol 232 ◽  
pp. 02047
Author(s):  
Hui Xu ◽  
Shichang Zhang ◽  
Jie Jiang
Keyword(s):  
Reading Comprehension ◽  
Flow Model ◽  
Original Text ◽  
Focus Attention ◽  
Fixed Size ◽  
Attention Model ◽  
Multi Stage ◽  
L Value ◽  
Machine Reading ◽  
Coarse To Fine

Machine Reading Comprehension (MRC) refers to the task that aims to read the context through the machine and answer the question about the original text, which needs to be modeled in the interaction between the context and the question. Recently, attention mechanisms in deep learning have been successfully extended to MRC tasks. In general, the attention-based approach is to focus attention on a small part of the context and to generalize it using a fixed-size vector. This paper introduces a network of attention from coarse to fine, which is a multi-stage hierarchical process. Firstly, the context and questions are encoded by bi-directional LSTM RNN; Then, more accurate interaction information is obtained after multiple iterations of the attention mechanism; Finally, a cursor-based approach is used to predicts the answer at the beginning and end of the original text. Experimental evaluation of shows that the BiDMF (Bi-Directional Multi-Attention Flow) model designed in this paper achieved 34.1% BLUE4 value and 39.5% Rouge-L value on the test set.

scholarly journals Multi-task Learning with Sample Re-weighting for Machine Reading Comprehension

2019 ◽  
Author(s):  
Yichong Xu ◽  
Xiaodong Liu ◽  
Yelong Shen ◽  
Jingjing Liu ◽  
Jianfeng Gao
Keyword(s):  
Reading Comprehension ◽  
Task Learning ◽  
Machine Reading

scholarly journals Keyword extraction method for machine reading comprehension based on natural language processing

2021 ◽  
Vol 1955 (1) ◽  
pp. 012072
Author(s):  
Ruiheng Li ◽  
Xuan Zhang ◽  
Chengdong Li ◽  
Zhongju Zheng ◽  
Zihang Zhou ◽  
...  
Keyword(s):  
Reading Comprehension ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Extraction Method ◽  
Keyword Extraction ◽  
Machine Reading
Sign in / Sign up

Export Citation Format

Share Document