scholarly journals MOSS: End-to-End Dialog System Framework with Modular Supervision

2020 ◽  
Vol 34 (05) ◽  
pp. 8327-8335
Author(s):  
Weixin Liang ◽  
Youzhi Tian ◽  
Chengcai Chen ◽  
Zhou Yu
Keyword(s):  
Natural Language ◽  
Customer Service ◽  
State Of The Art ◽  
Training Data ◽  
Language Understanding ◽  
Language Generation ◽  
Dialog System ◽  
Major Bottleneck ◽  
End To End ◽  
Task Oriented

A major bottleneck in training end-to-end task-oriented dialog system is the lack of data. To utilize limited training data more efficiently, we propose Modular Supervision Network (MOSS), an encoder-decoder training framework that could incorporate supervision from various intermediate dialog system modules including natural language understanding, dialog state tracking, dialog policy learning and natural language generation. With only 60% of the training data, MOSS-all (i.e., MOSS with supervision from all four dialog modules) outperforms state-of-the-art models on CamRest676. Moreover, introducing modular supervision has even bigger benefits when the dialog task has a more complex dialog state and action space. With only 40% of the training data, MOSS-all outperforms the state-of-the-art model on a complex laptop network trouble shooting dataset, LaptopNetwork, that we introduced. LaptopNetwork consists of conversations between real customers and customer service agents in Chinese. Moreover, MOSS framework can accommodate dialogs that have supervision from different dialog modules at both framework level and model level. Therefore, MOSS is extremely flexible to update in real-world deployment.

scholarly journals Triple-to-Text Generation with an Anchor-to-Prototype Framework

2020 ◽  
Author(s):  
Ziran Li ◽  
Zibo Lin ◽  
Ning Ding ◽  
Hai-Tao Zheng ◽  
Ying Shen
Keyword(s):  
Natural Language ◽  
State Of The Art ◽  
Experimental Results ◽  
Training Data ◽  
Generation Process ◽  
Text Generation ◽  
Language Generation ◽  
Structured Input ◽  
Textual Description ◽  
Specific Description

Generating a textual description from a set of RDF triplets is a challenging task in natural language generation. Recent neural methods have become the mainstream for this task, which often generate sentences from scratch. However, due to the huge gap between the structured input and the unstructured output, the input triples alone are insufficient to decide an expressive and specific description. In this paper, we propose a novel anchor-to-prototype framework to bridge the gap between structured RDF triples and natural text. The model retrieves a set of prototype descriptions from the training data and extracts writing patterns from them to guide the generation process. Furthermore, to make a more precise use of the retrieved prototypes, we employ a triple anchor that aligns the input triples into groups so as to better match the prototypes. Experimental results on both English and Chinese datasets show that our method significantly outperforms the state-of-the-art baselines in terms of both automatic and manual evaluation, demonstrating the benefit of learning guidance from retrieved prototypes to facilitate triple-to-text generation.

scholarly journals Coreference Resolution: Toward End-to-End and Cross-Lingual Systems

Information ◽  
2020 ◽  
Vol 11 (2) ◽  
pp. 74 ◽  
Author(s):  
André Ferreira Cruz ◽  
Gil Rocha ◽  
Henrique Lopes Cardoso
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
State Of The Art ◽  
Coreference Resolution ◽  
Language Understanding ◽  
External Resources ◽  
End To End ◽  
Cross Lingual ◽  
Open Issues

The task of coreference resolution has attracted considerable attention in the literature due to its importance in deep language understanding and its potential as a subtask in a variety of complex natural language processing problems. In this study, we outlined the field’s terminology, describe existing metrics, their differences and shortcomings, as well as the available corpora and external resources. We analyzed existing state-of-the-art models and approaches, and reviewed recent advances and trends in the field, namely end-to-end systems that jointly model different subtasks of coreference resolution, and cross-lingual systems that aim to overcome the challenges of less-resourced languages. Finally, we discussed the main challenges and open issues faced by coreference resolution systems.

The Story Of Computational Narratology

2018 ◽  
Author(s):  
Sharath Srivatsa ◽  
Shyam Kumar V N ◽  
Srinath Srinivasa
Keyword(s):  
Natural Language ◽  
Computational Modeling ◽  
Narrative Structure ◽  
Complex Problem ◽  
General Intelligence ◽  
Language Understanding ◽  
Language Generation ◽  
Narrative Generation ◽  
Growing Body ◽  
Computational Narratology

In recent times, computational modeling of narratives has gained enormous interest in fields like Natural Language Understanding (NLU), Natural Language Generation (NLG), and Artificial General Intelligence (AGI). There is a growing body of literature addressing understanding of narrative structure and generation of narratives. Narrative generation is known to be a far more complex problem than narrative understanding [20].

scholarly journals Prediction and Analysis of Extracting Relations using Spacy Model

2020 ◽  
Vol 8 (6) ◽  
pp. 3281-3287
Keyword(s):  
Natural Language ◽  
Information Extraction ◽  
Performance Measures ◽  
Text Summarization ◽  
Language Understanding ◽  
Language Generation ◽  
Automatic Text Summarization ◽  
Structured Information ◽  
Automatic Text ◽  
F Measure

Text is an extremely rich resources of information. Each and every second, minutes, peoples are sending or receiving hundreds of millions of data. There are various tasks involved in NLP are machine learning, information extraction, information retrieval, automatic text summarization, question-answered system, parsing, sentiment analysis, natural language understanding and natural language generation. The information extraction is an important task which is used to find the structured information from unstructured or semi-structured text. The paper presents a methodology for extracting the relations of biomedical entities using spacy. The framework consists of following phases such as data creation, load and converting the data into spacy object, preprocessing, define the pattern and extract the relations. The dataset is downloaded from NCBI database which contains only the sentences. The created model evaluated with performance measures like precision, recall and f-measure. The model achieved 87% of accuracy in retrieving of entities relation.

scholarly journals Story Realization: Expanding Plot Events into Sentences

2020 ◽  
Vol 34 (05) ◽  
pp. 7375-7382
Author(s):  
Prithviraj Ammanabrolu ◽  
Ethan Tien ◽  
Wesley Cheung ◽  
Zhaochen Luo ◽  
William Ma ◽  
...  
Keyword(s):  
Neural Network ◽  
Natural Language ◽  
Human Subjects ◽  
Prior Work ◽  
Generation System ◽  
Language Generation ◽  
Story Generation ◽  
Sequence Of Events ◽  
End To End

Neural network based approaches to automated story plot generation attempt to learn how to generate novel plots from a corpus of natural language plot summaries. Prior work has shown that a semantic abstraction of sentences called events improves neural plot generation and and allows one to decompose the problem into: (1) the generation of a sequence of events (event-to-event) and (2) the transformation of these events into natural language sentences (event-to-sentence). However, typical neural language generation approaches to event-to-sentence can ignore the event details and produce grammatically-correct but semantically-unrelated sentences. We present an ensemble-based model that generates natural language guided by events. We provide results—including a human subjects study—for a full end-to-end automated story generation system showing that our method generates more coherent and plausible stories than baseline approaches 1.

scholarly journals Pre-Trained Transformer-Based Language Models for Sundanese

2021 ◽  
Author(s):  
Wilson Wongso ◽  
Henry Lucky ◽  
Derwin Suhartono
Keyword(s):  
Natural Language ◽  
Text Classification ◽  
Training Data ◽  
Language Models ◽  
Classification Task ◽  
Language Understanding ◽  
Training Corpus ◽  
Low Resource ◽  
Corpus Size ◽  
Fine Tune

Abstract The Sundanese language has over 32 million speakers worldwide, but the language has reaped little to no benefits from the recent advances in natural language understanding. Like other low-resource languages, the only alternative is to fine-tune existing multilingual models. In this paper, we pre-trained three monolingual Transformer-based language models on Sundanese data. When evaluated on a downstream text classification task, we found that most of our monolingual models outperformed larger multilingual models despite the smaller overall pre-training data. In the subsequent analyses, our models benefited strongly from the Sundanese pre-training corpus size and do not exhibit socially biased behavior. We released our models for other researchers and practitioners to use.

Sign in / Sign up

Export Citation Format

Share Document