Appendix A: Experimental Evaluation of Sorted Beam Search

Training Deterministic Parsers with Non-Deterministic Oracles

Transactions of the Association for Computational Linguistics ◽

10.1162/tacl_a_00237 ◽

2013 ◽

Vol 1 ◽

pp. 403-414 ◽

Cited By ~ 16

Author(s):

Yoav Goldberg ◽

Joakim Nivre

Keyword(s):

Experimental Evaluation ◽

Error Propagation ◽

Optimal Path ◽

Transition System ◽

Data Sets ◽

Beam Search ◽

Transition Systems ◽

General Characterization ◽

Wide Range

Greedy transition-based parsers are very fast but tend to suffer from error propagation. This problem is aggravated by the fact that they are normally trained using oracles that are deterministic and incomplete in the sense that they assume a unique canonical path through the transition system and are only valid as long as the parser does not stray from this path. In this paper, we give a general characterization of oracles that are nondeterministic and complete, present a method for deriving such oracles for transition systems that satisfy a property we call arc decomposition, and instantiate this method for three well-known transition systems from the literature. We say that these oracles are dynamic, because they allow us to dynamically explore alternative and nonoptimal paths during training — in contrast to oracles that statically assume a unique optimal path. Experimental evaluation on a wide range of data sets clearly shows that using dynamic oracles to train greedy parsers gives substantial improvements in accuracy. Moreover, this improvement comes at no cost in terms of efficiency, unlike other techniques like beam search.

Download Full-text

Diverse Decoding for Abstractive Document Summarization

Applied Sciences ◽

10.3390/app9030386 ◽

2019 ◽

Vol 9 (3) ◽

pp. 386 ◽

Cited By ~ 2

Author(s):

Xu-Wang Han ◽

Hai-Tao Zheng ◽

Jin-Yuan Chen ◽

Cong-Zhi Zhao

Keyword(s):

Experimental Evaluation ◽

State Of The Art ◽

Attention Mechanism ◽

Beam Search ◽

Daily Mail ◽

Document Summarization ◽

Novel Method ◽

Search Approach ◽

Abstractive Summarization ◽

Information Coverage

Recently, neural sequence-to-sequence models have made impressive progress in abstractive document summarization. Unfortunately, as neural abstractive summarization research is in a primitive stage, the performance of these models is still far from ideal. In this paper, we propose a novel method called Neural Abstractive Summarization with Diverse Decoding (NASDD). This method augments the standard attentional sequence-to-sequence model in two aspects. First, we introduce a diversity-promoting beam search approach in the decoding process, which alleviates the serious diversity issue caused by standard beam search and hence increases the possibility of generating summary sequences that are more informative. Second, we creatively utilize the attention mechanism combined with the key information of the input document as an estimation of the salient information coverage, which aids in finding the optimal summary sequence. We carry out the experimental evaluation with state-of-the-art methods on the CNN/Daily Mail summarization dataset, and the results demonstrate the superiority of our proposed method.

Download Full-text