scholarly journals Neural Unsupervised Semantic Role Labeling

2021 ◽  
Vol 20 (6) ◽  
pp. 1-16
Author(s):  
Kashif Munir ◽  
Hai Zhao ◽  
Zuchao Li
Keyword(s):  
Argument Structure ◽  
State Of The Art ◽  
Neural Model ◽  
Semantic Structure ◽  
Semantic Role ◽  
Semantic Role Labeling ◽  
Neural Models ◽  
Previous State ◽  
Dependency Relations ◽  
Predicate Argument Structure

The task of semantic role labeling ( SRL ) is dedicated to finding the predicate-argument structure. Previous works on SRL are mostly supervised and do not consider the difficulty in labeling each example which can be very expensive and time-consuming. In this article, we present the first neural unsupervised model for SRL. To decompose the task as two argument related subtasks, identification and clustering, we propose a pipeline that correspondingly consists of two neural modules. First, we train a neural model on two syntax-aware statistically developed rules. The neural model gets the relevance signal for each token in a sentence, to feed into a BiLSTM, and then an adversarial layer for noise-adding and classifying simultaneously, thus enabling the model to learn the semantic structure of a sentence. Then we propose another neural model for argument role clustering, which is done through clustering the learned argument embeddings biased toward their dependency relations. Experiments on the CoNLL-2009 English dataset demonstrate that our model outperforms the previous state-of-the-art baseline in terms of non-neural models for argument identification and classification.

scholarly journals Labeling Chinese Predicates with Semantic Roles

2008 ◽  
Vol 34 (2) ◽  
pp. 225-255 ◽  
Author(s):  
Nianwen Xue
Keyword(s):  
Gold Standard ◽  
Argument Structure ◽  
High Performance ◽  
State Of The Art ◽  
Semantic Role ◽  
Semantic Role Labeling ◽  
Pos Tagging ◽  
Fully Automatic ◽  
Syntactic Annotation ◽  
Predicate Argument Structure

In this article we report work on Chinese semantic role labeling, taking advantage of two recently completed corpora, the Chinese PropBank, a semantically annotated corpus of Chinese verbs, and the Chinese Nombank, a companion corpus that annotates the predicate-argument structure of nominalized predicates. Because the semantic role labels are assigned to the constituents in a parse tree, we first report experiments in which semantic role labels are automatically assigned to hand-crafted parses in the Chinese Treebank. This gives us a measure of the extent to which semantic role labels can be bootstrapped from the syntactic annotation provided in the treebank. We then report experiments using automatic parses with decreasing levels of human annotation in the input to the syntactic parser: parses that use gold-standard segmentation and POS-tagging, parses that use only gold-standard segmentation, and fully automatic parses. These experiments gauge how successful semantic role labeling for Chinese can be in more realistic situations. Our results show that when hand-crafted parses are used, semantic role labeling accuracy for Chinese is comparable to what has been reported for the state-of-the-art English semantic role labeling systems trained and tested on the English PropBank, even though the Chinese PropBank is significantly smaller in size. When an automatic parser is used, however, the accuracy of our system is significantly lower than the English state of the art. This indicates that an improvement in Chinese parsing is critical to high-performance semantic role labeling for Chinese.

scholarly journals Syntax Role for Neural Semantic Role Labeling

2021 ◽  
pp. 1-48
Author(s):  
Zuchao Li ◽  
Hai Zhao ◽  
Shexia He ◽  
Jiaxun Cai
Keyword(s):  
Argument Structure ◽  
Large Scale ◽  
Language Models ◽  
Semantic Role ◽  
Semantic Role Labeling ◽  
Empirical Survey ◽  
Learning Framework ◽  
Syntactic Information ◽  
Feature Based ◽  
Predicate Argument Structure

Abstract Semantic role labeling (SRL) is dedicated to recognizing the semantic predicate-argument structure of a sentence. Previous studies in terms of traditional models have shown syntactic information can make remarkable contributions to SRL performance; however, the necessity of syntactic information was challenged by a few recent neural SRL studies that demonstrate impressive performance without syntactic backbones and suggest that syntax information becomes much less important for neural semantic role labeling, especially when paired with recent deep neural network and large-scale pre-trained language models. Despite this notion, the neural SRL field still lacks a systematic and full investigation on the relevance of syntactic information in SRL, for both dependency and both monolingual and multilingual settings. This paper intends to quantify the importance of syntactic information for neural SRL in the deep learning framework. We introduce three typical SRL frameworks (baselines), sequence-based, tree-based, and graph-based, which are accompanied by two categories of exploiting syntactic information: syntax pruningbased and syntax feature-based. Experiments are conducted on the CoNLL-2005, 2009, and 2012 benchmarks for all languages available, and results show that neural SRL models can still benefit from syntactic information under certain conditions. Furthermore, we show the quantitative significance of syntax to neural SRL models together with a thorough empirical survey using existing models.

scholarly journals Selectively Connected Self-Attentions for Semantic Role Labeling

2019 ◽  
Vol 9 (8) ◽  
pp. 1716
Author(s):  
Jaehui Park
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Hierarchical Structures ◽  
Large Data ◽  
Neural Model ◽  
Semantic Role ◽  
Semantic Role Labeling ◽  
Data Set ◽  
Training Time

Semantic role labeling is an effective approach to understand underlying meanings associated with word relationships in natural language sentences. Recent studies using deep neural networks, specifically, recurrent neural networks, have significantly improved traditional shallow models. However, due to the limitation of recurrent updates, they require long training time over a large data set. Moreover, they could not capture the hierarchical structures of languages. We propose a novel deep neural model, providing selective connections among attentive representations, which remove the recurrent updates, for semantic role labeling. Experimental results show that our model performs better in accuracy compared to the state-of-the-art studies. Our model achieves 86.6 F1 scores and 83.6 F1 scores on the CoNLL 2005 and CoNLL 2012 shared tasks, respectively. The accuracy gains are improved by capturing the hierarchical information using the connection module. Moreover, we show that our model can be parallelized to avoid the repetitive updates of the model. As a result, our model reduces the training time by 62 percentages from the baseline.

Thesaurus with Predicate-Argument Structure to Provide Base Framework to Determine States, Actions, and Change-of-States

2016 ◽  
pp. 192-212 ◽  
Author(s):  
Koichi Takeuchi
Keyword(s):  
Argument Structure ◽  
Conceptual Structure ◽  
Semantic Structure ◽  
Semantic Role ◽  
Language Resources ◽  
Narrative Processing ◽  
Lexical Conceptual Structure ◽  
Language Resource ◽  
Predicate Argument Structure ◽  
High Possibility

The goal with this chapter is to discuss the possibility of language resources in determining the states, actions, and change-of-states of characters in narratives. An overview of previous work on linguistic theory and language resources is given then the Predicate-Argument Structure Thesaurus (PT), a Japanese language resource constructed based on the extended framework of the Lexical Conceptual Structure (LCS), is proposed. The PT provides hierarchical clusters of synonyms for 11,900 predicates and 22,000 example sentences annotated with semantic role labels. Each concept has an abstracted LCS, and example sentences are attached to each concept. By virtue of the structure, a correlation of the arguments between other clusters can be determined. The semantic structure of the PT is investigated to enrich generated texts of narratives, and the high possibility of lexical semantics contributing to narrative processing is revealed.

scholarly journals A Goal-Driven Tree-Structured Neural Model for Math Word Problems

2019 ◽  
Author(s):  
Zhipeng Xie ◽  
Shichao Sun
Keyword(s):  
Word Problems ◽  
State Of The Art ◽  
Neural Model ◽  
Neural Models ◽  
Top Down ◽  
Feedforward Networks ◽  
Whole Process ◽  
Expression Tree ◽  
Generate Solution ◽  
Human Problem

Most existing neural models for math word problems exploit Seq2Seq model to generate solution expressions sequentially from left to right, whose results are far from satisfactory due to the lack of goal-driven mechanism commonly seen in human problem solving. This paper proposes a tree-structured neural model to generate expression tree in a goal-driven manner. Given a math word problem, the model first identifies and encodes its goal to achieve, and then the goal gets decomposed into sub-goals combined by an operator in a top-down recursive way. The whole process is repeated until the goal is simple enough to be realized by a known quantity as leaf node. During the process, two-layer gated-feedforward networks are designed to implement each step of goal decomposition, and a recursive neural network is used to encode fulfilled subtrees into subtree embeddings, which provides a better representation of subtrees than the simple goals of subtrees. Experimental results on the dataset Math23K have shown that our tree-structured model outperforms significantly several state-of-the-art models.

scholarly journals Context-aware Frame-Semantic Role Labeling

2015 ◽  
Vol 3 ◽  
pp. 449-460 ◽  
Author(s):  
Michael Roth ◽  
Mirella Lapata
Keyword(s):  
Social Network ◽  
Question Answering ◽  
State Of The Art ◽  
Semantic Role ◽  
Context Aware ◽  
Semantic Role Labeling ◽  
Current State ◽  
Sentence Level ◽  
Small Set ◽  
Labeling System

Frame semantic representations have been useful in several applications ranging from text-to-scene generation, to question answering and social network analysis. Predicting such representations from raw text is, however, a challenging task and corresponding models are typically only trained on a small set of sentence-level annotations. In this paper, we present a semantic role labeling system that takes into account sentence and discourse context. We introduce several new features which we motivate based on linguistic insights and experimentally demonstrate that they lead to significant improvements over the current state-of-the-art in FrameNet-based semantic role labeling.

scholarly journals A Global Joint Model for Semantic Role Labeling

2008 ◽  
Vol 34 (2) ◽  
pp. 161-191 ◽  
Author(s):  
Kristina Toutanova ◽  
Aria Haghighi ◽  
Christopher D. Manning
Keyword(s):  
State Of The Art ◽  
Joint Model ◽  
Semantic Role ◽  
Semantic Role Labeling ◽  
Shared Task ◽  
Data Set ◽  
Joint Structure ◽  
Proposed Model ◽  
Parse Trees ◽  
Linguistic Intuition

We present a model for semantic role labeling that effectively captures the linguistic intuition that a semantic argument frame is a joint structure, with strong dependencies among the arguments. We show how to incorporate these strong dependencies in a statistical joint model with a rich set of features over multiple argument phrases. The proposed model substantially outperforms a similar state-of-the-art local model that does not include dependencies among different arguments. We evaluate the gains from incorporating this joint information on the Propbank corpus, when using correct syntactic parse trees as input, and when using automatically derived parse trees. The gains amount to 24.1% error reduction on all arguments and 36.8% on core arguments for gold-standard parse trees on Propbank. For automatic parse trees, the error reductions are 8.3% and 10.3% on all and core arguments, respectively. We also present results on the CoNLL 2005 shared task data set. Additionally, we explore considering multiple syntactic analyses to cope with parser noise and uncertainty.

A NEW SHALLOW SEMANTIC PARSER FOR DESCRIBING THE CONCEPT STRUCTURE OF TEXT

2009 ◽  
Vol 03 (01) ◽  
pp. 131-149
Author(s):  
YULAN YAN ◽  
YUTAKA MATSUO ◽  
MITSURU ISHIZUKA
Keyword(s):  
Natural Language ◽  
Argument Structure ◽  
Semantic Annotation ◽  
Relation Extraction ◽  
Extraction Process ◽  
Preliminary Evaluation ◽  
Semantic Role Labeling ◽  
Thematic Relations ◽  
Relation Classification ◽  
Predicate Argument Structure

Recently, Semantic Role Labeling (SRL) systems have been used to examine a semantic predicate-argument structure for natural occurring texts. Facing the challenge of extracting a universal set of semantic or thematic relations covering various types of semantic relationships between entities, based on the Concept Description Language for Natural Language (CDL.nl) which defines a set of semantic relations to describe the concept structure of text, we develop a shallow semantic parser to add a new layer of semantic annotation of natural language sentences as an extension of SRL. The parsing task is a relation extraction process with two steps: relation detection and relation classification. Firstly, based on dependency analysis, a rule-based algorithm is presented to detect all entity pairs between each pair for which there exists a relationship; secondly, we use a kernel-based method to assign CDL.nl relations to detected entity pairs by leveraging diverse features. Preliminary evaluation on a manual dataset shows that CDL.nl relations can be extracted with good performance.

Adding semantic roles to the Chinese Treebank

2009 ◽  
Vol 15 (1) ◽  
pp. 143-172 ◽  
Author(s):  
NIANWEN XUE ◽  
MARTHA PALMER
Keyword(s):  
Argument Structure ◽  
Syntactic Structure ◽  
Parse Tree ◽  
Semantic Role ◽  
Lexical Database ◽  
Semantic Roles ◽  
Structure Annotation ◽  
Predicate Argument Structure ◽  
Event Result

AbstractWe report work on adding semantic role labels to the Chinese Treebank, a corpus already annotated with phrase structures. The work involves locating all verbs and their nominalizations in the corpus, and semi-automatically adding semantic role labels to their arguments, which are constituents in a parse tree. Although the same procedure is followed, different issues arise in the annotation of verbs and nominalized predicates. For verbs, identifying their arguments is generally straightforward given their syntactic structure in the Chinese Treebank as they tend to occupy well-defined syntactic positions. Our discussion focuses on the syntactic variations in the realization of the arguments as well as our approach to annotating dislocated and discontinuous arguments. In comparison, identifying the arguments for nominalized predicates is more challenging and we discuss criteria and procedures for distinguishing arguments from non-arguments. In particular we focus on the role of support verbs as well as the relevance of event/result distinctions in the annotation of the predicate-argument structure of nominalized predicates. We also present our approach to taking advantage of the syntactic structure in the Chinese Treebank to bootstrap the predicate-argument structure annotation of verbs. Finally, we discuss the creation of a lexical database of frame files and its role in guiding predicate-argument annotation. Procedures for ensuring annotation consistency and inter-annotator agreement evaluation results are also presented.

Sign in / Sign up

Export Citation Format

Share Document