scholarly journals Syntax Role for Neural Semantic Role Labeling

2021 ◽  
pp. 1-48
Author(s):  
Zuchao Li ◽  
Hai Zhao ◽  
Shexia He ◽  
Jiaxun Cai
Keyword(s):  
Argument Structure ◽  
Large Scale ◽  
Language Models ◽  
Semantic Role ◽  
Semantic Role Labeling ◽  
Empirical Survey ◽  
Learning Framework ◽  
Syntactic Information ◽  
Feature Based ◽  
Predicate Argument Structure

Abstract Semantic role labeling (SRL) is dedicated to recognizing the semantic predicate-argument structure of a sentence. Previous studies in terms of traditional models have shown syntactic information can make remarkable contributions to SRL performance; however, the necessity of syntactic information was challenged by a few recent neural SRL studies that demonstrate impressive performance without syntactic backbones and suggest that syntax information becomes much less important for neural semantic role labeling, especially when paired with recent deep neural network and large-scale pre-trained language models. Despite this notion, the neural SRL field still lacks a systematic and full investigation on the relevance of syntactic information in SRL, for both dependency and both monolingual and multilingual settings. This paper intends to quantify the importance of syntactic information for neural SRL in the deep learning framework. We introduce three typical SRL frameworks (baselines), sequence-based, tree-based, and graph-based, which are accompanied by two categories of exploiting syntactic information: syntax pruningbased and syntax feature-based. Experiments are conducted on the CoNLL-2005, 2009, and 2012 benchmarks for all languages available, and results show that neural SRL models can still benefit from syntactic information under certain conditions. Furthermore, we show the quantitative significance of syntax to neural SRL models together with a thorough empirical survey using existing models.

scholarly journals Labeling Chinese Predicates with Semantic Roles

2008 ◽  
Vol 34 (2) ◽  
pp. 225-255 ◽  
Author(s):  
Nianwen Xue
Keyword(s):  
Gold Standard ◽  
Argument Structure ◽  
High Performance ◽  
State Of The Art ◽  
Semantic Role ◽  
Semantic Role Labeling ◽  
Pos Tagging ◽  
Fully Automatic ◽  
Syntactic Annotation ◽  
Predicate Argument Structure

In this article we report work on Chinese semantic role labeling, taking advantage of two recently completed corpora, the Chinese PropBank, a semantically annotated corpus of Chinese verbs, and the Chinese Nombank, a companion corpus that annotates the predicate-argument structure of nominalized predicates. Because the semantic role labels are assigned to the constituents in a parse tree, we first report experiments in which semantic role labels are automatically assigned to hand-crafted parses in the Chinese Treebank. This gives us a measure of the extent to which semantic role labels can be bootstrapped from the syntactic annotation provided in the treebank. We then report experiments using automatic parses with decreasing levels of human annotation in the input to the syntactic parser: parses that use gold-standard segmentation and POS-tagging, parses that use only gold-standard segmentation, and fully automatic parses. These experiments gauge how successful semantic role labeling for Chinese can be in more realistic situations. Our results show that when hand-crafted parses are used, semantic role labeling accuracy for Chinese is comparable to what has been reported for the state-of-the-art English semantic role labeling systems trained and tested on the English PropBank, even though the Chinese PropBank is significantly smaller in size. When an automatic parser is used, however, the accuracy of our system is significantly lower than the English state of the art. This indicates that an improvement in Chinese parsing is critical to high-performance semantic role labeling for Chinese.

scholarly journals Neural Unsupervised Semantic Role Labeling

2021 ◽  
Vol 20 (6) ◽  
pp. 1-16
Author(s):  
Kashif Munir ◽  
Hai Zhao ◽  
Zuchao Li
Keyword(s):  
Argument Structure ◽  
State Of The Art ◽  
Neural Model ◽  
Semantic Structure ◽  
Semantic Role ◽  
Semantic Role Labeling ◽  
Neural Models ◽  
Previous State ◽  
Dependency Relations ◽  
Predicate Argument Structure

The task of semantic role labeling ( SRL ) is dedicated to finding the predicate-argument structure. Previous works on SRL are mostly supervised and do not consider the difficulty in labeling each example which can be very expensive and time-consuming. In this article, we present the first neural unsupervised model for SRL. To decompose the task as two argument related subtasks, identification and clustering, we propose a pipeline that correspondingly consists of two neural modules. First, we train a neural model on two syntax-aware statistically developed rules. The neural model gets the relevance signal for each token in a sentence, to feed into a BiLSTM, and then an adversarial layer for noise-adding and classifying simultaneously, thus enabling the model to learn the semantic structure of a sentence. Then we propose another neural model for argument role clustering, which is done through clustering the learned argument embeddings biased toward their dependency relations. Experiments on the CoNLL-2009 English dataset demonstrate that our model outperforms the previous state-of-the-art baseline in terms of non-neural models for argument identification and classification.

A Typed Feature-based Grammar of Danish

1997 ◽  
Vol 20 (1) ◽  
pp. 31-62
Author(s):  
Nancy L. Underwood
Keyword(s):  
Argument Structure ◽  
Phrase Structure ◽  
Inflectional Morphology ◽  
Feature Structure ◽  
Formal Framework ◽  
Feature Based ◽  
Predicate Argument Structure

This paper presents an overview of the first broad coverage grammatical description of Danish in a Typed Feature Structure (TFS) based unification formalism inspired by HPSG. These linguistic specifications encompass phenomena within inflectional morphology, phrase structure and predicate argument structure, and have been developed with a view to implementation. The emphasis on implementability and re-usability of the specifications has led to the adoption of a rather leaner formal framework than that underlying HPSG. However, the paper shows that the adoption of such a framework does not lead to a loss of expressibility, but in fact enables certain phenomena, such as the interface between morphology and syntax and local discontinuities, to be treated in a simple and elegant fashion.

scholarly journals Syntax-Aware Neural Semantic Role Labeling

2019 ◽  
Vol 33 ◽  
pp. 7305-7313 ◽  
Author(s):  
Qingrong Xia ◽  
Zhenghua Li ◽  
Min Zhang ◽  
Meishan Zhang ◽  
Guohong Fu ◽  
...  
Keyword(s):  
Close Correlation ◽  
Semantic Role ◽  
Semantic Role Labeling ◽  
Semantic Parsing ◽  
Single Model ◽  
Improve Performance ◽  
Syntactic Structures ◽  
Word Sequence ◽  
Input Sentence ◽  
Feature Based

Semantic role labeling (SRL), also known as shallow semantic parsing, is an important yet challenging task in NLP. Motivated by the close correlation between syntactic and semantic structures, traditional discrete-feature-based SRL approaches make heavy use of syntactic features. In contrast, deep-neural-network-based approaches usually encode the input sentence as a word sequence without considering the syntactic structures. In this work, we investigate several previous approaches for encoding syntactic trees, and make a thorough study on whether extra syntax-aware representations are beneficial for neural SRL models. Experiments on the benchmark CoNLL-2005 dataset show that syntax-aware SRL approaches can effectively improve performance over a strong baseline with external word representations from ELMo. With the extra syntax-aware representations, our approaches achieve new state-of-the-art 85.6 F1 (single model) and 86.6 F1 (ensemble) on the test data, outperforming the corresponding strong baselines with ELMo by 0.8 and 1.0, respectively. Detailed error analysis are conducted to gain more insights on the investigated approaches.

A NEW SHALLOW SEMANTIC PARSER FOR DESCRIBING THE CONCEPT STRUCTURE OF TEXT

2009 ◽  
Vol 03 (01) ◽  
pp. 131-149
Author(s):  
YULAN YAN ◽  
YUTAKA MATSUO ◽  
MITSURU ISHIZUKA
Keyword(s):  
Natural Language ◽  
Argument Structure ◽  
Semantic Annotation ◽  
Relation Extraction ◽  
Extraction Process ◽  
Preliminary Evaluation ◽  
Semantic Role Labeling ◽  
Thematic Relations ◽  
Relation Classification ◽  
Predicate Argument Structure

Recently, Semantic Role Labeling (SRL) systems have been used to examine a semantic predicate-argument structure for natural occurring texts. Facing the challenge of extracting a universal set of semantic or thematic relations covering various types of semantic relationships between entities, based on the Concept Description Language for Natural Language (CDL.nl) which defines a set of semantic relations to describe the concept structure of text, we develop a shallow semantic parser to add a new layer of semantic annotation of natural language sentences as an extension of SRL. The parsing task is a relation extraction process with two steps: relation detection and relation classification. Firstly, based on dependency analysis, a rule-based algorithm is presented to detect all entity pairs between each pair for which there exists a relationship; secondly, we use a kernel-based method to assign CDL.nl relations to detected entity pairs by leveraging diverse features. Preliminary evaluation on a manual dataset shows that CDL.nl relations can be extracted with good performance.

Incident detection and classification in renewable energy news using pre-trained language models on deep neural networks

2021 ◽  
pp. 1-20
Author(s):  
Qiqing Wang ◽  
Cunbin Li
Keyword(s):  
Neural Networks ◽  
Renewable Energy ◽  
Large Scale ◽  
Deep Neural Networks ◽  
Energy Systems ◽  
Language Models ◽  
Causal Effects ◽  
Incident Detection ◽  
Renewable Energy Systems ◽  
Learning Framework

The surge of renewable energy systems can lead to increasing incidents that negatively impact economics and society, rendering incident detection paramount to understand the mechanism and range of those impacts. In this paper, a deep learning framework is proposed to detect renewable energy incidents from news articles containing accidents in various renewable energy systems. The pre-trained language models like Bidirectional Encoder Representations from Transformers (BERT) and word2vec are utilized to represent textual inputs, which are trained by the Text Convolutional Neural Networks (TCNNs) and Text Recurrent Neural Networks. Two types of classifiers for incident detection are trained and tested in this paper, one is a binary classifier for detecting the existence of an incident, the other is a multi-label classifier for identifying different incident attributes such as causal-effects and consequences, etc. The proposed incident detection framework is implemented on a hand-annotated dataset with 5 190 records. The results show that the proposed framework performs well on both the incident existence detection task (F1-score 91.4%) and the incident attributes identification task (micro F1-score 81.7%). It is also shown that the BERT-based TCNNs are effective and robust in detecting renewable energy incidents from large-scale textual materials.

scholarly journals Tree Kernels for Semantic Role Labeling

2008 ◽  
Vol 34 (2) ◽  
pp. 193-224 ◽  
Author(s):  
Alessandro Moschitti ◽  
Daniele Pighin ◽  
Roberto Basili
Keyword(s):  
Language Processing ◽  
Large Scale ◽  
Kernel Functions ◽  
Feature Representation ◽  
Training Data ◽  
Support Vector ◽  
Feature Engineering ◽  
Semantic Role ◽  
Learning Approaches ◽  
Semantic Role Labeling

The availability of large scale data sets of manually annotated predicate-argument structures has recently favored the use of machine learning approaches to the design of automated semantic role labeling (SRL) systems. The main research in this area relates to the design choices for feature representation and for effective decompositions of the task in different learning models. Regarding the former choice, structural properties of full syntactic parses are largely employed as they represent ways to encode different principles suggested by the linking theory between syntax and semantics. The latter choice relates to several learning schemes over global views of the parses. For example, re-ranking stages operating over alternative predicate-argument sequences of the same sentence have shown to be very effective. In this article, we propose several kernel functions to model parse tree properties in kernel-based machines, for example, perceptrons or support vector machines. In particular, we define different kinds of tree kernels as general approaches to feature engineering in SRL. Moreover, we extensively experiment with such kernels to investigate their contribution to individual stages of an SRL architecture both in isolation and in combination with other traditional manually coded features. The results for boundary recognition, classification, and re-ranking stages provide systematic evidence about the significant impact of tree kernels on the overall accuracy, especially when the amount of training data is small. As a conclusive result, tree kernels allow for a general and easily portable feature engineering method which is applicable to a large family of natural language processing tasks.

Adding semantic roles to the Chinese Treebank

2009 ◽  
Vol 15 (1) ◽  
pp. 143-172 ◽  
Author(s):  
NIANWEN XUE ◽  
MARTHA PALMER
Keyword(s):  
Argument Structure ◽  
Syntactic Structure ◽  
Parse Tree ◽  
Semantic Role ◽  
Lexical Database ◽  
Semantic Roles ◽  
Structure Annotation ◽  
Predicate Argument Structure ◽  
Event Result

AbstractWe report work on adding semantic role labels to the Chinese Treebank, a corpus already annotated with phrase structures. The work involves locating all verbs and their nominalizations in the corpus, and semi-automatically adding semantic role labels to their arguments, which are constituents in a parse tree. Although the same procedure is followed, different issues arise in the annotation of verbs and nominalized predicates. For verbs, identifying their arguments is generally straightforward given their syntactic structure in the Chinese Treebank as they tend to occupy well-defined syntactic positions. Our discussion focuses on the syntactic variations in the realization of the arguments as well as our approach to annotating dislocated and discontinuous arguments. In comparison, identifying the arguments for nominalized predicates is more challenging and we discuss criteria and procedures for distinguishing arguments from non-arguments. In particular we focus on the role of support verbs as well as the relevance of event/result distinctions in the annotation of the predicate-argument structure of nominalized predicates. We also present our approach to taking advantage of the syntactic structure in the Chinese Treebank to bootstrap the predicate-argument structure annotation of verbs. Finally, we discuss the creation of a lexical database of frame files and its role in guiding predicate-argument annotation. Procedures for ensuring annotation consistency and inter-annotator agreement evaluation results are also presented.

scholarly journals Selectional Preferences for Semantic Role Classification

2013 ◽  
Vol 39 (3) ◽  
pp. 631-663 ◽  
Author(s):  
Beñat Zapirain ◽  
Eneko Agirre ◽  
Lluís Màrquez ◽  
Mihai Surdeanu
Keyword(s):  
Error Analysis ◽  
State Of The Art ◽  
Error Reduction ◽  
Semantic Role ◽  
Semantic Role Labeling ◽  
Syntactic Information ◽  
Open Issue ◽  
Selectional Preferences ◽  
Post Hoc ◽  
Labeling System

This paper focuses on a well-known open issue in Semantic Role Classification (SRC) research: the limited influence and sparseness of lexical features. We mitigate this problem using models that integrate automatically learned selectional preferences (SP). We explore a range of models based on WordNet and distributional-similarity SPs. Furthermore, we demonstrate that the SRC task is better modeled by SP models centered on both verbs and prepositions, rather than verbs alone. Our experiments with SP-based models in isolation indicate that they outperform a lexical baseline with 20 F1 points in domain and almost 40 F1 points out of domain. Furthermore, we show that a state-of-the-art SRC system extended with features based on selectional preferences performs significantly better, both in domain (17% error reduction) and out of domain (13% error reduction). Finally, we show that in an end-to-end semantic role labeling system we obtain small but statistically significant improvements, even though our modified SRC model affects only approximately 4% of the argument candidates. Our post hoc error analysis indicates that the SP-based features help mostly in situations where syntactic information is either incorrect or insufficient to disambiguate the correct role.

Sign in / Sign up

Export Citation Format

Share Document