Neural Unsupervised Semantic Role Labeling

The task of semantic role labeling ( SRL ) is dedicated to finding the predicate-argument structure. Previous works on SRL are mostly supervised and do not consider the difficulty in labeling each example which can be very expensive and time-consuming. In this article, we present the first neural unsupervised model for SRL. To decompose the task as two argument related subtasks, identification and clustering, we propose a pipeline that correspondingly consists of two neural modules. First, we train a neural model on two syntax-aware statistically developed rules. The neural model gets the relevance signal for each token in a sentence, to feed into a BiLSTM, and then an adversarial layer for noise-adding and classifying simultaneously, thus enabling the model to learn the semantic structure of a sentence. Then we propose another neural model for argument role clustering, which is done through clustering the learned argument embeddings biased toward their dependency relations. Experiments on the CoNLL-2009 English dataset demonstrate that our model outperforms the previous state-of-the-art baseline in terms of non-neural models for argument identification and classification.

Syntax Role for Neural Semantic Role Labeling

Semantic role labeling (SRL) is dedicated to recognizing the semantic predicate-argument structure of a sentence. Previous studies in terms of traditional models have shown syntactic information can make remarkable contributions to SRL performance; however, the necessity of syntactic information was challenged by a few recent neural SRL studies that demonstrate impressive performance without syntactic backbones and suggest that syntax information becomes much less important for neural semantic role labeling, especially when paired with recent deep neural network and large-scale pre-trained language models. Despite this notion, the neural SRL field still lacks a systematic and full investigation on the relevance of syntactic information in SRL, for both dependency and both monolingual and multilingual settings. This paper intends to quantify the importance of syntactic information for neural SRL in the deep learning framework. We introduce three typical SRL frameworks (baselines), sequence-based, tree-based, and graph-based, which are accompanied by two categories of exploiting syntactic information: syntax pruningbased and syntax feature-based. Experiments are conducted on the CoNLL-2005, 2009, and 2012 benchmarks for all languages available, and results show that neural SRL models can still benefit from syntactic information under certain conditions. Furthermore, we show the quantitative significance of syntax to neural SRL models together with a thorough empirical survey using existing models.

Predicate-Argument Structure in a Valence Dictionary (on the Example of the Verb Reward)

Valence dictionaries are very often specialized works for advanced readers which present how particular linguistic units combine with its subordinates. The article is a critical analysis of a dictionary entry for the lexical unit of reward contained in A Valency Dictionary of English, a Corpus-Based Analysis of the Complementation Patterns of English Verbs, Nouns and Adjectives [2004]. A complementary proposal regarding the predicate-argument structure and its annotation system will be provided based on the theoretical model proposed by S. Karolak [1984; 2002] called Semantic Syntax (SS) and more specifically its extended model called explicative syntax [Kiklewicz et al. 2010; 2019]. The research findings demonstrate the need for coordinated international projects that should integrate both the syntactic as well as the semantic levels in order to gradually meet the objective of an integrated language description encompassing both the grammar and the lexicon.

A General Framework for Domain-Specialization of Stance Detection

We present a generalized framework for domain-specialized stance detection, focusing on Covid-19 as a use case. We define a stance as a predicate-argument structure (combination of an action and its participants) in a simplified one-argument format, e.g., wear(a mask), coupled with a task-specific belief category representing the purpose (e.g., protection) of an argument (e.g., mask) in the context of its predicate (e.g., wear), as constrained by the domain (e.g., Covid-19). A belief category PROTECT captures a belief such as “masks provide protection,” whereas RESTRICT captures a belief such as “mask mandates limit freedom.” A stance combines a belief proposition, e.g., PROTECT(wear(a mask)), with a sentiment toward this proposition. From this, an overall positive attitude toward mask wearing is extracted. The notions purpose and function serve as natural constraints on the choice of belief categories during resource building which, in turn, constrains stance detection. We demonstrate that linguistic constraints (e.g., light verb processing) further refine the choice of predicate-argument pairings for belief and sentiment assignments, yielding significant increases in F1 score for stance detection over a strong baseline.

A deep-learning model for semantic role labelling in medical documents

We built a model labelling the Predicate Argument Structure (PAS) for biomedical documents. PAS is an important semantic information of any document, because it reveals the main event mentioned in each sentence. Extracting PAS in a sentence is an important premise for the computer to solve a series of other problems related to the semantics in text such as event extraction, named entity extraction, question answering system… The predicate argument structure is domain dependent. Therefore, in Biomedical field, it is required to define a completely new Predicate Argument frame compared to the general field. For a machine learning model to work well with a new argument frame, identifying a new feature set is required. This is difficult, manual and requires a lot of expert labor. To address this challenge, we chose to train our model with Deep Learning method utilizing Bi-directional Long Short Term Memory. Deep learning is a machine learning method that does not require defining the feature sets manually. In addition, we also integrate Highway Connection between hidden neuron layers to minimize derivative loss. Besides, to overcome the problem of small training corpus, we integrate Deep Learning with Multi-task Learning technique. Multi-task Learning helps the main task (PAS tagging) to be complemented with knowledge learnt from a closely related task, the NER. Our model achieved F1 = 75.13% without any manually designed feature, thereby showing the prospect of Deep Learning in this domain. In addition, the experiment results also show that Multi-task Learning is an appropriate technique to overcome the problem of little training data in biomedical fields, by improving the F1 score.

Composite Abstract Lexical Structure in Interlanguage Production

Most previous studies of difficulties in learning a second/foreign language focused on sources of learner errors caused by cross-linguistic differences in various levels of linguistic structure, but most of such studies remain at a rather superficial level of description. This study explores sources of learning difficulties at an abstract level by studying the nature and activity of the bilingual mental lexicon during interlanguage production. The bilingual mental lexicon is defined as the mental lexicon containing abstract entries called cross-linguistic “lemmas” underlying particular lexeme. This study claims that it is language-specific lemma which drives interlanguage production at three levels of abstract lexical structure: lexical-conceptual structure, predicate-argument structure, and morphological realization patterns. It further claims that it is cross-linguistic lemma variations in abstract lexical-conceptual structure which result in not only inappropriate lexical choices but also errors in interlanguage production of target language predicate-argument structure and morphological realization. Naturally occurring interlanguage production date for the study include several native and target language pairs: Japanese-English, Chinese-English, and English-Japanese. Some typical instances of language transfer involving other language pairs are also cited in support of the argument that the lexical-conceptual approach to interlanguage production is fundamental in any study of the nature of learner errors in interlanguage development.

Interlanguage Driven by Bilingual Abstract Lexical Structure

This study explores the nature of interlanguage (IL) in terms of bilingual abstract lexical structure and its role in the formulation and development of IL as learners’ developing linguistic system. Adopting the Bilingual Lemma Activation Model (BLAM) (Wei, 2002, 2003), it assumes that IL is a composite developing linguistic system because at different times different linguistic systems are in contact, such as learners’ first language (L1), the developing IL, and the target language (TL), and each contributes different amounts to the developing system of IL. The important claim of this study is that the mental lexicon contains abstract entries, called “lemmas”, which contain pieces of information about particular lexemes, and the bilingual mental lexicon contains language-specific lemmas, which are in contact in IL speech production. The other important claim of this study is that IL is fundamentally driven by bilingual abstract lexical structure, which contains several discrete but interacting subsystems: lexical-conceptual structure, predicate-argument structure, and morphological realization patterns, and such an abstract lexical structure in IL may have different sources, such as those from learners’ L1 and/or the TL. The typical instances of learner errors discussed in this study offer some evidence that IL is a composite developing linguistic system.

Infusing Finetuning with Semantic Dependencies

For natural language processing systems, two kinds of evidence support the use of text representations from neural language models “pretrained” on large unannotated corpora: performance on application-inspired benchmarks (Peters et al., 2018, inter alia), and the emergence of syntactic abstractions in those representations (Tenney et al., 2019, inter alia). On the other hand, the lack of grounded supervision calls into question how well these representations can ever capture meaning (Bender and Koller, 2020). We apply novel probes to recent language models— specifically focusing on predicate-argument structure as operationalized by semantic dependencies (Ivanova et al., 2012)—and find that, unlike syntax, semantics is not brought to the surface by today’s pretrained models. We then use convolutional graph encoders to explicitly incorporate semantic parses into task-specific finetuning, yielding benefits to natural language understanding (NLU) tasks in the GLUE benchmark. This approach demonstrates the potential for general-purpose (rather than task-specific) linguistic supervision, above and beyond conventional pretraining and finetuning. Several diagnostics help to localize the benefits of our approach.1

Semantičko-sintaktičke osobine perifernih glagola osjetilne percepcije u hrvatskom jeziku

The subject of the paper is the semantic and syntactic description of the peripheral verbs of sensory perception in the Croatian language. Peripherality refers to the occurrence of these verbs with different semantic features, which makes some of them function as hyperonyms of the verbs of sensory perception, some may refer to all types of perception, but also create connections with other semantic fields. The analysis is based on componential analysis and cognitive linguistics, and the predicate-argument structure of those verbs is also presented. The aim of the paper is to show that these verbs have a wider range of meanings and thus greater possibilities in usage than dictionaries suggest within lexicographic description.