scholarly journals Learn from Syntax: Improving Pair-wise Aspect and Opinion Terms Extraction with Rich Syntactic Knowledge

2021 ◽  
Author(s):  
Shengqiong Wu ◽  
Hao Fei ◽  
Yafeng Ren ◽  
Donghong Ji ◽  
Jingye Li
Keyword(s):  
State Of The Art ◽  
Boundary Detection ◽  
High Order ◽  
Experimental Results ◽  
Convolutional Network ◽  
Syntactic Knowledge ◽  
Current State ◽  
Syntactic Features ◽  
Benchmark Datasets

In this paper, we propose to enhance the pair-wise aspect and opinion terms extraction (PAOTE) task by incorporating rich syntactic knowledge. We first build a syntax fusion encoder for encoding syntactic features, including a label-aware graph convolutional network (LAGCN) for modeling the dependency edges and labels, as well as the POS tags unifiedly, and a local-attention module encoding POS tags for better term boundary detection. During pairing, we then adopt Biaffine and Triaffine scoring for high-order aspect-opinion term pairing, in the meantime re-harnessing the syntax-enriched representations in LAGCN for syntactic-aware scoring. Experimental results on four benchmark datasets demonstrate that our model outperforms current state-of-the-art baselines, meanwhile yielding explainable predictions with syntactic knowledge.

Multi-Graph Cooperative Learning Towards Distant Supervised Relation Extraction

2021 ◽  
Vol 12 (5) ◽  
pp. 1-21
Author(s):  
Changsen Yuan ◽  
Heyan Huang ◽  
Chong Feng
Keyword(s):  
Cooperative Learning ◽  
State Of The Art ◽  
Relation Extraction ◽  
Sentence Length ◽  
Universal Relation ◽  
Dependency Parsing ◽  
Convolutional Network ◽  
Syntactic Features ◽  
Use Dependency

The Graph Convolutional Network (GCN) is a universal relation extraction method that can predict relations of entity pairs by capturing sentences’ syntactic features. However, existing GCN methods often use dependency parsing to generate graph matrices and learn syntactic features. The quality of the dependency parsing will directly affect the accuracy of the graph matrix and change the whole GCN’s performance. Because of the influence of noisy words and sentence length in the distant supervised dataset, using dependency parsing on sentences causes errors and leads to unreliable information. Therefore, it is difficult to obtain credible graph matrices and relational features for some special sentences. In this article, we present a Multi-Graph Cooperative Learning model (MGCL), which focuses on extracting the reliable syntactic features of relations by different graphs and harnessing them to improve the representations of sentences. We conduct experiments on a widely used real-world dataset, and the experimental results show that our model achieves the state-of-the-art performance of relation extraction.

scholarly journals Random Fourier Features via Fast Surrogate Leverage Weighted Sampling

2020 ◽  
Vol 34 (04) ◽  
pp. 4844-4851
Author(s):  
Fanghui Liu ◽  
Xiaolin Huang ◽  
Yudong Chen ◽  
Jie Yang ◽  
Johan Suykens
Keyword(s):  
State Of The Art ◽  
Sampling Strategy ◽  
Prediction Performance ◽  
Current State ◽  
Kernel Approximation ◽  
Sampling Process ◽  
New Strategy ◽  
The Matrix ◽  
Benchmark Datasets ◽  
Random Fourier Features

In this paper, we propose a fast surrogate leverage weighted sampling strategy to generate refined random Fourier features for kernel approximation. Compared to the current state-of-the-art method that uses the leverage weighted scheme (Li et al. 2019), our new strategy is simpler and more effective. It uses kernel alignment to guide the sampling process and it can avoid the matrix inversion operator when we compute the leverage function. Given n observations and s random features, our strategy can reduce the time complexity for sampling from O(ns2+s3) to O(ns2), while achieving comparable (or even slightly better) prediction performance when applied to kernel ridge regression (KRR). In addition, we provide theoretical guarantees on the generalization performance of our approach, and in particular characterize the number of random features required to achieve statistical guarantees in KRR. Experiments on several benchmark datasets demonstrate that our algorithm achieves comparable prediction performance and takes less time cost when compared to (Li et al. 2019).

scholarly journals MDAN-UNet: Multi-Scale and Dual Attention Enhanced Nested U-Net Architecture for Segmentation of Optical Coherence Tomography Images

Algorithms ◽  
2020 ◽  
Vol 13 (3) ◽  
pp. 60 ◽  
Author(s):  
Wen Liu ◽  
Yankui Sun ◽  
Qingge Ji
Keyword(s):  
Optical Coherence Tomography ◽  
State Of The Art ◽  
Imaging Technique ◽  
Optical Coherence ◽  
Convolutional Network ◽  
Multi Scale ◽  
Segmentation Methods ◽  
Benchmark Datasets ◽  
Resolution Imaging ◽  
Layer Segmentation

Optical coherence tomography (OCT) is an optical high-resolution imaging technique for ophthalmic diagnosis. In this paper, we take advantages of multi-scale input, multi-scale side output and dual attention mechanism and present an enhanced nested U-Net architecture (MDAN-UNet), a new powerful fully convolutional network for automatic end-to-end segmentation of OCT images. We have evaluated two versions of MDAN-UNet (MDAN-UNet-16 and MDAN-UNet-32) on two publicly available benchmark datasets which are the Duke Diabetic Macular Edema (DME) dataset and the RETOUCH dataset, in comparison with other state-of-the-art segmentation methods. Our experiment demonstrates that MDAN-UNet-32 achieved the best performance, followed by MDAN-UNet-16 with smaller parameter, for multi-layer segmentation and multi-fluid segmentation respectively.

AUTOMATIC LOOP TRANSFORMATIONS AND PARALLELIZATION FOR JAVA

2000 ◽  
Vol 10 (02n03) ◽  
pp. 153-164 ◽  
Author(s):  
PEDRO V. ARTIGAS ◽  
MANISH GUPTA ◽  
SAMUEL P. MIKIFF ◽  
JOSÉ E. MOREIRA
Keyword(s):  
State Of The Art ◽  
Automatic Parallelization ◽  
High Order ◽  
Performance Levels ◽  
Loop Transformations ◽  
Current State ◽  
Fortran Code

This paper describes a prototype Java compiler that achieves performance levels approaching those of current state-of-the-art Fortran compilers on numerical codes. We present a new transformation called alias versioning that takes advantage of the simplicity of pointers in Java. This transformation, combined with other techniques that we have developed, enables the compiler to perform high order loop transformations and parallelization completely automatically. We believe that our compiler is the first to have such capabilities of optimizing numerical Java codes. By exploiting synergies between our compiler and the Array package for Java, we achieve between 80 and 100% of the performance of highly optimized Fortran code in a variety of benchmarks. Furthermore, automatic parallelization achieves speedups of up to 3.8 on four processors. Our compiler technology makes Java a serious contender for implementing numerical applications.

scholarly journals Bilateral Multi-Perspective Matching for Natural Language Sentences

2017 ◽  
Author(s):  
Zhiguo Wang ◽  
Wael Hamza ◽  
Radu Florian
Keyword(s):  
Natural Language ◽  
State Of The Art ◽  
The State ◽  
Experimental Results ◽  
The Other ◽  
Multiple Perspectives ◽  
Time Step ◽  
Benchmark Datasets ◽  
Sentence Matching ◽  
Fully Connected

Natural language sentence matching is a fundamental technology for a variety of tasks. Previous approaches either match sentences from a single direction or only apply single granular (word-by-word or sentence-by-sentence) matching. In this work, we propose a bilateral multi-perspective matching (BiMPM) model. Given two sentences P and Q, our model first encodes them with a BiLSTM encoder. Next, we match the two encoded sentences in two directions P against Q and P against Q. In each matching direction, each time step of one sentence is matched against all time-steps of the other sentence from multiple perspectives. Then, another BiLSTM layer is utilized to aggregate the matching results into a fix-length matching vector. Finally, based on the matching vector, a decision is made through a fully connected layer. We evaluate our model on three tasks: paraphrase identification, natural language inference and answer sentence selection. Experimental results on standard benchmark datasets show that our model achieves the state-of-the-art performance on all tasks.

scholarly journals Document-level Relation Extraction as Semantic Segmentation

2021 ◽  
Author(s):  
Ningyu Zhang ◽  
Xiang Chen ◽  
Xin Xie ◽  
Shumin Deng ◽  
Chuanqi Tan ◽  
...  
Keyword(s):  
Computer Vision ◽  
State Of The Art ◽  
Relation Extraction ◽  
Semantic Segmentation ◽  
Experimental Results ◽  
Context Information ◽  
Global Information ◽  
Benchmark Datasets ◽  
Segmentation Task ◽  
Document Level

Document-level relation extraction aims to extract relations among multiple entity pairs from a document. Previously proposed graph-based or transformer-based models utilize the entities independently, regardless of global information among relational triples. This paper approaches the problem by predicting an entity-level relation matrix to capture local and global information, parallel to the semantic segmentation task in computer vision. Herein, we propose a Document U-shaped Network for document-level relation extraction. Specifically, we leverage an encoder module to capture the context information of entities and a U-shaped segmentation module over the image-style feature map to capture global interdependency among triples. Experimental results show that our approach can obtain state-of-the-art performance on three benchmark datasets DocRED, CDR, and GDA.

scholarly journals LCF: A Local Context Focus Mechanism for Aspect-Based Sentiment Classification

2019 ◽  
Vol 9 (16) ◽  
pp. 3389 ◽  
Author(s):  
Biqing Zeng ◽  
Heng Yang ◽  
Ruyang Xu ◽  
Wu Zhou ◽  
Xuli Han
Keyword(s):  
State Of The Art ◽  
Sentiment Classification ◽  
Experimental Results ◽  
Local Context ◽  
Baseline Model ◽  
Global Context ◽  
Benchmark Datasets ◽  
Art Performance ◽  
Context Features

Aspect-based sentiment classification (ABSC) aims to predict sentiment polarities of different aspects within sentences or documents. Many previous studies have been conducted to solve this problem, but previous works fail to notice the correlation between the aspect’s sentiment polarity and the local context. In this paper, a Local Context Focus (LCF) mechanism is proposed for aspect-based sentiment classification based on Multi-head Self-Attention (MHSA). This mechanism is called LCF design, and utilizes the Context features Dynamic Mask (CDM) and Context Features Dynamic Weighted (CDW) layers to pay more attention to the local context words. Moreover, a BERT-shared layer is adopted to LCF design to capture internal long-term dependencies of local context and global context. Experiments are conducted on three common ABSC datasets: the laptop and restaurant datasets of SemEval-2014 and the ACL twitter dataset. Experimental results demonstrate that the LCF baseline model achieves considerable performance. In addition, we conduct ablation experiments to prove the significance and effectiveness of LCF design. Especially, by incorporating with BERT-shared layer, the LCF-BERT model refreshes state-of-the-art performance on all three benchmark datasets.

scholarly journals An External Knowledge Enhanced Graph-based Neural Network for Sentence Ordering

2021 ◽  
Vol 70 ◽  
pp. 545-566
Author(s):  
Yongjing Yin ◽  
Shaopeng Lai ◽  
Linfeng Song ◽  
Chulun Zhou ◽  
Xianpei Han ◽  
...  
Keyword(s):  
Neural Network ◽  
State Of The Art ◽  
Recurrent Network ◽  
Experimental Results ◽  
Graph Representation ◽  
External Knowledge ◽  
Text Coherence ◽  
Depth Analysis ◽  
Benchmark Datasets ◽  
Modeling Task

As an important text coherence modeling task, sentence ordering aims to coherently organize a given set of unordered sentences. To achieve this goal, the most important step is to effectively capture and exploit global dependencies among these sentences. In this paper, we propose a novel and flexible external knowledge enhanced graph-based neural network for sentence ordering. Specifically, we first represent the input sentences as a graph, where various kinds of relations (i.e., entity-entity, sentence-sentence and entity-sentence) are exploited to make the graph representation more expressive and less noisy. Then, we introduce graph recurrent network to learn semantic representations of the sentences. To demonstrate the effectiveness of our model, we conduct experiments on several benchmark datasets. The experimental results and in-depth analysis show our model significantly outperforms the existing state-of-the-art models.

scholarly journals Learning semantic sentence representations from visually grounded language without lexical knowledge

2019 ◽  
Vol 25 (4) ◽  
pp. 451-466 ◽  
Author(s):  
Danny Merkx ◽  
Stefan L. Frank
Keyword(s):  
Visual Information ◽  
State Of The Art ◽  
Approaches To Learning ◽  
Semantic Content ◽  
Lexical Knowledge ◽  
Multimodal Data ◽  
Word Level ◽  
Current State ◽  
Sentence Level ◽  
Benchmark Datasets

AbstractCurrent approaches to learning semantic representations of sentences often use prior word-level knowledge. The current study aims to leverage visual information in order to capture sentence level semantics without the need for word embeddings. We use a multimodal sentence encoder trained on a corpus of images with matching text captions to produce visually grounded sentence embeddings. Deep Neural Networks are trained to map the two modalities to a common embedding space such that for an image the corresponding caption can be retrieved and vice versa. We show that our model achieves results comparable to the current state of the art on two popular image-caption retrieval benchmark datasets: Microsoft Common Objects in Context (MSCOCO) and Flickr8k. We evaluate the semantic content of the resulting sentence embeddings using the data from the Semantic Textual Similarity (STS) benchmark task and show that the multimodal embeddings correlate well with human semantic similarity judgements. The system achieves state-of-the-art results on several of these benchmarks, which shows that a system trained solely on multimodal data, without assuming any word representations, is able to capture sentence level semantics. Importantly, this result shows that we do not need prior knowledge of lexical level semantics in order to model sentence level semantics. These findings demonstrate the importance of visual information in semantics.

scholarly journals The FaceChannel: A Fast and Furious Deep Neural Network for Facial Expression Recognition

2020 ◽  
Vol 1 (6) ◽  
Author(s):  
Pablo Barros ◽  
Nikhil Churamani ◽  
Alessandra Sciutti
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Facial Expression ◽  
Facial Expression Recognition ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Facial Features ◽  
Expression Recognition ◽  
Current State ◽  
Benchmark Datasets

AbstractCurrent state-of-the-art models for automatic facial expression recognition (FER) are based on very deep neural networks that are effective but rather expensive to train. Given the dynamic conditions of FER, this characteristic hinders such models of been used as a general affect recognition. In this paper, we address this problem by formalizing the FaceChannel, a light-weight neural network that has much fewer parameters than common deep neural networks. We introduce an inhibitory layer that helps to shape the learning of facial features in the last layer of the network and, thus, improving performance while reducing the number of trainable parameters. To evaluate our model, we perform a series of experiments on different benchmark datasets and demonstrate how the FaceChannel achieves a comparable, if not better, performance to the current state-of-the-art in FER. Our experiments include cross-dataset analysis, to estimate how our model behaves on different affective recognition conditions. We conclude our paper with an analysis of how FaceChannel learns and adapts the learned facial features towards the different datasets.

Sign in / Sign up

Export Citation Format

Share Document