scholarly journals Multi-Labeled Relation Extraction with Attentive Capsule Network

2019 ◽  
Vol 33 ◽  
pp. 7484-7491 ◽  
Author(s):  
Xinsong Zhang ◽  
Pengshuai Li ◽  
Weijia Jia ◽  
Hai Zhao
Keyword(s):  
Performance Improvement ◽  
Loss Function ◽  
State Of The Art ◽  
Routing Algorithm ◽  
Relation Extraction ◽  
Neural Models ◽  
Novel Approach ◽  
Art Works ◽  
Significant Performance ◽  
Better Than

To disclose overlapped multiple relations from a sentence still keeps challenging. Most current works in terms of neural models inconveniently assuming that each sentence is explicitly mapped to a relation label, cannot handle multiple relations properly as the overlapped features of the relations are either ignored or very difficult to identify. To tackle with the new issue, we propose a novel approach for multi-labeled relation extraction with capsule network which acts considerably better than current convolutional or recurrent net in identifying the highly overlapped relations within an individual sentence. To better cluster the features and precisely extract the relations, we further devise attention-based routing algorithm and sliding-margin loss function, and embed them into our capsule network. The experimental results show that the proposed approach can indeed extract the highly overlapped features and achieve significant performance improvement for relation extraction comparing to the state-of-the-art works.

scholarly journals Effective Deep Memory Networks for Distant Supervised Relation Extraction

2017 ◽  
Author(s):  
Xiaocheng Feng ◽  
Jiang Guo ◽  
Bing Qin ◽  
Ting Liu ◽  
Yongjie Liu
Keyword(s):  
Relation Extraction ◽  
Training Data ◽  
Context Word ◽  
Neural Models ◽  
Attention Model ◽  
Memory Experiment ◽  
Feature Based ◽  
Real World Datasets ◽  
Major Attention ◽  
Better Than

Distant supervised relation extraction (RE) has been an effective way of finding novel relational facts from text without labeled training data. Typically it can be formalized as a multi-instance multi-label problem.In this paper, we introduce a novel neural approach for distant supervised (RE) with specific focus on attention mechanisms.Unlike the feature-based logistic regression model and compositional neural models such as CNN, our approach includes two major attention-based memory components, which is capable of explicitly capturing the importance of each context word for modeling the representation of the entity pair, as well as the intrinsic dependencies between relations.Such importance degree and dependency relationship are calculated with multiple computational layers, each of which is a neural attention model over an external memory. Experiment on real-world datasets shows that our approach performs significantly and consistently better than various baselines.

scholarly journals Learning to Transfer Relational Representations through Analogy

2019 ◽  
Vol 33 ◽  
pp. 10015-10016
Author(s):  
Gaetano Rossiello ◽  
Alfio Gliozzo ◽  
Michael Glass
Keyword(s):  
State Of The Art ◽  
Relation Extraction ◽  
Knowledge Bases ◽  
The State ◽  
Large Set ◽  
Relational Information ◽  
Siamese Network ◽  
Distant Supervision ◽  
Novel Approach ◽  
Art Methods

We propose a novel approach to learn representations of relations expressed by their textual mentions. In our assumption, if two pairs of entities belong to the same relation, then those two pairs are analogous. We collect a large set of analogous pairs by matching triples in knowledge bases with web-scale corpora through distant supervision. This dataset is adopted to train a hierarchical siamese network in order to learn entity-entity embeddings which encode relational information through the different linguistic paraphrasing expressing the same relation. The model can be used to generate pre-trained embeddings which provide a valuable signal when integrated into an existing neural-based model by outperforming the state-of-the-art methods on a relation extraction task.

scholarly journals Video Person Re-ID: Fantastic Techniques and Where to Find Them (Student Abstract)

2020 ◽  
Vol 34 (10) ◽  
pp. 13893-13894
Author(s):  
Priyank Pathak ◽  
Amir Erfan Eshratifar ◽  
Michael Gormish
Keyword(s):  
Neural Network ◽  
Loss Function ◽  
State Of The Art ◽  
Source Code ◽  
Status Quo ◽  
Current Status ◽  
Neural Models ◽  
Academic Interest ◽  
Multiple Camera ◽  
The Common

The ability to identify the same person from multiple camera views without the explicit use of facial recognition is receiving commercial and academic interest. The current status-quo solutions are based on attention neural models. In this paper, we propose Attention and CL loss, which is a hybrid of center and Online Soft Mining (OSM) loss added to the attention loss on top of a temporal attention-based neural network. The proposed loss function applied with bag-of-tricks for training surpasses the state of the art on the common person Re-ID datasets, MARS and PRID 2011. Our source code is publicly available on github1.

scholarly journals Transformer-Capsule Model for Intent Detection (Student Abstract)

2020 ◽  
Vol 34 (10) ◽  
pp. 13885-13886
Author(s):  
Aleksander Obuchowski ◽  
Michał Lew
Keyword(s):  
State Of The Art ◽  
Intent Recognition ◽  
Novel Approach ◽  
Better Than

Intent recognition is one of the most crucial tasks in NLU systems, which are nowadays especially important for designing intelligent conversation. We propose a novel approach to intent recognition which involves combining transformer architecture with capsule networks. Our results show that such architecture performs better than original capsule-NLU network implementations and achieves state-of-the-art results on datasets such as ATIS, AskUbuntu ,and WebApp.

scholarly journals Classifier for Activities with Variations

Sensors ◽  
2018 ◽  
Vol 18 (10) ◽  
pp. 3529 ◽  
Author(s):  
Rabih Younes ◽  
Mark Jones ◽  
Thomas Martin
Keyword(s):  
Activity Recognition ◽  
State Of The Art ◽  
The State ◽  
Daily Activities ◽  
Controlled Environment ◽  
Novel Approach ◽  
Specific Activities ◽  
Application Specific ◽  
Complex Activities ◽  
Better Than

Most activity classifiers focus on recognizing application-specific activities that are mostly performed in a scripted manner, where there is very little room for variation within the activity. These classifiers are mainly good at recognizing short scripted activities that are performed in a specific way. In reality, especially when considering daily activities, humans perform complex activities in a variety of ways. In this work, we aim to make activity recognition more practical by proposing a novel approach to recognize complex heterogeneous activities that could be performed in a wide variety of ways. We collect data from 15 subjects performing eight complex activities and test our approach while analyzing it from different aspects. The results show the validity of our approach. They also show how it performs better than the state-of-the-art approaches that tried to recognize the same activities in a more controlled environment.

scholarly journals PSCNN: PatchShuffle Convolutional Neural Network for COVID-19 Explainable Diagnosis

2021 ◽  
Vol 9 ◽  
Author(s):  
Shui-Hua Wang ◽  
Ziquan Zhu ◽  
Yu-Dong Zhang
Keyword(s):  
Neural Network ◽  
Infectious Disease ◽  
Convolutional Neural Network ◽  
Loss Function ◽  
Data Augmentation ◽  
State Of The Art ◽  
Diagnosis System ◽  
Backbone Network ◽  
The Mean ◽  
Better Than

Objective: COVID-19 is a sort of infectious disease caused by a new strain of coronavirus. This study aims to develop a more accurate COVID-19 diagnosis system.Methods: First, the n-conv module (nCM) is introduced. Then we built a 12-layer convolutional neural network (12l-CNN) as the backbone network. Afterwards, PatchShuffle was introduced to integrate with 12l-CNN as a regularization term of the loss function. Our model was named PSCNN. Moreover, multiple-way data augmentation and Grad-CAM are employed to avoid overfitting and locating lung lesions.Results: The mean and standard variation values of the seven measures of our model were 95.28 ± 1.03 (sensitivity), 95.78 ± 0.87 (specificity), 95.76 ± 0.86 (precision), 95.53 ± 0.83 (accuracy), 95.52 ± 0.83 (F1 score), 91.7 ± 1.65 (MCC), and 95.52 ± 0.83 (FMI).Conclusion: Our PSCNN is better than 10 state-of-the-art models. Further, we validate the optimal hyperparameters in our model and demonstrate the effectiveness of PatchShuffle.

scholarly journals Distant Supervision for Relation Extraction with Sentence Selection and Interaction Representation

2021 ◽  
Vol 2021 ◽  
pp. 1-16
Author(s):  
Tiantian Chen ◽  
Nianbin Wang ◽  
Hongbin Wang ◽  
Haomin Zhan
Keyword(s):  
Large Scale ◽  
Semantic Information ◽  
State Of The Art ◽  
Relation Extraction ◽  
Semantic Features ◽  
Distant Supervision ◽  
Word Level ◽  
Proposed Model ◽  
Relation Prediction ◽  
Better Than

Distant supervision (DS) has been widely used for relation extraction (RE), which automatically generates large-scale labeled data. However, there is a wrong labeling problem, which affects the performance of RE. Besides, the existing method suffers from the lack of useful semantic features for some positive training instances. To address the above problems, we propose a novel RE model with sentence selection and interaction representation for distantly supervised RE. First, we propose a pattern method based on the relation trigger words as a sentence selector to filter out noisy sentences to alleviate the wrong labeling problem. After clean instances are obtained, we propose the interaction representation using the word-level attention mechanism-based entity pairs to dynamically increase the weights of the words related to entity pairs, which can provide more useful semantic information for relation prediction. The proposed model outperforms the strongest baseline by 2.61 in F1-score on a widely used dataset, which proves that our model performs significantly better than the state-of-the-art RE systems.

scholarly journals Distractor-Aware Deep Regression for Visual Tracking

Sensors ◽  
2019 ◽  
Vol 19 (2) ◽  
pp. 387 ◽  
Author(s):  
Ming Du ◽  
Yan Ding ◽  
Xiuyun Meng ◽  
Hua-Liang Wei ◽  
Yifan Zhao
Keyword(s):  
Object Tracking ◽  
Visual Tracking ◽  
Test Data ◽  
Loss Function ◽  
State Of The Art ◽  
Target Object ◽  
Visual Object ◽  
Visual Object Tracking ◽  
Training Samples ◽  
Better Than

In recent years, regression trackers have drawn increasing attention in the visual-object tracking community due to their favorable performance and easy implementation. The tracker algorithms directly learn mapping from dense samples around the target object to Gaussian-like soft labels. However, in many real applications, when applied to test data, the extreme imbalanced distribution of training samples usually hinders the robustness and accuracy of regression trackers. In this paper, we propose a novel effective distractor-aware loss function to balance this issue by highlighting the significant domain and by severely penalizing the pure background. In addition, we introduce a full differentiable hierarchy-normalized concatenation connection to exploit abstractions across multiple convolutional layers. Extensive experiments were conducted on five challenging benchmark-tracking datasets, that is, OTB-13, OTB-15, TC-128, UAV-123, and VOT17. The experimental results are promising and show that the proposed tracker performs much better than nearly all the compared state-of-the-art approaches.

scholarly journals From Zero-Shot Learning to Cold-Start Recommendation

2019 ◽  
Vol 33 ◽  
pp. 4189-4196 ◽  
Author(s):  
Jingjing Li ◽  
Mengmeng Jing ◽  
Ke Lu ◽  
Lei Zhu ◽  
Yang Yang ◽  
...  
Keyword(s):  
Computer Vision ◽  
Performance Improvement ◽  
User Behavior ◽  
State Of The Art ◽  
Cold Start ◽  
Feature Representation ◽  
The Other ◽  
Low Rank ◽  
Significant Performance ◽  
First Time

Zero-shot learning (ZSL) and cold-start recommendation (CSR) are two challenging problems in computer vision and recommender system, respectively. In general, they are independently investigated in different communities. This paper, however, reveals that ZSL and CSR are two extensions of the same intension. Both of them, for instance, attempt to predict unseen classes and involve two spaces, one for direct feature representation and the other for supplementary description. Yet there is no existing approach which addresses CSR from the ZSL perspective. This work, for the first time, formulates CSR as a ZSL problem, and a tailor-made ZSL method is proposed to handle CSR. Specifically, we propose a Lowrank Linear Auto-Encoder (LLAE), which challenges three cruxes, i.e., domain shift, spurious correlations and computing efficiency, in this paper. LLAE consists of two parts, a low-rank encoder maps user behavior into user attributes and a symmetric decoder reconstructs user behavior from user attributes. Extensive experiments on both ZSL and CSR tasks verify that the proposed method is a win-win formulation, i.e., not only can CSR be handled by ZSL models with a significant performance improvement compared with several conventional state-of-the-art methods, but the consideration of CSR can benefit ZSL as well.

scholarly journals Teacher-Student Networks with Multiple Decoders for Solving Math Word Problem

2020 ◽  
Author(s):  
Jipeng Zhang ◽  
Roy Ka-Wei Lee ◽  
Ee-Peng Lim ◽  
Wei Qin ◽  
Lei Wang ◽  
...  
Keyword(s):  
Word Problem ◽  
Large Scale ◽  
State Of The Art ◽  
Training Data ◽  
Teacher Student ◽  
Candidate Solution ◽  
Novel Approach ◽  
Art Works ◽  
Standard Solution ◽  
Final Answer

Math word problem (MWP) is challenging due to the limitation in training data where only one “standard” solution is available. MWP models often simply fit this solution rather than truly understand or solve the problem. The generalization of models (to diverse word scenarios) is thus limited. To address this problem, this paper proposes a novel approach, TSN-MD, by leveraging the teacher network to integrate the knowledge of equivalent solution expressions and then to regularize the learning behavior of the student network. In addition, we introduce the multiple-decoder student network to generate multiple candidate solution expressions by which the final answer is voted. In experiments, we conduct extensive comparisons and ablative studies on two large-scale MWP benchmarks, and show that using TSN-MD can surpass the state-of-the-art works by a large margin. More intriguingly, the visualization results demonstrate that TSN-MD not only produces correct final answers but also generates diverse equivalent expressions of the solution.

Sign in / Sign up

Export Citation Format

Share Document