scholarly journals Differentiated Attentive Representation Learning for Sentence Classification

2018 ◽  
Author(s):  
Qianrong Zhou ◽  
Xiaojie Wang ◽  
Xuan Dong
Keyword(s):  
Representation Learning ◽  
Learning Model ◽  
Attention Mechanism ◽  
Experimental Results ◽  
Sentence Classification ◽  
Synthetic Datasets

Attention-based models have shown to be effective in learning representations for sentence classification. They are typically equipped with multi-hop attention mechanism. However, existing multi-hop models still suffer from the problem of paying much attention to the most frequently noticed words, which might not be important to classify the current sentence. And there is a lack of explicitly effective way that helps the attention to be shifted out of a wrong part in the sentence. In this paper, we alleviate this problem by proposing a differentiated attentive learning model. It is composed of two branches of attention subnets and an example discriminator. An explicit signal with the loss information of the first attention subnet is passed on to the second one to drive them to learn different attentive preference. The example discriminator then selects the suitable attention subnet for sentence classification. Experimental results on real and synthetic datasets demonstrate the effectiveness of our model.

scholarly journals Image-embodied Knowledge Representation Learning

2017 ◽  
Author(s):  
Ruobing Xie ◽  
Zhiyuan Liu ◽  
Huanbo Luan ◽  
Maosong Sun
Keyword(s):  
Knowledge Representation ◽  
Visual Information ◽  
Representation Learning ◽  
Learning Model ◽  
Experimental Results ◽  
Knowledge Graph ◽  
Embodied Knowledge ◽  
Knowledge Representations ◽  
Conventional Methods ◽  
Image Representations

Entity images could provide significant visual information for knowledge representation learning. Most conventional methods learn knowledge representations merely from structured triples, ignoring rich visual information extracted from entity images. In this paper, we propose a novel Image-embodied Knowledge Representation Learning model (IKRL), where knowledge representations are learned with both triple facts and images. More specifically, we first construct representations for all images of an entity with a neural image encoder. These image representations are then integrated into an aggregated image-based representation via an attention-based method. We evaluate our IKRL models on knowledge graph completion and triple classification. Experimental results demonstrate that our models outperform all baselines on both tasks, which indicates the significance of visual information for knowledge representations and the capability of our models in learning knowledge representations with images.

scholarly journals Rapid Classification and Analysis for E-Commerce Goods Based on Multitask Learning

2021 ◽  
Vol 2021 ◽  
pp. 1-8
Author(s):  
Nana Liu
Keyword(s):  
Classification Accuracy ◽  
Learning Model ◽  
Multitask Learning ◽  
Attention Mechanism ◽  
Experimental Results ◽  
Training Data ◽  
Classification Task ◽  
Product Classification ◽  
Multiple Tasks ◽  
Fast Classification

Today’s E-commerce is hot, while the categorization of goods cannot be handled better, especially to achieve the demand of multiple tasks. In this paper, we propose a multitask learning model based on a CNN in parallel with a BiLSTM optimized by an attention mechanism as a training network for E-commerce. The results showed that the fast classification task of E-commerce was performed using only 10% of the total number of products. The experimental results show that the accuracy of w-item2vec for product classification can be close to 50% with only 10% of the training data. Both models significantly outperform other models in terms of classification accuracy.

scholarly journals Neural News Recommendation with Attentive Multi-View Learning

2019 ◽  
Author(s):  
Chuhan Wu ◽  
Fangzhao Wu ◽  
Mingxiao An ◽  
Jianqiang Huang ◽  
Yongfeng Huang ◽  
...  
Keyword(s):  
User Experience ◽  
Real World ◽  
Representation Learning ◽  
Learning Model ◽  
Online News ◽  
Attention Mechanism ◽  
The Core ◽  
Word Level ◽  
News Recommendation

Personalized news recommendation is very important for online news platforms to help users find interested news and improve user experience. News and user representation learning is critical for news recommendation. Existing news recommendation methods usually learn these representations based on single news information, e.g., title, which may be insufficient. In this paper we propose a neural news recommendation approach which can learn informative representations of users and news by exploiting different kinds of news information. The core of our approach is a news encoder and a user encoder. In the news encoder we propose an attentive multi-view learning model to learn unified news representations from titles, bodies and topic categories by regarding them as different views of news. In addition, we apply both word-level and view-level attention mechanism to news encoder to select important words and views for learning informative news representations. In the user encoder we learn the representations of users based on their browsed news and apply attention mechanism to select informative news for user representation learning. Extensive experiments on a real-world dataset show our approach can effectively improve the performance of news recommendation.

scholarly journals Multi-Level Representation Learning for Chinese Medical Entity Recognition: Model Development and Validation

10.2196/17637 ◽  
2020 ◽  
Vol 8 (5) ◽  
pp. e17637
Author(s):  
Zhichang Zhang ◽  
Lin Zhu ◽  
Peilin Yu
Keyword(s):  
Chinese Language ◽  
Language Model ◽  
Recognition Task ◽  
Representation Learning ◽  
Learning Model ◽  
Attention Mechanism ◽  
Entity Recognition ◽  
Semantic Representations ◽  
Medical Entity ◽  
Multi Level

Background Medical entity recognition is a key technology that supports the development of smart medicine. Existing methods on English medical entity recognition have undergone great development, but their progress in the Chinese language has been slow. Because of limitations due to the complexity of the Chinese language and annotated corpora, these methods are based on simple neural networks, which cannot effectively extract the deep semantic representations of electronic medical records (EMRs) and be used on the scarce medical corpora. We thus developed a new Chinese EMR (CEMR) dataset with six types of entities and proposed a multi-level representation learning model based on Bidirectional Encoder Representation from Transformers (BERT) for Chinese medical entity recognition. Objective This study aimed to improve the performance of the language model by having it learn multi-level representation and recognize Chinese medical entities. Methods In this paper, the pretraining language representation model was investigated; utilizing information not only from the final layer but from intermediate layers was found to affect the performance of the Chinese medical entity recognition task. Therefore, we proposed a multi-level representation learning model for entity recognition in Chinese EMRs. Specifically, we first used the BERT language model to extract semantic representations. Then, the multi-head attention mechanism was leveraged to automatically extract deeper semantic information from each layer. Finally, semantic representations from multi-level representation extraction were utilized as the final semantic context embedding for each token and we used softmax to predict the entity tags. Results The best F1 score reached by the experiment was 82.11% when using the CEMR dataset, and the F1 score when using the CCKS (China Conference on Knowledge Graph and Semantic Computing) 2018 benchmark dataset further increased to 83.18%. Various comparative experiments showed that our proposed method outperforms methods from previous work and performs as a new state-of-the-art method. Conclusions The multi-level representation learning model is proposed as a method to perform the Chinese EMRs entity recognition task. Experiments on two clinical datasets demonstrate the usefulness of using the multi-head attention mechanism to extract multi-level representation as part of the language model.

scholarly journals A Global-Local Blur Disentangling Network for Dynamic Scene Deblurring

2021 ◽  
Vol 11 (5) ◽  
pp. 2174
Author(s):  
Xiaoguang Li ◽  
Feifan Yang ◽  
Jianglu Huang ◽  
Li Zhuo
Keyword(s):  
Local Features ◽  
Attention Mechanism ◽  
Experimental Results ◽  
Dynamic Scene ◽  
Feature Maps ◽  
Training Scheme ◽  
Real Scene ◽  
Global And Local

Images captured in a real scene usually suffer from complex non-uniform degradation, which includes both global and local blurs. It is difficult to handle the complex blur variances by a unified processing model. We propose a global-local blur disentangling network, which can effectively extract global and local blur features via two branches. A phased training scheme is designed to disentangle the global and local blur features, that is the branches are trained with task-specific datasets, respectively. A branch attention mechanism is introduced to dynamically fuse global and local features. Complex blurry images are used to train the attention module and the reconstruction module. The visualized feature maps of different branches indicated that our dual-branch network can decouple the global and local blur features efficiently. Experimental results show that the proposed dual-branch blur disentangling network can improve both the subjective and objective deblurring effects for real captured images.

scholarly journals COMBINATORIAL RECONSTRUCTION OF HALF-SIBLING GROUPS FROM MICROSATELLITE DATA

2010 ◽  
Vol 08 (02) ◽  
pp. 337-356 ◽  
Author(s):  
SAAD I. SHEIKH ◽  
TANYA Y. BERGER-WOLF ◽  
ASHFAQ A. KHOKHAR ◽  
ISABEL C. CABALLERO ◽  
MARY V. ASHLEY ◽  
...  
Keyword(s):  
Exact Solutions ◽  
Experimental Results ◽  
Microsatellite Data ◽  
Reconstruction Problem ◽  
Full Sibling ◽  
Synthetic Datasets ◽  
Sibling Group ◽  
Half Sibling ◽  
Sibling Groups

While full-sibling group reconstruction from microsatellite data is a well-studied problem, reconstruction of half-sibling groups is much less studied, theoretically challenging, and computationally demanding. In this paper, we present a formulation of the half-sibling reconstruction problem and prove its APX-hardness. We also present exact solutions for this formulation and develop heuristics. Using biological and synthetic datasets we present experimental results and compare them with the leading alternative software COLONY. We show that our results are competitive and allow half-sibling group reconstruction in the presence of polygamy, which is prevalent in nature.

scholarly journals Multi-Task Self-Supervised Learning for Disfluency Detection

2020 ◽  
Vol 34 (05) ◽  
pp. 9193-9200
Author(s):  
Shaolei Wang ◽  
Wangxiang Che ◽  
Qi Liu ◽  
Pengda Qin ◽  
Ting Liu ◽  
...  
Keyword(s):  
Supervised Learning ◽  
Large Scale ◽  
Experimental Results ◽  
Training Data ◽  
Competitive Performance ◽  
Test Set ◽  
Full Dataset ◽  
Sentence Classification ◽  
Trained Network

Most existing approaches to disfluency detection heavily rely on human-annotated data, which is expensive to obtain in practice. To tackle the training data bottleneck, we investigate methods for combining multiple self-supervised tasks-i.e., supervised tasks where data can be collected without manual labeling. First, we construct large-scale pseudo training data by randomly adding or deleting words from unlabeled news data, and propose two self-supervised pre-training tasks: (i) tagging task to detect the added noisy words. (ii) sentence classification to distinguish original sentences from grammatically-incorrect sentences. We then combine these two tasks to jointly train a network. The pre-trained network is then fine-tuned using human-annotated disfluency detection training data. Experimental results on the commonly used English Switchboard test set show that our approach can achieve competitive performance compared to the previous systems (trained using the full dataset) by using less than 1% (1000 sentences) of the training data. Our method trained on the full dataset significantly outperforms previous methods, reducing the error by 21% on English Switchboard.

Sign in / Sign up

Export Citation Format

Share Document