scholarly journals Neural News Recommendation with Attentive Multi-View Learning

2019 ◽  
Author(s):  
Chuhan Wu ◽  
Fangzhao Wu ◽  
Mingxiao An ◽  
Jianqiang Huang ◽  
Yongfeng Huang ◽  
...  
Keyword(s):  
User Experience ◽  
Real World ◽  
Representation Learning ◽  
Learning Model ◽  
Online News ◽  
Attention Mechanism ◽  
The Core ◽  
Word Level ◽  
News Recommendation

Personalized news recommendation is very important for online news platforms to help users find interested news and improve user experience. News and user representation learning is critical for news recommendation. Existing news recommendation methods usually learn these representations based on single news information, e.g., title, which may be insufficient. In this paper we propose a neural news recommendation approach which can learn informative representations of users and news by exploiting different kinds of news information. The core of our approach is a news encoder and a user encoder. In the news encoder we propose an attentive multi-view learning model to learn unified news representations from titles, bodies and topic categories by regarding them as different views of news. In addition, we apply both word-level and view-level attention mechanism to news encoder to select important words and views for learning informative news representations. In the user encoder we learn the representations of users based on their browsed news and apply attention mechanism to select informative news for user representation learning. Extensive experiments on a real-world dataset show our approach can effectively improve the performance of news recommendation.

scholarly journals Differentiated Attentive Representation Learning for Sentence Classification

2018 ◽  
Author(s):  
Qianrong Zhou ◽  
Xiaojie Wang ◽  
Xuan Dong
Keyword(s):  
Representation Learning ◽  
Learning Model ◽  
Attention Mechanism ◽  
Experimental Results ◽  
Sentence Classification ◽  
Synthetic Datasets

Attention-based models have shown to be effective in learning representations for sentence classification. They are typically equipped with multi-hop attention mechanism. However, existing multi-hop models still suffer from the problem of paying much attention to the most frequently noticed words, which might not be important to classify the current sentence. And there is a lack of explicitly effective way that helps the attention to be shifted out of a wrong part in the sentence. In this paper, we alleviate this problem by proposing a differentiated attentive learning model. It is composed of two branches of attention subnets and an example discriminator. An explicit signal with the loss information of the first attention subnet is passed on to the second one to drive them to learn different attentive preference. The example discriminator then selects the suitable attention subnet for sentence classification. Experimental results on real and synthetic datasets demonstrate the effectiveness of our model.

scholarly journals Which Attributes Matter the Most for Loan Origination? A Neural Attention Approach

2020 ◽  
Author(s):  
Antonios Alexos ◽  
Sotirios Chatzis
Keyword(s):  
Deep Learning ◽  
Real World ◽  
Individual Case ◽  
Learning Model ◽  
Attention Mechanism ◽  
Novel Approach ◽  
Specific Individual ◽  
Real World Datasets ◽  
Deep Learning Model

In this paper we address the understanding of the problem, of why a deep learning model decides that an individual is eligible for a loan or not. Here we propose a novel approach for inferring, which attributes matter the most, for making a decision in each specific individual case. Specifically we leverage concepts from neural attention to devise a novel feature wise attention mechanism. As we show, using real world datasets, our approach offers unique insights into the importance of various features, by producing a decision explanation for each specific loan case. At the same time, we observe that our novel mechanism, generates decisions which are much closer to the decisions generated by human experts, compared to the existent competitors.

scholarly journals UNBERT: User-News Matching BERT for News Recommendation

2021 ◽  
Author(s):  
Qi Zhang ◽  
Jingjie Li ◽  
Qinglin Jia ◽  
Chuyuan Wang ◽  
Jieming Zhu ◽  
...  
Keyword(s):  
State Of The Art ◽  
Cold Start ◽  
The Core ◽  
Textual Representations ◽  
Word Level ◽  
Training Technique ◽  
Language Knowledge ◽  
Textual Representation ◽  
Matching Score ◽  
News Recommendation

Nowadays, news recommendation has become a popular channel for users to access news of their interests. How to represent rich textual contents of news and precisely match users' interests and candidate news lies in the core of news recommendation. However, existing recommendation methods merely learn textual representations from in-domain news data, which limits their generalization ability to new news that are common in cold-start scenarios. Meanwhile, many of these methods represent each user by aggregating the historically browsed news into a single vector and then compute the matching score with the candidate news vector, which may lose the low-level matching signals. In this paper, we explore the use of the successful BERT pre-training technique in NLP for news recommendation and propose a BERT-based user-news matching model, called UNBERT. In contrast to existing research, our UNBERT model not only leverages the pre-trained model with rich language knowledge to enhance textual representation, but also captures multi-grained user-news matching signals at both word-level and news-level. Extensive experiments on the Microsoft News Dataset (MIND) demonstrate that our approach constantly outperforms the state-of-the-art methods.

scholarly journals DAN: Deep Attention Neural Network for News Recommendation

2019 ◽  
Vol 33 ◽  
pp. 5973-5980 ◽  
Author(s):  
Qiannan Zhu ◽  
Xiaofei Zhou ◽  
Zeliang Song ◽  
Jianlong Tan ◽  
Li Guo
Keyword(s):  
Neural Network ◽  
Recurrent Neural Network ◽  
Real World ◽  
Attention Mechanism ◽  
Experimental Results ◽  
Data Sets ◽  
Challenging Problem ◽  
World News ◽  
Sequential Information ◽  
News Recommendation

With the rapid information explosion of news, making personalized news recommendation for users becomes an increasingly challenging problem. Many existing recommendation methods that regard the recommendation procedure as the static process, have achieved better recommendation performance. However, they usually fail with the dynamic diversity of news and user’s interests, or ignore the importance of sequential information of user’s clicking selection. In this paper, taking full advantages of convolution neural network (CNN), recurrent neural network (RNN) and attention mechanism, we propose a deep attention neural network DAN for news recommendation. Our DAN model presents to use attention-based parallel CNN for aggregating user’s interest features and attention-based RNN for capturing richer hidden sequential features of user’s clicks, and combines these features for new recommendation. We conduct experiment on real-world news data sets, and the experimental results demonstrate the superiority and effectiveness of our proposed DAN model.

scholarly journals Towards Better Representation Learning for Personalized News Recommendation: a Multi-Channel Deep Fusion Approach

2018 ◽  
Author(s):  
Jianxun Lian ◽  
Fuzheng Zhang ◽  
Xing Xie ◽  
Guangzhong Sun
Keyword(s):  
Deep Learning ◽  
Service Providers ◽  
Representation Learning ◽  
Attention Mechanism ◽  
Fusion Model ◽  
Learning Abilities ◽  
Visual Computing ◽  
Latent Factor Models ◽  
Latent Representations ◽  
News Recommendation

Millions of news articles emerge every day. How to provide personalized news recommendations has become a critical task for service providers. In the past few decades, latent factor models has been widely used for building recommender systems (RSs). With the remarkable success of deep learning techniques especially in visual computing and natural language understanding, more and more researchers have been trying to leverage deep neural networks to learn latent representations for advanced RSs. Following mainstream deep learning-based RSs, we propose a novel deep fusion model (DFM), which aims to improve the representation learning abilities in deep RSs and can be used for both candidate retrieval and item re-ranking. There are two key components in our DFM approach, namely an inception module and an attention mechanism. The inception module improves the plain multi-layer network via leveraging of various levels of interaction simultaneously, while the attention mechanism merges latent representations learnt from different channels in a customized fashion. We conduct extensive experiments on a commercial news reading dataset, and the results demonstrate that the proposed DFM is superior to several state-of-the-art models.

scholarly journals Multi-Level Representation Learning for Chinese Medical Entity Recognition: Model Development and Validation

10.2196/17637 ◽  
2020 ◽  
Vol 8 (5) ◽  
pp. e17637
Author(s):  
Zhichang Zhang ◽  
Lin Zhu ◽  
Peilin Yu
Keyword(s):  
Chinese Language ◽  
Language Model ◽  
Recognition Task ◽  
Representation Learning ◽  
Learning Model ◽  
Attention Mechanism ◽  
Entity Recognition ◽  
Semantic Representations ◽  
Medical Entity ◽  
Multi Level

Background Medical entity recognition is a key technology that supports the development of smart medicine. Existing methods on English medical entity recognition have undergone great development, but their progress in the Chinese language has been slow. Because of limitations due to the complexity of the Chinese language and annotated corpora, these methods are based on simple neural networks, which cannot effectively extract the deep semantic representations of electronic medical records (EMRs) and be used on the scarce medical corpora. We thus developed a new Chinese EMR (CEMR) dataset with six types of entities and proposed a multi-level representation learning model based on Bidirectional Encoder Representation from Transformers (BERT) for Chinese medical entity recognition. Objective This study aimed to improve the performance of the language model by having it learn multi-level representation and recognize Chinese medical entities. Methods In this paper, the pretraining language representation model was investigated; utilizing information not only from the final layer but from intermediate layers was found to affect the performance of the Chinese medical entity recognition task. Therefore, we proposed a multi-level representation learning model for entity recognition in Chinese EMRs. Specifically, we first used the BERT language model to extract semantic representations. Then, the multi-head attention mechanism was leveraged to automatically extract deeper semantic information from each layer. Finally, semantic representations from multi-level representation extraction were utilized as the final semantic context embedding for each token and we used softmax to predict the entity tags. Results The best F1 score reached by the experiment was 82.11% when using the CEMR dataset, and the F1 score when using the CCKS (China Conference on Knowledge Graph and Semantic Computing) 2018 benchmark dataset further increased to 83.18%. Various comparative experiments showed that our proposed method outperforms methods from previous work and performs as a new state-of-the-art method. Conclusions The multi-level representation learning model is proposed as a method to perform the Chinese EMRs entity recognition task. Experiments on two clinical datasets demonstrate the usefulness of using the multi-head attention mechanism to extract multi-level representation as part of the language model.

scholarly journals Sublinear domination and core–periphery networks

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Marios Papachristou
Keyword(s):  
Network Model ◽  
Power Law ◽  
Real World ◽  
Random Network ◽  
Small World ◽  
The Core ◽  
Entire Network

AbstractIn this paper we devise a generative random network model with core–periphery properties whose core nodes act as sublinear dominators, that is, if the network has n nodes, the core has size o(n) and dominates the entire network. We show that instances generated by this model exhibit power law degree distributions, and incorporates small-world phenomena. We also fit our model in a variety of real-world networks.

Sign in / Sign up

Export Citation Format

Share Document