scholarly journals An Attention-Based Latent Information Extraction Network (ALIEN) for High-Order Feature Interactions

2020 ◽  
Vol 10 (16) ◽  
pp. 5468
Author(s):  
Ruo Huang ◽  
Shelby McIntyre ◽  
Meina Song ◽  
Haihong E ◽  
Zhonghong Ou
Keyword(s):  
Information Extraction ◽  
Recommender Systems ◽  
Vital Role ◽  
High Order ◽  
Sequential Patterns ◽  
Sequence Information ◽  
Recommendation Algorithm ◽  
Feature Interactions ◽  
Real World Datasets ◽  
Click Through Rate

One of the primary tasks for commercial recommender systems is to predict the probabilities of users clicking items, e.g., advertisements, music and products. This is because such predictions have a decisive impact on profitability. The classic recommendation algorithm, collaborative filtering (CF), still plays a vital role in many industrial recommender systems. However, although straight CF is good at capturing similar users’ preferences for items based on their past interactions, it lacks regarding (1) modeling the influences of users’ sequential patterns from their individual history interaction sequences and (2) the relevance of users’ and items’ attributes. In this work, we developed an attention-based latent information extraction network (ALIEN) for click-through rate prediction, to integrate (1) implicit user similarity in terms of click patterns (analogous to CF), and (2) modeling the low and high-order feature interactions and (3) historical sequence information. The new model is based on the deep learning, which goes beyond the capabilities of econometric approaches, such as matrix factorization (MF) and k-means. In addition, the approach provides explainability to the recommendation by interpreting the contributions of different features and historical interactions. We have conducted experiments on real-world datasets that demonstrate considerable improvements over strong baselines.

scholarly journals sDeepFM: Multi-Scale Stacking Feature Interactions for Click-Through Rate Prediction

Electronics ◽  
2020 ◽  
Vol 9 (2) ◽  
pp. 350
Author(s):  
Baohua Qiang ◽  
Yongquan Lu ◽  
Minghao Yang ◽  
Xianjun Chen ◽  
Jinlong Chen ◽  
...  
Keyword(s):  
State Of The Art ◽  
Receptive Fields ◽  
Area Under The Curve ◽  
Sparse Data ◽  
High Order ◽  
Multi Scale ◽  
Feature Interactions ◽  
Novel Structure ◽  
Real World Datasets ◽  
Click Through Rate

For estimating the click-through rate of advertisements, there are some problems in that the features cannot be automatically constructed, or the features built are relatively simple, or the high-order combination features are difficult to learn under sparse data. To solve these problems, we propose a novel structure multi-scale stacking pooling (MSSP) to construct multi-scale features based on different receptive fields. The structure stacks multi-scale features bi-directionally from the angles of depth and width by constructing multiple observers with different angles and different fields of view, ensuring the diversity of extracted features. Furthermore, by learning the parameters through factorization, the structure can ensure high-order features being effectively learned in sparse data. We further combine the MSSP with the classical deep neural network (DNN) to form a unified model named sDeepFM. Experimental results on two real-world datasets show that the sDeepFM outperforms state-of-the-art models with respect to area under the curve (AUC) and log loss.

HSRec: Hierarchical self-attention incorporating knowledge graph for sequential recommendation

2021 ◽  
pp. 1-12
Author(s):  
Zuoxi Yang ◽  
Shoubin Dong
Keyword(s):  
Semantic Information ◽  
High Order ◽  
Attention Mechanism ◽  
Sequential Patterns ◽  
Knowledge Graph ◽  
Intrinsic Interest ◽  
Fine Grained ◽  
Potential Interest ◽  
Preference Evolution ◽  
Real World Datasets

Modeling user’s fine-grained preferences and dynamic preference evolution from their chronological behaviors are challenging and crucial for sequential recommendation. In this paper, we develop a Hierarchical Self-Attention Incorporating Knowledge Graph for Sequential Recommendation (HSRec). HSRec models not only the user’s intrinsic preferences but also the user’s external potential interests to capture the user’s fine-grained preferences. Specifically, the intrinsic interest module and potential interest module are designed to capture these two preferences respectively. In the intrinsic interest module, user’s sequential patterns are characterized from their behaviors via the self-attention mechanism. As for the potential interest module, high-order paths can be generated with the help of the knowledge graph. Therefore, a hierarchical self-attention mechanism is designed to aggregate the semantic information of user interaction from these paths. Specifically, an entity-level self-attention mechanism is applied to capture the sequential patterns contained in the high-order paths while an interaction-level self-attention mechanism is designed to further capture the semantic information from user interactions. Moreover, according to the high-order semantic relevance, HSRec can explore the user’s dynamic preferences at each time, thus describing the user’s dynamic preference evolution. Finally, experiments conducted on three real world datasets demonstrate the state-of-the-art performance of the HSRec.

Deep context interaction network based on attention mechanism for click-through rate prediction

2021 ◽  
pp. 1-16
Author(s):  
Ling Yuan ◽  
Zhuwen Pan ◽  
Ping Sun ◽  
Yinzhen Wei ◽  
Haiping Yu
Keyword(s):  
Online Advertising ◽  
Dimensional Space ◽  
Interaction Network ◽  
High Order ◽  
Attention Mechanism ◽  
Feature Interaction ◽  
Feature Interactions ◽  
Low Dimensional ◽  
Click Through Rate ◽  
The Relationship

Click-through rate (CTR) prediction, which aims to predict the probability of a user clicking on an ad, is a critical task in online advertising systems. The problem is very challenging since(1) an effective prediction relies on high-order combinatorial features, and(2)the relationship to auxiliary ads that may impact the CTR. In this paper, we propose Deep Context Interaction Network on Attention Mechanism(DCIN-Attention) to process feature interaction and context at the same time. The context includes other ads in the current search page, historically clicked and unclicked ads of the user. Specifically, we use the attention mechanism to learn the interactions between the target ad and each type of auxiliary ad. The residual network is used to model the feature interactions in the low-dimensional space, and with the multi-head self-attention neural network, high-order feature interactions can be modeled. Experimental results on Avito dataset show that DCIN outperform several existing methods for CTR prediction.

scholarly journals CFM: Convolutional Factorization Machines for Context-Aware Recommendation

2019 ◽  
Author(s):  
Xin Xin ◽  
Bo Chen ◽  
Xiangnan He ◽  
Dong Wang ◽  
Yue Ding ◽  
...  
Keyword(s):  
Second Order ◽  
High Order ◽  
Inner Product ◽  
Context Aware ◽  
Effective Solution ◽  
High Order Interaction ◽  
Outer Product ◽  
Feature Interactions ◽  
Factorization Machine ◽  
Real World Datasets

Factorization Machine (FM) is an effective solution for context-aware recommender systems (CARS) which models second-order feature interactions by inner product. However, it is insufficient to capture high-order and nonlinear interaction signals. While several recent efforts have enhanced FM with neural networks, they assume the embedding dimensions are independent from each other and model high-order interactions in a rather implicit manner. In this paper, we propose Convolutional Factorization Machine (CFM) to address above limitations. Specifically, CFM models second-order interactions with outer product, resulting in ''images'' which capture correlations between embedding dimensions. Then all generated ''images'' are stacked, forming an interaction cube. 3D convolution is applied above it to learn high-order interaction signals in an explicit approach. Besides, we also leverage a self-attention mechanism to perform the pooling of features to reduce time complexity. We conduct extensive experiments on three real-world datasets, demonstrating significant improvement of CFM over competing methods for context-aware top-k recommendation.

scholarly journals An Attention-Based Recommender System to Predict Contextual Intent Based on Choice Histories across and within Sessions

2018 ◽  
Vol 8 (12) ◽  
pp. 2426 ◽  
Author(s):  
Ruo Huang ◽  
Shelby McIntyre ◽  
Meina Song ◽  
Haihong E ◽  
Zhonghong Ou
Keyword(s):  
Recommender Systems ◽  
State Of The Art ◽  
User Profile ◽  
Vital Role ◽  
Short Term ◽  
Novel Approach ◽  
Learning Techniques ◽  
Real World Datasets ◽  
Near Future

Recent years have witnessed the growth of recommender systems, with the help of deep learning techniques. Recurrent Neural Networks (RNNs) play an increasingly vital role in various session-based recommender systems, since they use the user’s sequential history to build a comprehensive user profile, which helps improve the recommendation. However, a problem arises regarding how to be aware of the variation in the user’s contextual preference, especially the short-term intent in the near future, and make the best use of it to produce a precise recommendation at the start of a session. We propose a novel approach named Attention-based Short-term and Long-term Model (ASLM), to improve the next-item recommendation, by using an attention-based RNNs integrating both the user’s short-term intent and the long-term preference at the same time with a two-layer network. The experimental study on three real-world datasets and two sub-datasets demonstrates that, compared with other state-of-the-art methods, the proposed approach can significantly improve the next-item recommendation, especially at the start of sessions. As a result, our proposed approach is capable of coping with the cold-start problem at the beginning of each session.

An Overview of Collaborative Filtering Recommendation Algorithm

2013 ◽  
Vol 756-759 ◽  
pp. 3899-3903
Author(s):  
Ping Sun ◽  
Zheng Yu Li ◽  
Zi Yang Han ◽  
Feng Ying Wang
Keyword(s):  
Collaborative Filtering ◽  
Recommender Systems ◽  
Evaluation Method ◽  
Decisive Role ◽  
Classical Solutions ◽  
Critical Problem ◽  
Recommendation Algorithm ◽  
Data Sparseness ◽  
Collaborative Filtering Algorithm ◽  
And Performance

Recommendation algorithm is the most core and key point in recommender systems, and plays a decisive role in type and performance evaluation. At present collaborative filtering recommendation not only is the most widely useful and successful recommend technology, but also is a promotion for the study of the whole recommender systems. The research on the recommender systems is coming into a focus and critical problem at home and abroad. Firstly, the latest development and research in the collaborative filtering recommendation algorithm are introduced. Secondly, the primary idea and difficulties faced with the algorithm are explained in detail. Some classical solutions are used to deal with the problems such as data sparseness, cold start and augmentability. Thirdly, the particular evaluation method of the algorithm is put forward and the developments of collaborative filtering algorithm are prospected.

scholarly journals Seq-SymRF: a random forest model predicts potential miRNA-disease associations based on information of sequences and clinical symptoms

2020 ◽  
Vol 10 (1) ◽  
Author(s):  
Jinlong Li ◽  
Xingyu Chen ◽  
Qixing Huang ◽  
Yang Wang ◽  
Yun Xie ◽  
...  
Keyword(s):  
Random Forest ◽  
Breast Neoplasms ◽  
Clinical Symptoms ◽  
Characteristic Curve ◽  
Vital Role ◽  
Sequence Information ◽  
Source Codes ◽  
Drug Identification ◽  
Disease Associations ◽  
Precision Recall Curve

Abstract Increasing evidence indicates that miRNAs play a vital role in biological processes and are closely related to various human diseases. Research on miRNA-disease associations is helpful not only for disease prevention, diagnosis and treatment, but also for new drug identification and lead compound discovery. A novel sequence- and symptom-based random forest algorithm model (Seq-SymRF) was developed to identify potential associations between miRNA and disease. Features derived from sequence information and clinical symptoms were utilized to characterize miRNA and disease, respectively. Moreover, the clustering method by calculating the Euclidean distance was adopted to construct reliable negative samples. Based on the fivefold cross-validation, Seq-SymRF achieved the accuracy of 98.00%, specificity of 99.43%, sensitivity of 96.58%, precision of 99.40% and Matthews correlation coefficient of 0.9604, respectively. The areas under the receiver operating characteristic curve and precision recall curve were 0.9967 and 0.9975, respectively. Additionally, case studies were implemented with leukemia, breast neoplasms and hsa-mir-21. Most of the top-25 predicted disease-related miRNAs (19/25 for leukemia; 20/25 for breast neoplasms) and 15 of top-25 predicted miRNA-related diseases were verified by literature and dbDEMC database. It is anticipated that Seq-SymRF could be regarded as a powerful high-throughput virtual screening tool for drug research and development. All source codes can be downloaded from https://github.com/LeeKamlong/Seq-SymRF.

scholarly journals Embedding-Based Complex Feature Value Coupling Learning for Detecting Outliers in Non-IID Categorical Data

2019 ◽  
Vol 33 ◽  
pp. 5541-5548 ◽  
Author(s):  
Hongzuo Xu ◽  
Yongjun Wang ◽  
Zhiyue Wu ◽  
Yijie Wang
Keyword(s):  
Outlier Detection ◽  
Categorical Data ◽  
State Of The Art ◽  
High Order ◽  
Detection Methods ◽  
Order Complex ◽  
Value Network ◽  
Learning Framework ◽  
A Value ◽  
Real World Datasets

Non-IID categorical data is ubiquitous and common in realworld applications. Learning various kinds of couplings has been proved to be a reliable measure when detecting outliers in such non-IID data. However, it is a critical yet challenging problem to model, represent, and utilise high-order complex value couplings. Existing outlier detection methods normally only focus on pairwise primary value couplings and fail to uncover real relations that hide in complex couplings, resulting in suboptimal and unstable performance. This paper introduces a novel unsupervised embedding-based complex value coupling learning framework EMAC and its instance SCAN to address these issues. SCAN first models primary value couplings. Then, coupling bias is defined to capture complex value couplings with different granularities and highlight the essence of outliers. An embedding method is performed on the value network constructed via biased value couplings, which further learns high-order complex value couplings and embeds these couplings into a value representation matrix. Bidirectional selective value coupling learning is proposed to show how to estimate value and object outlierness through value couplings. Substantial experiments show that SCAN (i) significantly outperforms five state-of-the-art outlier detection methods on thirteen real-world datasets; and (ii) has much better resilience to noise than its competitors.

Expert Detection and Recommendation Model With User-Generated Tags in Collaborative Tagging Systems

2020 ◽  
Vol 31 (4) ◽  
pp. 24-45
Author(s):  
Mengmeng Shen ◽  
Jun Wang ◽  
Ou Liu ◽  
Haiying Wang
Keyword(s):  
Collaborative Filtering ◽  
Community Detection ◽  
Real World ◽  
Agglomerative Hierarchical Clustering ◽  
Collaborative Tagging ◽  
Excellent Performance ◽  
Recommendation Algorithm ◽  
Proposed Model ◽  
Target User ◽  
Real World Datasets

Tags generated in collaborative tagging systems (CTSs) may help users describe, categorize, search, discover, and navigate content, whereas the difficulty is how to go beyond the information explosion and obtain experts and the required information quickly and accurately. This paper proposes an expert detection and recommendation (EDAR) model based on semantics of tags; the framework consists of community detection and EDAR. Specifically, this paper firstly mines communities based on an improved agglomerative hierarchical clustering (I-AHC) to cluster tags and then presents a community expert detection (CED) algorithm for identifying community experts, and finally, an expert recommendation algorithm is proposed based the improved collaborative filtering (CF) algorithm to recommend relevant experts for the target user. Experiments are carried out on real world datasets, and the results from data experiments and user evaluations have shown that the proposed model can provide excellent performance compared to the benchmark method.

Two Phase Recommendation Algorithm Based on Clustering User Interest and Trust

2014 ◽  
Vol 543-547 ◽  
pp. 1856-1859
Author(s):  
Xiang Cui ◽  
Gui Sheng Yin
Keyword(s):  
Social Networks ◽  
Electronic Commerce ◽  
Recommender Systems ◽  
Information Overload ◽  
User Interest ◽  
Recommendation Algorithm ◽  
Two Phase ◽  
Rating Data ◽  
User Interests ◽  
The Future

Recommender systems have been proven to be valuable means for Web online users to cope with the information overload and have become one of the most powerful and popular tools in electronic commerce. We need a method to solve such as what items to buy, what music to listen, or what news to read. The diversification of user interests and untruthfulness of rating data are the important problems of recommendation. In this article, we propose to use two phase recommendation based on user interest and trust ratings that have been given by actors to items. In the paper, we deal with the uncertain user interests by clustering firstly. In the algorithm, we compute the between-class entropy of any two clusters and get the stable classes. Secondly, we construct trust based social networks, and work out the trust scoring, in the class. At last, we provide some evaluation of the algorithms and propose the more improve ideas in the future.

Sign in / Sign up

Export Citation Format

Share Document