An adaptive hybrid XdeepFM based deep Interest network model for click-through rate prediction system

Recent advances in communication enable individuals to use phones and computers to access information on the web. E-commerce has seen rapid development, e.g., Alibaba has nearly 12 hundred million customers in China. Click-Through Rate (CTR) forecasting is a primary task in the e-commerce advertisement system. From the traditional Logistic Regression algorithm to the latest popular deep neural network methods that follow a similar embedding and MLP, several algorithms are used to predict CTR. This research proposes a hybrid model combining the Deep Interest Network (DIN) and eXtreme Deep Factorization Machine (xDeepFM) to perform CTR prediction robustly. The cores of DIN and xDeepFM are attention and feature cross, respectively. DIN follows an adaptive local activation unit that incorporates the attention mechanism to adaptively learn user interest from historical behaviors related to specific advertisements. xDeepFM further includes a critical part, a Compressed Interactions Network (CIN), aiming to generate feature interactions at a vectorwise level implicitly. Furthermore, a CIN, plain DNN, and a linear part are combined into one unified model to form xDeepFM. The proposed end-to-end hybrid model is a parallel ensemble of models via multilayer perceptron. CIN and xDeepFM are trained in parallel, and their output is fed into a multilayer perceptron. We used the e-commerce Alibaba dataset with the focal loss as the loss function for experimental evaluation through online complex example mining (OHEM) in the training process. The experimental result indicates that the proposed hybrid model has better performance than other models.

Download Full-text

Deep Interest Evolution Network for Click-Through Rate Prediction

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v33i01.33015941 ◽

2019 ◽

Vol 33 ◽

pp. 5941-5948 ◽

Cited By ~ 54

Author(s):

Guorui Zhou ◽

Na Mou ◽

Ying Fan ◽

Qi Pi ◽

Weijie Bian ◽

...

Keyword(s):

User Behavior ◽

Target Item ◽

State Of The Art ◽

User Interest ◽

User Interests ◽

Sequential Structure ◽

Changing Trend ◽

Click Through Rate ◽

Novel Model ◽

Deep Interest

Click-through rate (CTR) prediction, whose goal is to estimate the probability of a user clicking on the item, has become one of the core tasks in the advertising system. For CTR prediction model, it is necessary to capture the latent user interest behind the user behavior data. Besides, considering the changing of the external environment and the internal cognition, user interest evolves over time dynamically. There are several CTR prediction methods for interest modeling, while most of them regard the representation of behavior as the interest directly, and lack specially modeling for latent interest behind the concrete behavior. Moreover, little work considers the changing trend of the interest. In this paper, we propose a novel model, named Deep Interest Evolution Network (DIEN), for CTR prediction. Specifically, we design interest extractor layer to capture temporal interests from history behavior sequence. At this layer, we introduce an auxiliary loss to supervise interest extracting at each step. As user interests are diverse, especially in the e-commerce system, we propose interest evolving layer to capture interest evolving process that is relative to the target item. At interest evolving layer, attention mechanism is embedded into the sequential structure novelly, and the effects of relative interests are strengthened during interest evolution. In the experiments on both public and industrial datasets, DIEN significantly outperforms the state-of-the-art solutions. Notably, DIEN has been deployed in the display advertisement system of Taobao, and obtained 20.7% improvement on CTR.

Download Full-text

A new hybrid model combining EMD and neural network for multi-step ahead load forecasting

Journal of Intelligent & Fuzzy Systems ◽

10.3233/jifs-189775 ◽

2021 ◽

pp. 1-16

Author(s):

Hasmat Malik ◽

Majed A. Alotaibi ◽

Abdulaziz Almutairi

Keyword(s):

Neural Network ◽

Real Time ◽

Hybrid Model ◽

Storage System ◽

Load Forecasting ◽

Model Performance ◽

Energy Storage System ◽

Electric Load ◽

Real Time Analysis ◽

Model Combining

The electric load forecasting (ELF) is a key area of the modern power system (MPS) applications and also for the virtual power plant (VPP) analysis. The ELF is most prominent for the distinct applications of MPS and VPP such as real-time analysis of energy storage system, distributed energy resources, demand side management and electric vehicles etc. To manage the real-time challenges and map the stable power demand, in different time steps, the ELF is evaluated in yearly, monthly, weekly, daily, and hourly, etc. basis. In this study, an intelligent load predictor which is able to forecast the electric load for next month or day or hour is proposed. The proposed approach is a hybrid model combining empirical mode decomposition (EMD) and neural network (NN) for multi-step ahead load forecasting. The model performance is demonstrated by suing historical dataset collected form GEFCom2012 and GEFCom2014. For the demonstration of the performance, three case studies are analyzed into two categories. The demonstrated results represents the higher acceptability of the proposed approach with respect to the standard value of MAPE (mean absolute percent error).

Download Full-text

Research on Advertising Click-Through Rate Prediction Based on CNN-FM Hybrid Model

2018 10th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC) ◽

10.1109/ihmsc.2018.10119 ◽

2018 ◽

Author(s):

Xiangyang She ◽

Shaopeng Wang

Keyword(s):

Hybrid Model ◽

Rate Prediction ◽

Click Through Rate

Download Full-text

Liquid State Machine to Generate the Movement Profiles for the Gait Cycle of a 6 DOF Bipedal Robot in a Sagittal Plane

Volume 5B: 42nd Mechanisms and Robotics Conference ◽

10.1115/detc2018-86206 ◽

2018 ◽

Author(s):

Jesús Franco-Robles ◽

Alejandro De Lucio-Rangel ◽

Karla A. Camarillo-Gómez ◽

Gerardo I. Pérez-Soto ◽

Jesús Rivera-Guillén

Keyword(s):

Time Series ◽

Multilayer Perceptron ◽

Liquid State ◽

Gait Cycle ◽

Sagittal Plane ◽

Experimental Result ◽

Forward Kinematics ◽

Neuronal System ◽

Bipedal Robot ◽

Input Time

In this paper, a neuronal system with the ability to generate motion profiles and profiles of the ZMP in a 6DoF bipedal robot in the sagittal plane, is presented. The input time series for LSM training are movement profiles of the oscillating foot trajectory obtained by forward kinematics performed by a previously trained ANN multilayer perceptron. The profiles of objective movement for training are acquired from the analysis of the human walk. Based on a previous simulation of the bipedal robot, a profile of the objective ZMP will be generated for the y–axis and another for the z–axis to know its behavior during the training walk. As an experimental result, the LSM generates new motion profiles and ZMP, given a different trajectory with which it was trained. With the LSM it will be possible to propose new trajectories of the oscillating foot, where it will be known if this trajectory will be stable, by the ZMP, and what movement profile for each articulation will be required to reach this trajectory.

Download Full-text

Hybrid model using logit and nonparametric methods for predicting micro-entity failure

Investment Management and Financial Innovations ◽

10.21511/imfi.13(3).2016.03 ◽

2016 ◽

Vol 13 (3) ◽

pp. 35-46 ◽

Cited By ~ 1

Author(s):

A. Blanco-Oliver ◽

A. Irimia-Dieguez ◽

M.D. Oliver-Alfonso ◽

M.J. Vázquez-Cueto

Keyword(s):

Logistic Regression ◽

Hybrid Model ◽

Rough Set ◽

Multilayer Perceptron ◽

Predictive Power ◽

Bankruptcy Prediction ◽

Parametric Methods ◽

Macroeconomic Variables ◽

Accuracy Performance ◽

Non Parametric

Following the calls from literature on bankruptcy, a parsimonious hybrid bankruptcy model is developed in this paper by combining parametric and non-parametric approaches.To this end, the variables with the highest predictive power to detect bankruptcy are selected using logistic regression (LR). Subsequently, alternative non-parametric methods (Multilayer Perceptron, Rough Set, and Classification-Regression Trees) are applied, in turn, to firms classified as either “bankrupt” or “not bankrupt”. Our findings show that hybrid models, particularly those combining LR and Multilayer Perceptron, offer better accuracy performance and interpretability and converge faster than each method implemented in isolation. Moreover, the authors demonstrate that the introduction of non-financial and macroeconomic variables complement financial ratios for bankruptcy prediction

Download Full-text

Development of a Diabetes Melitus Detection and Prediction Model Using Light Gradient Boosting Machine and K-Nearest Neighbour

10.36108/ujees/1202.30.0160 ◽

2021 ◽

Vol 3 (1) ◽

Author(s):

B. A Omodunbi

Keyword(s):

Diabetes Mellitus ◽

Machine Learning ◽

Hybrid Model ◽

Learning Model ◽

Experimental Result ◽

Gradient Boosting ◽

Light Gradient ◽

Machine Learning Model ◽

Gradient Boosting Machine ◽

Receiver Operating

Diabetes mellitus is a health disorder that occurs when the blood sugar level becomes extremely high due to body resistance in producing the required amount of insulin. The aliment happens to be among the major causes of death in Nigeria and the world at large. This study was carried out to detect diabetes mellitus by developing a hybrid model that comprises of two machine learning model namely Light Gradient Boosting Machine (LGBM) and K-Nearest Neighbor (KNN). This research is aimed at developing a machine learning model for detecting the occurrence of diabetes in patients. The performance metrics employed in evaluating the finding for this study are Receiver Operating Characteristics (ROC) Curve, Five-fold Cross-validation, precision, and accuracy score. The proposed system had an accuracy of 91% and the area under the Receiver Operating Characteristic Curve was 93%. The experimental result shows that the prediction accuracy of the hybrid model is better than traditional machine learning

Download Full-text

Flood susceptibility mapping and assessment using a novel deep learning model combining multilayer perceptron and autoencoder neural networks

Journal of Flood Risk Management ◽

10.1111/jfr3.12683 ◽

2020 ◽

Author(s):

Mohammad Ahmadlou ◽

A'kif Al‐Fugara ◽

Abdel Rahman Al‐Shabeeb ◽

Aman Arora ◽

Rida Al‐Adamat ◽

...

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Multilayer Perceptron ◽

Learning Model ◽

Susceptibility Mapping ◽

Flood Susceptibility ◽

Model Combining ◽

Flood Susceptibility Mapping ◽

Deep Learning Model

Download Full-text

Deep Time-Stream Framework for Click-through Rate Prediction by Tracking Interest Evolution

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i04.6028 ◽

2020 ◽

Vol 34 (04) ◽

pp. 5726-5733

Author(s):

Shu-Ting Shi ◽

Wenhao Zheng ◽

Jun Tang ◽

Qing-Guo Chen ◽

Yao Hu ◽

...

Keyword(s):

Continuous Time ◽

Industrial Applications ◽

Superior Performance ◽

User Interest ◽

Additional Time ◽

Deep Time ◽

Public Dataset ◽

Click Through Rate ◽

Time Information ◽

Video Recommendation

Click-through rate (CTR) prediction is an essential task in industrial applications such as video recommendation. Recently, deep learning models have been proposed to learn the representation of users' overall interests, while ignoring the fact that interests may dynamically change over time. We argue that it is necessary to consider the continuous-time information in CTR models to track user interest trend from rich historical behaviors. In this paper, we propose a novel Deep Time-Stream framework (DTS) which introduces the time information by an ordinary differential equations (ODE). DTS continuously models the evolution of interests using a neural network, and thus is able to tackle the challenge of dynamically representing users' interests based on their historical behaviors. In addition, our framework can be seamlessly applied to any existing deep CTR models by leveraging the additional Time-Stream Module, while no changes are made to the original CTR models. Experiments on public dataset as well as real industry dataset with billions of samples demonstrate the effectiveness of proposed approaches, which achieve superior performance compared with existing methods.

Download Full-text