scholarly journals A Novel Integrated Network with LightGBM for Click-Through Rate Prediction

2021 ◽  
Author(s):  
Zhen Xia ◽  
Senlin Mao ◽  
Jing Bai ◽  
Xinyu Geng ◽  
Liu Yi
Keyword(s):  
Recommendation System ◽  
Prediction Models ◽  
Online Advertising ◽  
Product Layer ◽  
Gradient Boosting ◽  
Integrated Network ◽  
Feature Interactions ◽  
Light Gradient ◽  
Click Through Rate ◽  
Paper Product

Abstract Click-through Rate (CTR) prediction has become one of the core tasks of the recommendation system and its online advertising with the development of e-commerce. In the CTR prediction field, different features extraction schemes are used to mine the user click behavior to achieve the maximum CTR, which helps the advertisers maximize their profits. At present, achievements have been made in CTR prediction based on Deep Neural Network (DNN), but insufficiently, DNN can only learn high-order features combination. In this paper, Product & Cross supported Stacking Network with LightGBM (PCSNL) is proposed for CTR prediction to solve such problems. Firstly, the L 1 and L 2 regularizations are imposed on Light Gradient Boosting Machine (LightGBM) to prevent overfitting. Secondly, the method of vector-wise feature interactions is added to product layer in product network to learn second-order feature combinations. Lastly, feature information is fully learned through the cross network, product network and stacking network in PCSNL. The online ads CTR prediction datasets released by Huawei and Avazu on the Kaggle platform are involved for experiments. It is shown that the PCSN model and PCSNL have better performance than the traditional CTR prediction models and deep learning models.

scholarly journals ECG-AI: electrocardiographic artificial intelligence model for prediction of heart failure

2021 ◽  
Author(s):  
Oguz Akbilgic ◽  
Liam Butler ◽  
Ibrahim Karabayir ◽  
Patricia P Chang ◽  
Dalane W Kitzman ◽  
...  
Keyword(s):  
Neural Network ◽  
Artificial Intelligence ◽  
Heart Failure ◽  
Convolutional Neural Network ◽  
Prediction Models ◽  
Smoking Status ◽  
Characteristic Curve ◽  
Gradient Boosting ◽  
Model Output ◽  
Light Gradient

Abstract Aims Heart failure (HF) is a leading cause of death. Early intervention is the key to reduce HF-related morbidity and mortality. This study assesses the utility of electrocardiograms (ECGs) in HF risk prediction. Methods and results Data from the baseline visits (1987–89) of the Atherosclerosis Risk in Communities (ARIC) study was used. Incident hospitalized HF events were ascertained by ICD codes. Participants with good quality baseline ECGs were included. Participants with prevalent HF were excluded. ECG-artificial intelligence (AI) model to predict HF was created as a deep residual convolutional neural network (CNN) utilizing standard 12-lead ECG. The area under the receiver operating characteristic curve (AUC) was used to evaluate prediction models including (CNN), light gradient boosting machines (LGBM), and Cox proportional hazards regression. A total of 14 613 (45% male, 73% of white, mean age ± standard deviation of 54 ± 5) participants were eligible. A total of 803 (5.5%) participants developed HF within 10 years from baseline. Convolutional neural network utilizing solely ECG achieved an AUC of 0.756 (0.717–0.795) on the hold-out test data. ARIC and Framingham Heart Study (FHS) HF risk calculators yielded AUC of 0.802 (0.750–0.850) and 0.780 (0.740–0.830). The highest AUC of 0.818 (0.778–0.859) was obtained when ECG-AI model output, age, gender, race, body mass index, smoking status, prevalent coronary heart disease, diabetes mellitus, systolic blood pressure, and heart rate were used as predictors of HF within LGBM. The ECG-AI model output was the most important predictor of HF. Conclusions ECG-AI model based solely on information extracted from ECG independently predicts HF with accuracy comparable to existing FHS and ARIC risk calculators.

scholarly journals Patient-Level Cancer Prediction Models From a Nationwide Patient Cohort: Model Development and Validation (Preprint)

2021 ◽  
Author(s):  
Eunsaem Lee ◽  
Se Young Jung ◽  
Hyung Ju Hwang ◽  
Jaewoo Jung
Keyword(s):  
Prediction Models ◽  
Area Under The Curve ◽  
Claim Data ◽  
Gradient Boosting ◽  
Health Checkup ◽  
Cancer Prediction ◽  
Patient Level ◽  
Light Gradient ◽  
Precision Recall Curve ◽  
Recall Curve

BACKGROUND Nationwide population-based cohorts provide a new opportunity to build automated risk prediction models at the patient level, and claim data are one of the more useful resources to this end. To avoid unnecessary diagnostic intervention after cancer screening tests, patient-level prediction models should be developed. OBJECTIVE We aimed to develop cancer prediction models using nationwide claim databases with machine learning algorithms, which are explainable and easily applicable in real-world environments. METHODS As source data, we used the Korean National Insurance System Database. Every Korean in ≥40 years old undergoes a national health checkup every 2 years. We gathered all variables from the database including demographic information, basic laboratory values, anthropometric values, and previous medical history. We applied conventional logistic regression methods, light gradient boosting methods, neural networks, survival analysis, and one-class embedding classifier methods to effectively analyze high dimension data based on deep learning–based anomaly detection. Performance was measured with area under the curve and area under precision recall curve. We validated our models externally with a health checkup database from a tertiary hospital. RESULTS The one-class embedding classifier model received the highest area under the curve scores with values of 0.868, 0.849, 0.798, 0.746, 0.800, 0.749, and 0.790 for liver, lung, colorectal, pancreatic, gastric, breast, and cervical cancers, respectively. For area under precision recall curve, the light gradient boosting models had the highest score with values of 0.383, 0.401, 0.387, 0.300, 0.385, 0.357, and 0.296 for liver, lung, colorectal, pancreatic, gastric, breast, and cervical cancers, respectively. CONCLUSIONS Our results show that it is possible to easily develop applicable cancer prediction models with nationwide claim data using machine learning. The 7 models showed acceptable performances and explainability, and thus can be distributed easily in real-world environments.

scholarly journals Predicting Hard Rock Pillar Stability Using GBDT, XGBoost, and LightGBM Algorithms

Mathematics ◽  
2020 ◽  
Vol 8 (5) ◽  
pp. 765 ◽  
Author(s):  
Weizhang Liang ◽  
Suizhi Luo ◽  
Guoyan Zhao ◽  
Hao Wu
Keyword(s):  
Large Scale ◽  
Prediction Models ◽  
Hard Rock ◽  
Gradient Boosting ◽  
Pillar Stability ◽  
Rock Pillar ◽  
Light Gradient ◽  
Gradient Boosting Machine ◽  
Extreme Gradient Boosting ◽  
Hard Rock Mines

Predicting pillar stability is a vital task in hard rock mines as pillar instability can cause large-scale collapse hazards. However, it is challenging because the pillar stability is affected by many factors. With the accumulation of pillar stability cases, machine learning (ML) has shown great potential to predict pillar stability. This study aims to predict hard rock pillar stability using gradient boosting decision tree (GBDT), extreme gradient boosting (XGBoost), and light gradient boosting machine (LightGBM) algorithms. First, 236 cases with five indicators were collected from seven hard rock mines. Afterwards, the hyperparameters of each model were tuned using a five-fold cross validation (CV) approach. Based on the optimal hyperparameters configuration, prediction models were constructed using training set (70% of the data). Finally, the test set (30% of the data) was adopted to evaluate the performance of each model. The precision, recall, and F1 indexes were utilized to analyze prediction results of each level, and the accuracy and their macro average values were used to assess the overall prediction performance. Based on the sensitivity analysis of indicators, the relative importance of each indicator was obtained. In addition, the safety factor approach and other ML algorithms were adopted as comparisons. The results showed that GBDT, XGBoost, and LightGBM algorithms achieved a better comprehensive performance, and their prediction accuracies were 0.8310, 0.8310, and 0.8169, respectively. The average pillar stress and ratio of pillar width to pillar height had the most important influences on prediction results. The proposed methodology can provide a reliable reference for pillar design and stability risk management.

scholarly journals Computational analysis and prediction of lysine malonylation sites by exploiting informative features in an integrative machine-learning framework

2018 ◽  
Vol 20 (6) ◽  
pp. 2185-2199 ◽  
Author(s):  
Yanju Zhang ◽  
Ruopeng Xie ◽  
Jiawei Wang ◽  
André Leier ◽  
Tatiana T Marquez-Lago ◽  
...  
Keyword(s):  
Machine Learning ◽  
Computational Methods ◽  
Prediction Models ◽  
Gradient Boosting ◽  
Support Vector ◽  
Post Translational Modification ◽  
K Nearest Neighbor ◽  
Ensemble Models ◽  
Light Gradient ◽  
Optimal Ensemble

AbstractAs a newly discovered post-translational modification (PTM), lysine malonylation (Kmal) regulates a myriad of cellular processes from prokaryotes to eukaryotes and has important implications in human diseases. Despite its functional significance, computational methods to accurately identify malonylation sites are still lacking and urgently needed. In particular, there is currently no comprehensive analysis and assessment of different features and machine learning (ML) methods that are required for constructing the necessary prediction models. Here, we review, analyze and compare 11 different feature encoding methods, with the goal of extracting key patterns and characteristics from residue sequences of Kmal sites. We identify optimized feature sets, with which four commonly used ML methods (random forest, support vector machines, K-nearest neighbor and logistic regression) and one recently proposed [Light Gradient Boosting Machine (LightGBM)] are trained on data from three species, namely, Escherichia coli, Mus musculus and Homo sapiens, and compared using randomized 10-fold cross-validation tests. We show that integration of the single method-based models through ensemble learning further improves the prediction performance and model robustness on the independent test. When compared to the existing state-of-the-art predictor, MaloPred, the optimal ensemble models were more accurate for all three species (AUC: 0.930, 0.923 and 0.944 for E. coli, M. musculus and H. sapiens, respectively). Using the ensemble models, we developed an accessible online predictor, kmal-sp, available at http://kmalsp.erc.monash.edu/. We hope that this comprehensive survey and the proposed strategy for building more accurate models can serve as a useful guide for inspiring future developments of computational methods for PTM site prediction, expedite the discovery of new malonylation and other PTM types and facilitate hypothesis-driven experimental validation of novel malonylated substrates and malonylation sites.

scholarly journals Application of Machine-Learning-Based Fusion Model in Visibility Forecast: A Case Study of Shanghai, China

2021 ◽  
Vol 13 (11) ◽  
pp. 2096
Author(s):  
Zhongqi Yu ◽  
Yuanhao Qu ◽  
Yunxin Wang ◽  
Jinghui Ma ◽  
Yu Cao
Keyword(s):  
Machine Learning ◽  
Prediction Models ◽  
Eastern China ◽  
Prediction Method ◽  
Sampling Technique ◽  
Environmental Modeling ◽  
Gradient Boosting ◽  
Fusion Model ◽  
Light Gradient ◽  
Extreme Gradient Boosting

A visibility forecast model called a boosting-based fusion model (BFM) was established in this study. The model uses a fusion machine learning model based on multisource data, including air pollutants, meteorological observations, moderate resolution imaging spectroradiometer (MODIS) aerosol optical depth (AOD) data, and an operational regional atmospheric environmental modeling System for eastern China (RAEMS) outputs. Extreme gradient boosting (XGBoost), a light gradient boosting machine (LightGBM), and a numerical prediction method, i.e., RAEMS were fused to establish this prediction model. Three sets of prediction models, that is, BFM, LightGBM based on multisource data (LGBM), and RAEMS, were used to conduct visibility prediction tasks. The training set was from 1 January 2015 to 31 December 2018 and used several data pre-processing methods, including a synthetic minority over-sampling technique (SMOTE) data resampling, a loss function adjustment, and a 10-fold cross verification. Moreover, apart from the basic features (variables), more spatial and temporal gradient features were considered. The testing set was from 1 January to 31 December 2019 and was adopted to validate the feasibility of the BFM, LGBM, and RAEMS. Statistical indicators confirmed that the machine learning methods improved the RAEMS forecast significantly and consistently. The root mean square error and correlation coefficient of BFM for the next 24/48 h were 5.01/5.47 km and 0.80/0.77, respectively, which were much higher than those of RAEMS. The statistics and binary score analysis for different areas in Shanghai also proved the reliability and accuracy of using BFM, particularly in low-visibility forecasting. Overall, BFM is a suitable tool for predicting the visibility. It provides a more accurate visibility forecast for the next 24 and 48 h in Shanghai than LGBM and RAEMS. The results of this study provide support for real-time operational visibility forecasts.

scholarly journals Buckling and ultimate load prediction models for perforated steel beams using machine learning algorithms

2021 ◽  
Author(s):  
Vitaliy Degtyarev ◽  
Konstantinos Daniel Tsavdaridis
Keyword(s):  
Machine Learning ◽  
Web Application ◽  
Failure Modes ◽  
Ultimate Load ◽  
Prediction Models ◽  
Machine Learning Algorithms ◽  
Gradient Boosting ◽  
Elastic Buckling ◽  
Light Gradient ◽  
Extreme Gradient Boosting

Large web openings introduce complex structural behaviors and additional failure modes of steel cellular beams, which must be considered in the design using laborious calculations (e.g., exercising SCI P355). This paper presents seven machine learning (ML) models, including decision tree (DT), random forest (RF), k-nearest neighbor (KNN), gradient boosting regressor (GBR), extreme gradient boosting (XGBoost), light gradient boosting machine (LightGBM), and gradient boosting with categorical features support (CatBoost), for predicting the elastic buckling and ultimate loads of steel cellular beams. Large datasets of finite element (FE) simulation results, validated against experimental data, were used to develop the models. The ML models were fine-tuned via an extensive hyperparameter search to obtain their best performance. The elastic buckling and ultimate loads predicted by the optimized ML models demonstrated excellent agreement with the numerical data. The accuracy of the ultimate load predictions by the ML models exceeded the accuracy provided by the existing design provisions for steel cellular beams published in SCI P355 and AISC Design Guide 31. The relative feature importance and feature dependence of the models were evaluated and discussed in the paper. An interactive Python-based notebook and a user-friendly web application for predicting the elastic buckling and ultimate loads of steel cellular beams using the developed optimized ML models were created and made publicly available. The web application deployed to the cloud allows for making predictions in any web browser on any device, including mobile. The source code of the application available on GitHub allows running the application locally and independently from the cloud service.

Deep context interaction network based on attention mechanism for click-through rate prediction

2021 ◽  
pp. 1-16
Author(s):  
Ling Yuan ◽  
Zhuwen Pan ◽  
Ping Sun ◽  
Yinzhen Wei ◽  
Haiping Yu
Keyword(s):  
Online Advertising ◽  
Dimensional Space ◽  
Interaction Network ◽  
High Order ◽  
Attention Mechanism ◽  
Feature Interaction ◽  
Feature Interactions ◽  
Low Dimensional ◽  
Click Through Rate ◽  
The Relationship

Click-through rate (CTR) prediction, which aims to predict the probability of a user clicking on an ad, is a critical task in online advertising systems. The problem is very challenging since(1) an effective prediction relies on high-order combinatorial features, and(2)the relationship to auxiliary ads that may impact the CTR. In this paper, we propose Deep Context Interaction Network on Attention Mechanism(DCIN-Attention) to process feature interaction and context at the same time. The context includes other ads in the current search page, historically clicked and unclicked ads of the user. Specifically, we use the attention mechanism to learn the interactions between the target ad and each type of auxiliary ad. The residual network is used to model the feature interactions in the low-dimensional space, and with the multi-head self-attention neural network, high-order feature interactions can be modeled. Experimental results on Avito dataset show that DCIN outperform several existing methods for CTR prediction.

scholarly journals XGBDeepFM for CTR Predictions in Mobile Advertising Benefits from Ad Context

2020 ◽  
Vol 2020 ◽  
pp. 1-7
Author(s):  
Han An ◽  
Jifan Ren
Keyword(s):  
Weather Condition ◽  
Feature Learning ◽  
Gradient Boosting ◽  
Feature Interaction ◽  
Mobile Advertising ◽  
Mobile Business ◽  
Feature Interactions ◽  
Extreme Gradient Boosting ◽  
Factorization Machine ◽  
Click Through Rate

The problem of click-through rate (CTR) prediction in mobile advertising is one of the most informative metrics used in mobile business activities, such as profit evaluation and resource management. In mobile advertising, CTR prediction is essential but challenging due to data sparsity. Moreover, existing methods often have difficulty in capturing the different orders of feature interactions simultaneously. In this study, a method was developed to obtain accurate CTR prediction by incorporating contextual features and feature interactions. We initially use extreme gradient boosting (XGBoost) as a feature engineering phase to select highly significant features. The selected features are mobile contextual attributes including time contextual, geography contextual, and other contextual attributes (e.g., weather condition) in actual mobile advertising situations. Our model, XGBoost deep factorization machine- (FM-) supported neutral network (XGBDeepFM), combines the power of XGBoost for feature selection, FM for two-order cross feature interaction, and the deep neural network for high-order feature learning in a united architecture. In a mobile advertising condition, our methods lead to significantly accurate CTR prediction in “wide and deep” type of model. In comparison with existing models, many experiments on commercial datasets show that the XGBDeepFM model has better value of area under curve and improves the effectiveness and efficiency of CTR prediction for mobile advertising.

scholarly journals Patient-Level Cancer Prediction Models From a Nationwide Patient Cohort: Model Development and Validation

10.2196/29807 ◽  
2021 ◽  
Vol 9 (8) ◽  
pp. e29807
Author(s):  
Eunsaem Lee ◽  
Se Young Jung ◽  
Hyung Ju Hwang ◽  
Jaewoo Jung
Keyword(s):  
Prediction Models ◽  
Area Under The Curve ◽  
Claim Data ◽  
Gradient Boosting ◽  
Health Checkup ◽  
Cancer Prediction ◽  
Patient Level ◽  
Light Gradient ◽  
Precision Recall Curve ◽  
Recall Curve

Background Nationwide population-based cohorts provide a new opportunity to build automated risk prediction models at the patient level, and claim data are one of the more useful resources to this end. To avoid unnecessary diagnostic intervention after cancer screening tests, patient-level prediction models should be developed. Objective We aimed to develop cancer prediction models using nationwide claim databases with machine learning algorithms, which are explainable and easily applicable in real-world environments. Methods As source data, we used the Korean National Insurance System Database. Every Korean in ≥40 years old undergoes a national health checkup every 2 years. We gathered all variables from the database including demographic information, basic laboratory values, anthropometric values, and previous medical history. We applied conventional logistic regression methods, light gradient boosting methods, neural networks, survival analysis, and one-class embedding classifier methods to effectively analyze high dimension data based on deep learning–based anomaly detection. Performance was measured with area under the curve and area under precision recall curve. We validated our models externally with a health checkup database from a tertiary hospital. Results The one-class embedding classifier model received the highest area under the curve scores with values of 0.868, 0.849, 0.798, 0.746, 0.800, 0.749, and 0.790 for liver, lung, colorectal, pancreatic, gastric, breast, and cervical cancers, respectively. For area under precision recall curve, the light gradient boosting models had the highest score with values of 0.383, 0.401, 0.387, 0.300, 0.385, 0.357, and 0.296 for liver, lung, colorectal, pancreatic, gastric, breast, and cervical cancers, respectively. Conclusions Our results show that it is possible to easily develop applicable cancer prediction models with nationwide claim data using machine learning. The 7 models showed acceptable performances and explainability, and thus can be distributed easily in real-world environments.

Sign in / Sign up

Export Citation Format

Share Document