scholarly journals PENERAPAN GRADIENT BOOSTING DENGAN HYPEROPT UNTUK MEMPREDIKSI KEBERHASILAN TELEMARKETING BANK

2022 ◽  
Vol 10 (4) ◽  
pp. 617-623
Author(s):  
Silvia Elsa Suryana ◽  
Budi Warsito ◽  
Suparti Suparti
Keyword(s):  
Machine Learning ◽  
Bayesian Optimization ◽  
Gradient Boosting ◽  
Machine Learning Method ◽  
Classification Result ◽  
Independent Variables ◽  
Binary Dependent Variable ◽  
Boosting Method ◽  
Optimal Classification

Telemarketing is another form of marketing which is conducted via telephone. Bank can use telemarketing to offer its products such as term deposit. One of the most important strategy to the success of telemarketing is opting the potential customer to create effective telemarketing. Predicting the success of telemarketing can use machine learning. Gradient boosting is machine learning method with advanced decision tree. Gardient boosting involves many classification trees which are continually upgraded from previous tree. The optimal classification result cannot be separated from the role of the optimal hyperparameter.  Hyperopt is Python library that can be used to tune hyperparameter effectively because it uses Bayesian optimization. Hyperopt uses hyperparameter prior distribution to find optimal hyperparameter. Data in this study including 20 independent variables and binary dependent variable which has ‘yes’ and ‘no’ classes. The study showed that gradient boosting reached classification accuracy up to 90,39%, precision 94,91%, and AUC 0,939. These values describe gradient boosting method is able to predict both classes ‘yes’ and ‘no’ relatively accurate.

scholarly journals A Machine Learning Method for Predicting Vegetation Indices in China

2021 ◽  
Vol 13 (6) ◽  
pp. 1147
Author(s):  
Xiangqian Li ◽  
Wenping Yuan ◽  
Wenjie Dong
Keyword(s):  
Machine Learning ◽  
Growing Season ◽  
Crop Growth ◽  
Spatiotemporal Distribution ◽  
Coefficient Of Determination ◽  
Gradient Boosting ◽  
Severe Drought ◽  
Vegetation Growth ◽  
Extreme Gradient Boosting ◽  
Boosting Method

To forecast the terrestrial carbon cycle and monitor food security, vegetation growth must be accurately predicted; however, current process-based ecosystem and crop-growth models are limited in their effectiveness. This study developed a machine learning model using the extreme gradient boosting method to predict vegetation growth throughout the growing season in China from 2001 to 2018. The model used satellite-derived vegetation data for the first month of each growing season, CO2 concentration, and several meteorological factors as data sources for the explanatory variables. Results showed that the model could reproduce the spatiotemporal distribution of vegetation growth as represented by the satellite-derived normalized difference vegetation index (NDVI). The predictive error for the growing season NDVI was less than 5% for more than 98% of vegetated areas in China; the model represented seasonal variations in NDVI well. The coefficient of determination (R2) between the monthly observed and predicted NDVI was 0.83, and more than 69% of vegetated areas had an R2 > 0.8. The effectiveness of the model was examined for a severe drought year (2009), and results showed that the model could reproduce the spatiotemporal distribution of NDVI even under extreme conditions. This model provides an alternative method for predicting vegetation growth and has great potential for monitoring vegetation dynamics and crop growth.

scholarly journals Proposing a machine-learning based method to predict stillbirth before and during delivery and ranking the features: nationwide retrospective cross-sectional study

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Toktam Khatibi ◽  
Elham Hanifi ◽  
Mohammad Mehdi Sepehri ◽  
Leila Allahqoli
Keyword(s):  
Machine Learning ◽  
External Validation ◽  
Fetal Loss ◽  
Null Distribution ◽  
Training Dataset ◽  
Gradient Boosting ◽  
Support Vector ◽  
Cross Sectional ◽  
Boosting Method ◽  
Demographic Features

Abstract Background Stillbirth is defined as fetal loss in pregnancy beyond 28 weeks by WHO. In this study, a machine-learning based method is proposed to predict stillbirth from livebirth and discriminate stillbirth before and during delivery and rank the features. Method A two-step stack ensemble classifier is proposed for classifying the instances into stillbirth and livebirth at the first step and then, classifying stillbirth before delivery from stillbirth during the labor at the second step. The proposed SE has two consecutive layers including the same classifiers. The base classifiers in each layer are decision tree, Gradient boosting classifier, logistics regression, random forest and support vector machines which are trained independently and aggregated based on Vote boosting method. Moreover, a new feature ranking method is proposed in this study based on mean decrease accuracy, Gini Index and model coefficients to find high-ranked features. Results IMAN registry dataset is used in this study considering all births at or beyond 28th gestational week from 2016/04/01 to 2017/01/01 including 1,415,623 live birth and 5502 stillbirth cases. A combination of maternal demographic features, clinical history, fetal properties, delivery descriptors, environmental features, healthcare service provider descriptors and socio-demographic features are considered. The experimental results show that our proposed SE outperforms the compared classifiers with the average accuracy of 90%, sensitivity of 91%, specificity of 88%. The discrimination of the proposed SE is assessed and the average AUC of ±95%, CI of 90.51% ±1.08 and 90% ±1.12 is obtained on training dataset for model development and test dataset for external validation, respectively. The proposed SE is calibrated using isotopic nonparametric calibration method with the score of 0.07. The process is repeated 10,000 times and AUC of SE classifiers using random different training datasets as null distribution. The obtained p-value to assess the specificity of the proposed SE is 0.0126 which shows the significance of the proposed SE. Conclusions Gestational age and fetal height are two most important features for discriminating livebirth from stillbirth. Moreover, hospital, province, delivery main cause, perinatal abnormality, miscarriage number and maternal age are the most important features for classifying stillbirth before and during delivery.

scholarly journals Comparison of Ensemble Machine Learning Methods for Soil Erosion Pin Measurements

2021 ◽  
Vol 10 (1) ◽  
pp. 42
Author(s):  
Kieu Anh Nguyen ◽  
Walter Chen ◽  
Bor-Shiun Lin ◽  
Uma Seeboonruang
Keyword(s):  
Machine Learning ◽  
Soil Erosion ◽  
Ensemble Methods ◽  
Machine Learning Algorithms ◽  
Multivariate Adaptive Regression Splines ◽  
Gradient Boosting ◽  
Support Vector ◽  
Ensemble Machine Learning ◽  
Boosting Method ◽  
Bagging Method

Although machine learning has been extensively used in various fields, it has only recently been applied to soil erosion pin modeling. To improve upon previous methods of quantifying soil erosion based on erosion pin measurements, this study explored the possible application of ensemble machine learning algorithms to the Shihmen Reservoir watershed in northern Taiwan. Three categories of ensemble methods were considered in this study: (a) Bagging, (b) boosting, and (c) stacking. The bagging method in this study refers to bagged multivariate adaptive regression splines (bagged MARS) and random forest (RF), and the boosting method includes Cubist and gradient boosting machine (GBM). Finally, the stacking method is an ensemble method that uses a meta-model to combine the predictions of base models. This study used RF and GBM as the meta-models, decision tree, linear regression, artificial neural network, and support vector machine as the base models. The dataset used in this study was sampled using stratified random sampling to achieve a 70/30 split for the training and test data, and the process was repeated three times. The performance of six ensemble methods in three categories was analyzed based on the average of three attempts. It was found that GBM performed the best among the ensemble models with the lowest root-mean-square error (RMSE = 1.72 mm/year), the highest Nash-Sutcliffe efficiency (NSE = 0.54), and the highest index of agreement (d = 0.81). This result was confirmed by the spatial comparison of the absolute differences (errors) between model predictions and observations using GBM and RF in the study area. In summary, the results show that as a group, the bagging method and the boosting method performed equally well, and the stacking method was third for the erosion pin dataset considered in this study.

Predictive Credit Risk Analytics Using Borrowers' Digital Footprint and Methods of Statistical Machine Learning

2021 ◽  
Vol 12 (7) ◽  
pp. 358-372
Author(s):  
E. V. Orlova ◽  
Keyword(s):  
Machine Learning ◽  
Credit Risk ◽  
Risk Profile ◽  
Classification Model ◽  
Gradient Boosting ◽  
Suggested Approach ◽  
Digital Footprint ◽  
Credit Risks ◽  
Stochastic Gradient Boosting ◽  
Boosting Method

The article considers the problem of reducing the banks credit risks associated with the insolvency of borrowers — individuals using financial, socio-economic factors and additional data about borrowers digital footprint. A critical analysis of existing approaches, methods and models in this area has been carried out and a number of significant shortcomings identified that limit their application. There is no comprehensive approach to identifying a borrowers creditworthiness based on information, including data from social networks and search engines. The new methodological approach for assessing the borrowers risk profile based on the phased processing of quantitative and qualitative data and modeling using methods of statistical analysis and machine learning is proposed. Machine learning methods are supposed to solve clustering and classification problems. They allow to automatically determine the data structure and make decisions through flexible and local training on the data. The method of hierarchical clustering and the k-means method are used to identify similar social, anthropometric and financial indicators, as well as indicators characterizing the digital footprint of borrowers, and to determine the borrowers risk profile over group. The obtained homogeneous groups of borrowers with a unique risk profile are further used for detailed data analysis in the predictive classification model. The classification model is based on the stochastic gradient boosting method to predict the risk profile of a potencial borrower. The suggested approach for individuals creditworthiness assessing will reduce the banks credit risks, increase its stability and profitability. The implementation results are of practical importance. Comparative analysis of the effectiveness of the existing and the proposed methodology for assessing credit risk showed that the new methodology provides predictive ana­lytics of heterogeneous information about a potential borrower and the accuracy of analytics is higher. The proposed techniques are the core for the decision support system for justification of individuals credit conditions, minimizing the aggregate credit risks.

Bayesian Optimization based Gradient Boosting Method of Fault Detection in Oil-Immersed Transformer and Reactors

2021 ◽  
pp. 1-1
Author(s):  
Devaprasad Paul ◽  
Arup Kumar Goswami ◽  
Rahul Lamichane Chetri ◽  
Rajesh Roy ◽  
Pritin Sen
Keyword(s):  
Fault Detection ◽  
Bayesian Optimization ◽  
Gradient Boosting ◽  
Boosting Method

scholarly journals Development and External Evaluation of Predictions Models for Mortality of COVID-19 Patients using Machine Learning Method

2020 ◽  
Author(s):  
Simin Li ◽  
Yulan Lin ◽  
Tong Zhu ◽  
Mengjie Fan ◽  
Shicheng Xu ◽  
...  
Keyword(s):  
Machine Learning ◽  
Predictive Value ◽  
Final Analysis ◽  
Mortality Prediction ◽  
Prediction Performance ◽  
Gradient Boosting ◽  
Machine Learning Method ◽  
External Evaluation ◽  
Machine Learning Model ◽  
Moderate Case

Abstract Background To develop and evaluate the prognostic machine-learning model for mortality in patients with coronavirus disease 2019 (COVID-19).Methods Clinical data of confirmed COVID-19 were retrospectively collected from Wuhan between 18th January and 29th March 2020. Gradient Boosting Decision Tree (GBDT), logistic regression (LR) model, and simplified LR with selected 5 features (LR-5) model were built to predict the mortality of COVID-19. 5-fold area under curve (AUC), accuracy, positive predictive value (PPV), and negative predictive value (NPV) were calculated and compared between models.Results A total of 2,924 patients were included in the final analysis, 257(8.8%) of whom died during hospitalization and 2,667 (91.2%) survived. There were 21(0.7%) mild cases, 2,051(70.1%) moderate case, 779(26.6%) severe cases, and 73(2.5%) critically severe cases of COVID-19 on admission. The overall 5-fold AUC was observed highest in GBDT model (0.941), followed by LR (0.928) and LR-5 (0.913). The diagnostic accuracy were 0.889 in GBDT, 0.868 in LR and 0.887 in LR-5. GBDT model also showed the highest sensitivity (0.899) and speciality (0.889). The NPV of all three models exceeded 97%, while the PPV were relatively low in all models, 0.381 for LR, 0.402 for LR-5 and 0.432 for GBDT. In subgroups analysis with severe cases only, GBDT model also performed the best with a accuracy of 0.799 and 5-fold AUC (0.918).Conclusion The finding revealed that mortality prediction performance of the GBDT was superior to the LR models in confirmed cases of COVID-19, regardless of disease severity.

scholarly journals Supporting an Expert-centric Process of New Product Introduction With Statistical Machine Learning

2021 ◽  
pp. 187-198
Author(s):  
Shima Zahmatkesh ◽  
Alessio Bernardo ◽  
Emanuele Falzone ◽  
Edgardo Di Nicola Carena ◽  
Emanuele Della Valle
Keyword(s):  
Machine Learning ◽  
New Product ◽  
Parameter Tuning ◽  
Life Cycles ◽  
New Product Introduction ◽  
Gradient Boosting ◽  
Statistical Machine Learning ◽  
Product Introduction ◽  
Better Than

Industries that sell products with short-term or seasonal life cycles must regularly introduce new products. Forecasting the demand for New Product Introduction (NPI) can be challenging due to the fluctuations of many factors such as trend, seasonality, or other external and unpredictable phenomena (e.g., COVID-19 pandemic). Traditionally, NPI is an expertcentric process. This paper presents a study on automating the forecast of NPI demands using statistical Machine Learning (namely, Gradient Boosting and XGBoost). We show how to overcome shortcomings of the traditional data preparation that underpins the manual process. Moreover, we illustrate the role of cross-validation techniques for the hyper-parameter tuning and the validation of the models. Finally, we provide empirical evidence that statistical Machine Learning can forecast NPI demand better than experts.

Deep Learning versus Conventional Machine Learning for Detection of Healthcare-Associated Infections in French Clinical Narratives

2019 ◽  
Vol 58 (01) ◽  
pp. 031-041 ◽  
Author(s):  
Sara Rabhi ◽  
Jérémie Jakubowicz ◽  
Marie-Helene Metzger
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Characteristic Curve ◽  
Bayesian Optimization ◽  
Gradient Boosting ◽  
Support Vector ◽  
University Hospitals ◽  
Infection Control Practitioner ◽  
Medical Reports ◽  
Conventional Machine

Objective The objective of this article was to compare the performances of health care-associated infection (HAI) detection between deep learning and conventional machine learning (ML) methods in French medical reports. Methods The corpus consisted in different types of medical reports (discharge summaries, surgery reports, consultation reports, etc.). A total of 1,531 medical text documents were extracted and deidentified in three French university hospitals. Each of them was labeled as presence (1) or absence (0) of HAI. We started by normalizing the records using a list of preprocessing techniques. We calculated an overall performance metric, the F1 Score, to compare a deep learning method (convolutional neural network [CNN]) with the most popular conventional ML models (Bernoulli and multi-naïve Bayes, k-nearest neighbors, logistic regression, random forests, extra-trees, gradient boosting, support vector machines). We applied the hyperparameter Bayesian optimization for each model based on its HAI identification performances. We included the set of text representation as an additional hyperparameter for each model, using four different text representations (bag of words, term frequency–inverse document frequency, word2vec, and Glove). Results CNN outperforms all other conventional ML algorithms for HAI classification. The best F1 Score of 97.7% ± 3.6% and best area under the curve score of 99.8% ± 0.41% were achieved when CNN was directly applied to the processed clinical notes without a pretrained word2vec embedding. Through receiver operating characteristic curve analysis, we could achieve a good balance between false notifications (with a specificity equal to 0.937) and system detection capability (with a sensitivity equal to 0.962) using the Youden's index reference. Conclusions The main drawback of CNNs is their opacity. To address this issue, we investigated CNN inner layers' activation values to visualize the most meaningful phrases in a document. This method could be used to build a phrase-based medical assistant algorithm to help the infection control practitioner to select relevant medical records. Our study demonstrated that deep learning approach outperforms other classification learning algorithms for automatically identifying HAIs in medical reports.

Can machine learning consistently improve the scoring power of classical scoring functions? Insights into the role of machine learning in scoring functions

2020 ◽  
Author(s):  
Chao Shen ◽  
Ye Hu ◽  
Zhe Wang ◽  
Xujun Zhang ◽  
Haiyang Zhong ◽  
...  
Keyword(s):  
Machine Learning ◽  
Biological Activities ◽  
Gradient Boosting ◽  
Linear Regression Method ◽  
Scoring Functions ◽  
Binding Affinities ◽  
Training Set ◽  
Sequence Similarities ◽  
Better Than

Abstract How to accurately estimate protein–ligand binding affinity remains a key challenge in computer-aided drug design (CADD). In many cases, it has been shown that the binding affinities predicted by classical scoring functions (SFs) cannot correlate well with experimentally measured biological activities. In the past few years, machine learning (ML)-based SFs have gradually emerged as potential alternatives and outperformed classical SFs in a series of studies. In this study, to better recognize the potential of classical SFs, we have conducted a comparative assessment of 25 commonly used SFs. Accordingly, the scoring power was systematically estimated by using the state-of-the-art ML methods that replaced the original multiple linear regression method to refit individual energy terms. The results show that the newly-developed ML-based SFs consistently performed better than classical ones. In particular, gradient boosting decision tree (GBDT) and random forest (RF) achieved the best predictions in most cases. The newly-developed ML-based SFs were also tested on another benchmark modified from PDBbind v2007, and the impacts of structural and sequence similarities were evaluated. The results indicated that the superiority of the ML-based SFs could be fully guaranteed when sufficient similar targets were contained in the training set. Moreover, the effect of the combinations of features from multiple SFs was explored, and the results indicated that combining NNscore2.0 with one to four other classical SFs could yield the best scoring power. However, it was not applicable to derive a generic target-specific SF or SF combination.

Machine Learning Reactivity in the Chemical Space Surrounding Vaska's Complex

2019 ◽  
Author(s):  
Pascal Friederich ◽  
Gabriel dos Passos Gomes ◽  
Riccardo De Bin ◽  
Alan Aspuru-Guzik ◽  
David Balcells
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Chemical Space ◽  
Bayesian Optimization ◽  
Gradient Boosting ◽  
Learning Models ◽  
Hydrogen Activation ◽  
Activation Barriers ◽  
Machine Learning Models ◽  
Vaska's Complex

Machine learning models, including neural networks, Bayesian optimization, gradient boosting and Gaussian processes, were trained with DFT data for the accurate, affordable and explainable prediction of hydrogen activation barriers in the chemical space surrounding Vaska's complex.

Sign in / Sign up

Export Citation Format

Share Document