Kernel and Acquisition Function Setup for Bayesian Optimization of Gradient Boosting Hyperparameters

2018 ◽  
pp. 297-306 ◽  
Author(s):  
Andrzej Szwabe
Keyword(s):  
Bayesian Optimization ◽  
Gradient Boosting

scholarly journals Passive Fetal Movement Recognition Approaches Using Hyperparameter Tuned LightGBM Model and Bayesian Optimization

2021 ◽  
Vol 2021 ◽  
pp. 1-18
Author(s):  
Sensong Liang ◽  
Jiansheng Peng ◽  
Yong Xu ◽  
Hemin Ye
Keyword(s):  
Frequency Domain ◽  
Kalman Filtering ◽  
Health Monitoring ◽  
Fetal Movement ◽  
Bayesian Optimization ◽  
Gradient Boosting ◽  
Wavelet Domain ◽  
Prenatal Health ◽  
Light Gradient ◽  
Low Amplitude

Fetal movement is an important clinical indicator to assess fetus growth and development status in the uterus. In recent years, a noninvasive intelligent sensing fetal movement detection system that can monitor high-risk pregnancies at home has received a lot of attention in the field of wearable health monitoring. However, recovering fetal movement signals from a continuous low-amplitude background that is heavily contaminated with noise and recognizing real fetal movements is a challenging task. In this paper, fetal movement can be efficiently recognized by combining the strength of Kalman filtering, time and frequency domain and wavelet domain feature extraction, and hyperparameter tuned Light Gradient Boosting Machine (LightGBM) model. Firstly, the Kalman filtering (KF) algorithm is used to recover the fetal movement signal in a continuous low-amplitude background contaminated by noise. Secondly, the time domain, frequency domain, and wavelet domain (TFWD) features of the preprocessed fetal movement signal are extracted. Finally, the Bayesian Optimization algorithm (BOA) is used to optimize the LightGBM model to obtain the optimal hyperparameters. Through this, the accurate prediction and recognition of fetal movement are successfully achieved. In the performance analysis of the Zenodo fetal movement dataset, the proposed KF + TFWD + BOA-LGBM approach’s recognition accuracy and F1-Score reached 94.06% and 96.85%, respectively. Compared with 8 existing advanced methods for fetal movement signal recognition, the proposed method has better accuracy and robustness, indicating its potential medical application in wearable smart sensing systems for fetal prenatal health monitoring.

Bayesian Optimization based Gradient Boosting Method of Fault Detection in Oil-Immersed Transformer and Reactors

2021 ◽  
pp. 1-1
Author(s):  
Devaprasad Paul ◽  
Arup Kumar Goswami ◽  
Rahul Lamichane Chetri ◽  
Rajesh Roy ◽  
Pritin Sen
Keyword(s):  
Fault Detection ◽  
Bayesian Optimization ◽  
Gradient Boosting ◽  
Boosting Method

scholarly journals Prediction of Train Arrival Delay Using Hybrid ELM-PSO Approach

2021 ◽  
Vol 2021 ◽  
pp. 1-15
Author(s):  
Xu Bao ◽  
Yanqiu Li ◽  
Jianmin Li ◽  
Rui Shi ◽  
Xin Ding
Keyword(s):  
Nearest Neighbor ◽  
Pso Algorithm ◽  
Bayesian Optimization ◽  
Gradient Boosting ◽  
K Nearest Neighbor ◽  
Delay Management ◽  
Buffer Time ◽  
Proposed Model ◽  
Learning Machine ◽  
Hidden Layer

In this study, a hybrid method combining extreme learning machine (ELM) and particle swarm optimization (PSO) is proposed to forecast train arrival delays that can be used for later delay management and timetable optimization. First, nine characteristics (e.g., buffer time, the train number, and station code) associated with train arrival delays are chosen and analyzed using extra trees classifier. Next, an ELM with one hidden layer is developed to predict train arrival delays by considering these characteristics mentioned before as input features. Furthermore, the PSO algorithm is chosen to optimize the hyperparameter of the ELM compared to Bayesian optimization and genetic algorithm solving the arduousness problem of manual regulating. Finally, a case is studied to confirm the advantage of the proposed model. Contrasted to four baseline models (k-nearest neighbor, categorical boosting, Lasso, and gradient boosting decision tree) across different metrics, the proposed model is demonstrated to be proficient and achieve the highest prediction accuracy. In addition, through a detailed analysis of the prediction error, it is found that our model possesses good robustness and correctness.

Deep Learning versus Conventional Machine Learning for Detection of Healthcare-Associated Infections in French Clinical Narratives

2019 ◽  
Vol 58 (01) ◽  
pp. 031-041 ◽  
Author(s):  
Sara Rabhi ◽  
Jérémie Jakubowicz ◽  
Marie-Helene Metzger
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Characteristic Curve ◽  
Bayesian Optimization ◽  
Gradient Boosting ◽  
Support Vector ◽  
University Hospitals ◽  
Infection Control Practitioner ◽  
Medical Reports ◽  
Conventional Machine

Objective The objective of this article was to compare the performances of health care-associated infection (HAI) detection between deep learning and conventional machine learning (ML) methods in French medical reports. Methods The corpus consisted in different types of medical reports (discharge summaries, surgery reports, consultation reports, etc.). A total of 1,531 medical text documents were extracted and deidentified in three French university hospitals. Each of them was labeled as presence (1) or absence (0) of HAI. We started by normalizing the records using a list of preprocessing techniques. We calculated an overall performance metric, the F1 Score, to compare a deep learning method (convolutional neural network [CNN]) with the most popular conventional ML models (Bernoulli and multi-naïve Bayes, k-nearest neighbors, logistic regression, random forests, extra-trees, gradient boosting, support vector machines). We applied the hyperparameter Bayesian optimization for each model based on its HAI identification performances. We included the set of text representation as an additional hyperparameter for each model, using four different text representations (bag of words, term frequency–inverse document frequency, word2vec, and Glove). Results CNN outperforms all other conventional ML algorithms for HAI classification. The best F1 Score of 97.7% ± 3.6% and best area under the curve score of 99.8% ± 0.41% were achieved when CNN was directly applied to the processed clinical notes without a pretrained word2vec embedding. Through receiver operating characteristic curve analysis, we could achieve a good balance between false notifications (with a specificity equal to 0.937) and system detection capability (with a sensitivity equal to 0.962) using the Youden's index reference. Conclusions The main drawback of CNNs is their opacity. To address this issue, we investigated CNN inner layers' activation values to visualize the most meaningful phrases in a document. This method could be used to build a phrase-based medical assistant algorithm to help the infection control practitioner to select relevant medical records. Our study demonstrated that deep learning approach outperforms other classification learning algorithms for automatically identifying HAIs in medical reports.

Machine Learning Reactivity in the Chemical Space Surrounding Vaska's Complex

2019 ◽  
Author(s):  
Pascal Friederich ◽  
Gabriel dos Passos Gomes ◽  
Riccardo De Bin ◽  
Alan Aspuru-Guzik ◽  
David Balcells
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Chemical Space ◽  
Bayesian Optimization ◽  
Gradient Boosting ◽  
Learning Models ◽  
Hydrogen Activation ◽  
Activation Barriers ◽  
Machine Learning Models ◽  
Vaska's Complex

Machine learning models, including neural networks, Bayesian optimization, gradient boosting and Gaussian processes, were trained with DFT data for the accurate, affordable and explainable prediction of hydrogen activation barriers in the chemical space surrounding Vaska's complex.

scholarly journals Applying Bayesian Optimization for Machine Learning Models in Predicting the Surface Roughness in Single-Point Diamond Turning Polycarbonate

2021 ◽  
Vol 2021 ◽  
pp. 1-16
Author(s):  
Van-Hai Nguyen ◽  
Tien-Thinh Le ◽  
Hoanh-Son Truong ◽  
Minh Vuong Le ◽  
Van-Luc Ngo ◽  
...  
Keyword(s):  
Machine Learning ◽  
Surface Roughness ◽  
Surface Quality ◽  
Feed Rate ◽  
Diamond Turning ◽  
Bayesian Optimization ◽  
Gradient Boosting ◽  
Significant Feature ◽  
Learning Models ◽  
Machine Learning Models

This paper deals with the prediction of surface roughness in manufacturing polycarbonate (PC) by applying Bayesian optimization for machine learning models. The input variables of ultraprecision turning—namely, feed rate, depth of cut, spindle speed, and vibration of the X-, Y-, and Z-axis—are the main factors affecting surface quality. In this research, six machine learning- (ML-) based models—artificial neural network (ANN), Cat Boost Regression (CAT), Support Vector Machine (SVR), Gradient Boosting Regression (GBR), Decision Tree Regression (DTR), and Extreme Gradient Boosting Regression (XGB)—were applied to predict the surface roughness (Ra). The predictive performance of the baseline models was quantitatively assessed through error metrics: root means square error (RMSE), mean absolute error (MAE), and coefficient of determination (R2). The overall results indicate that the XGB and CAT models predict Ra with the greatest accuracy. In improving baseline models such as XGB and CAT, the Bayesian optimization (BO) is next used to determine their best hyperparameters, and the results indicate that XGB is the best model according to the evaluation metrics. Results have shown that the performance of the models has been improved significantly with BO. For example, the values of RMSE and MAE of XGB have decreased from 0.0076 to 0.0047 and from 0.0063 to 0.0027, respectively, for the training dataset. Using the testing dataset, the values of RMSE and MAE of XGB have decreased from 0.4033 to 0.2512 and from 0.2845 to 0.2225, respectively. Moreover, the vibrations of the X, Y, and Z axes and feed rate are the most significant feature in predicting the results, which is in high accordance with the literature. We find that, in a specified value domain, the vibration of the axes has a greater influence on the surface quality than does the cutting condition.

Machine Learning Reactivity in the Chemical Space Surrounding Vaska's Complex

2019 ◽  
Author(s):  
Pascal Friederich ◽  
Gabriel dos Passos Gomes ◽  
Riccardo De Bin ◽  
Alan Aspuru-Guzik ◽  
David Balcells
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Chemical Space ◽  
Bayesian Optimization ◽  
Gradient Boosting ◽  
Learning Models ◽  
Hydrogen Activation ◽  
Activation Barriers ◽  
Machine Learning Models ◽  
Vaska's Complex

Machine learning models, including neural networks, Bayesian optimization, gradient boosting and Gaussian processes, were trained with DFT data for the accurate, affordable and explainable prediction of hydrogen activation barriers in the chemical space surrounding Vaska's complex.

scholarly journals Total Organic Carbon Content Prediction in Lacustrine Shale Using Extreme Gradient Boosting Machine Learning Based on Bayesian Optimization

Geofluids ◽  
2021 ◽  
Vol 2021 ◽  
pp. 1-18
Author(s):  
Xingzhou Liu ◽  
Zhi Tian ◽  
Chang Chen
Keyword(s):  
Machine Learning ◽  
Organic Carbon ◽  
Total Organic Carbon ◽  
Bayesian Optimization ◽  
Gradient Boosting ◽  
Support Vector ◽  
Shale Oil ◽  
The Core ◽  
Extreme Gradient Boosting ◽  
Lacustrine Shale

The total organic carbon (TOC) content is a critical parameter for estimating shale oil resources. However, common TOC prediction methods rely on empirical formulas, and their applicability varies widely from region to region. In this study, a novel data-driven Bayesian optimization extreme gradient boosting (XGBoost) model was proposed to predict the TOC content using wireline log data. The lacustrine shale in the Damintun Sag, Bohai Bay Basin, China, was used as a case study. Firstly, correlation analysis was used to analyze the relationship between the well logs and the core-measured TOC data. Based on the degree of correlation, six logging curves reflecting TOC content were selected to construct training dataset for machine learning. Then, the performance of the XGBoost model was tested using K -fold cross-validation, and the hyperparameters of the model were determined using a Bayesian optimization method to improve the search efficiency and reduce the uncertainty caused by the rule of thumb. Next, through the analysis of prediction errors, the coefficient of determination ( R 2 ) of the TOC content predicted by the XGBoost model and the core-measured TOC content reached 0.9135. The root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE) were 0.63, 0.77, and 12.55%, respectively. In addition, five commonly used methods, namely, Δ log R method, random forest, support vector machine, K -nearest neighbors, and multiple linear regression, were used to predict the TOC content to confirm that the XGBoost model has higher prediction accuracy and better robustness. Finally, the proposed approach was applied to predict the TOC curves of 20 exploration wells in the Damintun Sag. We obtained quantitative contour maps of the TOC content of this block for the first time. The results of this study facilitate the rapid detection of the sweet spots of the lacustrine shale oil.

Sign in / Sign up

Export Citation Format

Share Document