scholarly journals HyP-ABC: A Novel Automated Hyper-Parameter Tuning Algorithm Using Evolutionary Optimization

2021 ◽  
Author(s):  
Leila Zahedi ◽  
Farid Ghareh Mohammadi ◽  
M. Hadi Amini
Keyword(s):  
Real World ◽  
Large Scale ◽  
Convergence Rates ◽  
Parameter Tuning ◽  
Population Based ◽  
Machine Learning Techniques ◽  
Gradient Boosting ◽  
Support Vector ◽  
Wide Range ◽  
Extreme Gradient Boosting

<p>Machine learning techniques lend themselves as promising decision-making and analytic tools in a wide range of applications. Different ML algorithms have various hyper-parameters. In order to tailor an ML model towards a specific application working at its best, its hyper-parameters should be tuned. Tuning the hyper-parameters directly affects the performance. However, for large-scale search spaces, efficiently exploring the ample number of combinations of hyper-parameters is computationally expensive. Many of the automated hyper-parameter tuning techniques suffer from low convergence rates and high experimental time complexities. In this paper, we propose HyP-ABC, an automatic innovative hybrid hyper-parameter optimization algorithm using the modified artificial bee colony approach, to measure the classification accuracy of three ML algorithms: random forest, extreme gradient boosting, and support vector machine. In order to ensure the robustness of the proposed method, the algorithm takes a wide range of feasible hyper-parameter values and is tested using a real-world educational dataset. Experimental results show that HyP-ABC is competitive with state-of-the-art techniques. Also, it has fewer hyper-parameters to be tuned than other population-based algorithms, making it worthwhile for real-world HPO problems.</p>

scholarly journals HyP-ABC: A Novel Automated Hyper-Parameter Tuning Algorithm Using Evolutionary Optimization

2021 ◽  
Author(s):  
Leila Zahedi ◽  
Farid Ghareh Mohammadi ◽  
M. Hadi Amini
Keyword(s):  
Parameter Optimization ◽  
Real World ◽  
Optimization Problems ◽  
State Of The Art ◽  
Parameter Tuning ◽  
Gradient Boosting ◽  
Support Vector ◽  
Wide Range ◽  
Extreme Gradient Boosting ◽  
Art Techniques

Machine learning techniques lend themselves as promising decision-making and analytic tools in a wide range of applications. Different ML algorithms have various hyper-parameters. In order to tailor an ML model towards a specific application, a large number of hyper-parameters should be tuned. Tuning the hyper-parameters directly affects the performance (accuracy and run-time). However, for large-scale search spaces, efficiently exploring the ample number of combinations of hyper-parameters is computationally challenging. Existing automated hyper-parameter tuning techniques suffer from high time complexity. In this paper, we propose HyP-ABC, an automatic innovative hybrid hyper-parameter optimization algorithm using the modified artificial bee colony approach, to measure the classification accuracy of three ML algorithms, namely random forest, extreme gradient boosting, and support vector machine. Compared to the state-of-the-art techniques, HyP-ABC is more efficient and has a limited number of parameters to be tuned, making it worthwhile for real-world hyper-parameter optimization problems. We further compare our proposed HyP-ABC algorithm with state-of-the-art techniques. In order to ensure the robustness of the proposed method, the algorithm takes a wide range of feasible hyper-parameter values, and is tested using a real-world educational dataset.

scholarly journals Machine learning models to identify low adherence to influenza vaccination among Korean adults with cardiovascular disease

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Moojung Kim ◽  
Young Jae Kim ◽  
Sung Jin Park ◽  
Kwang Gi Kim ◽  
Pyung Chun Oh ◽  
...  
Keyword(s):  
Machine Learning ◽  
Cardiovascular Disease ◽  
Influenza Vaccination ◽  
Machine Learning Techniques ◽  
Gradient Boosting ◽  
Support Vector ◽  
Age Group ◽  
Learning Models ◽  
Extreme Gradient Boosting ◽  
Machine Learning Models

Abstract Background Annual influenza vaccination is an important public health measure to prevent influenza infections and is strongly recommended for cardiovascular disease (CVD) patients, especially in the current coronavirus disease 2019 (COVID-19) pandemic. The aim of this study is to develop a machine learning model to identify Korean adult CVD patients with low adherence to influenza vaccination Methods Adults with CVD (n = 815) from a nationally representative dataset of the Fifth Korea National Health and Nutrition Examination Survey (KNHANES V) were analyzed. Among these adults, 500 (61.4%) had answered "yes" to whether they had received seasonal influenza vaccinations in the past 12 months. The classification process was performed using the logistic regression (LR), random forest (RF), support vector machine (SVM), and extreme gradient boosting (XGB) machine learning techniques. Because the Ministry of Health and Welfare in Korea offers free influenza immunization for the elderly, separate models were developed for the < 65 and ≥ 65 age groups. Results The accuracy of machine learning models using 16 variables as predictors of low influenza vaccination adherence was compared; for the ≥ 65 age group, XGB (84.7%) and RF (84.7%) have the best accuracies, followed by LR (82.7%) and SVM (77.6%). For the < 65 age group, SVM has the best accuracy (68.4%), followed by RF (64.9%), LR (63.2%), and XGB (61.4%). Conclusions The machine leaning models show comparable performance in classifying adult CVD patients with low adherence to influenza vaccination.

scholarly journals DeepVF: a deep learning-based hybrid framework for identifying virulence factors using the stacking strategy

2020 ◽  
Author(s):  
Ruopeng Xie ◽  
Jiahui Li ◽  
Jiawei Wang ◽  
Wei Dai ◽  
André Leier ◽  
...  
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Virulence Factors ◽  
Bacterial Genome ◽  
Machine Learning Algorithms ◽  
Gradient Boosting ◽  
Support Vector ◽  
Wide Range ◽  
Extreme Gradient Boosting ◽  
Hybrid Framework

Abstract Virulence factors (VFs) enable pathogens to infect their hosts. A wealth of individual, disease-focused studies has identified a wide variety of VFs, and the growing mass of bacterial genome sequence data provides an opportunity for computational methods aimed at predicting VFs. Despite their attractive advantages and performance improvements, the existing methods have some limitations and drawbacks. Firstly, as the characteristics and mechanisms of VFs are continually evolving with the emergence of antibiotic resistance, it is more and more difficult to identify novel VFs using existing tools that were previously developed based on the outdated data sets; secondly, few systematic feature engineering efforts have been made to examine the utility of different types of features for model performances, as the majority of tools only focused on extracting very few types of features. By addressing the aforementioned issues, the accuracy of VF predictors can likely be significantly improved. This, in turn, would be particularly useful in the context of genome wide predictions of VFs. In this work, we present a deep learning (DL)-based hybrid framework (termed DeepVF) that is utilizing the stacking strategy to achieve more accurate identification of VFs. Using an enlarged, up-to-date dataset, DeepVF comprehensively explores a wide range of heterogeneous features with popular machine learning algorithms. Specifically, four classical algorithms, including random forest, support vector machines, extreme gradient boosting and multilayer perceptron, and three DL algorithms, including convolutional neural networks, long short-term memory networks and deep neural networks are employed to train 62 baseline models using these features. In order to integrate their individual strengths, DeepVF effectively combines these baseline models to construct the final meta model using the stacking strategy. Extensive benchmarking experiments demonstrate the effectiveness of DeepVF: it achieves a more accurate and stable performance compared with baseline models on the benchmark dataset and clearly outperforms state-of-the-art VF predictors on the independent test. Using the proposed hybrid ensemble model, a user-friendly online predictor of DeepVF (http://deepvf.erc.monash.edu/) is implemented. Furthermore, its utility, from the user’s viewpoint, is compared with that of existing toolkits. We believe that DeepVF will be exploited as a useful tool for screening and identifying potential VFs from protein-coding gene sequences in bacterial genomes.

scholarly journals Reliable photometric membership (RPM) of galaxies in clusters – I. A machine learning method and its performance in the local universe

2020 ◽  
Vol 493 (3) ◽  
pp. 3429-3441
Author(s):  
Paulo A A Lopes ◽  
André L B Ribeiro
Keyword(s):  
Machine Learning ◽  
Galaxy Evolution ◽  
Large Scale ◽  
Machine Learning Techniques ◽  
Gradient Boosting ◽  
Support Vector ◽  
Validation Data ◽  
Membership Probability ◽  
Cluster Membership ◽  
Stochastic Gradient Boosting

ABSTRACT We introduce a new method to determine galaxy cluster membership based solely on photometric properties. We adopt a machine learning approach to recover a cluster membership probability from galaxy photometric parameters and finally derive a membership classification. After testing several machine learning techniques (such as stochastic gradient boosting, model averaged neural network and k-nearest neighbours), we found the support vector machine algorithm to perform better when applied to our data. Our training and validation data are from the Sloan Digital Sky Survey main sample. Hence, to be complete to $M_r^* + 3$, we limit our work to 30 clusters with $z$phot-cl ≤ 0.045. Masses (M200) are larger than $\sim 0.6\times 10^{14} \, \mathrm{M}_{\odot }$ (most above $3\times 10^{14} \, \mathrm{M}_{\odot }$). Our results are derived taking in account all galaxies in the line of sight of each cluster, with no photometric redshift cuts or background corrections. Our method is non-parametric, making no assumptions on the number density or luminosity profiles of galaxies in clusters. Our approach delivers extremely accurate results (completeness, C $\sim 92{\rm{ per\ cent}}$ and purity, P $\sim 87{\rm{ per\ cent}}$) within R200, so that we named our code reliable photometric membership. We discuss possible dependencies on magnitude, colour, and cluster mass. Finally, we present some applications of our method, stressing its impact to galaxy evolution and cosmological studies based on future large-scale surveys, such as eROSITA, EUCLID, and LSST.

scholarly journals Predicting synergism of cancer drug combinations using NCI-ALMANAC data

2018 ◽  
Author(s):  
Pavel Sidorov ◽  
Stefan Naulaerts ◽  
Jérémy Ariey-Bonnet ◽  
Eddy Pasquier ◽  
Pedro J. Ballester
Keyword(s):  
Cell Line ◽  
In Silico ◽  
Large Scale ◽  
Alkylating Agents ◽  
Drug Combinations ◽  
Reliability Estimation ◽  
Machine Learning Techniques ◽  
Cancer Drug ◽  
Gradient Boosting ◽  
Extreme Gradient Boosting

AbstractBackgroundDrug combinations are of great interest for cancer treatment. Unfortunately, the discovery of synergistic combinations by purely experimental means is only feasible on small sets of drugs.In silicomodeling methods can substantially widen this search by providing tools able to predict which of all possible combinations in a large compound library are synergistic. Here we investigate to which extent drug combination synergy can be predicted by exploiting the largest available dataset to date (NCI-ALMANAC, with over 290,000 synergy determinations).MethodsEach cell line is modeled using primarily two machine learning techniques, Random Forest (RF) and Extreme Gradient Boosting (XGBoost), on the datasets provided by NCI-ALMANAC. This large-scale predictive modeling study comprises more than 5000 pair-wise drug combinations, 60 cell lines, 4 types of models and 5 types of chemical features. The application of a powerful, yet uncommonly used, RF-specific technique for reliability prediction is also investigated.ResultsThe evaluation of these models shows that it is possible to predict the synergy of unseen drug combinations with high accuracy (Pearson correlations between 0.43 and 0.86 depending on the considered cell line, with XGBoost providing slightly better predictions than RF). We have also found that restricting to the most reliable synergy predictions results in at least two-fold error decrease with respect to employing the best learning algorithm without any reliability estimation. Alkylating agents, tyrosine kinase inhibitors and topoisomerase inhibitors are the drugs whose synergy with other partner drugs are better predicted by the models.ConclusionsDespite its leading size, NCI-ALMANAC comprises an extremely small part of all conceivable combinations. Given their accuracy and reliability estimation, the developed models should drastically reduce the number of requiredin vitrotests by predictingin silicowhich of the considered combinations are likely to be synergistic.

Application of Artificial Intelligence and Machine Learning Techniques in Classifying Extent of Dementia Across Alzheimer's Image Data

2021 ◽  
Vol 6 (2) ◽  
pp. 29-46
Author(s):  
Robin Ghosh ◽  
Anirudh Reddy Cingreddy ◽  
Venkata Melapu ◽  
Sravanthi Joginipelli ◽  
Supratik Kar
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Nearest Neighbor ◽  
Image Data ◽  
Machine Learning Techniques ◽  
Gradient Boosting ◽  
Support Vector ◽  
K Nearest Neighbor ◽  
Mild Dementia ◽  
Extreme Gradient Boosting

Alzheimer's disease (AD) is one of the most common forms of dementia and the sixth-leading cause of death in older adults. The presented study has illustrated the applications of deep learning (DL) and associated methods, which could have a broader impact on identifying dementia stages and may guide therapy in the future for multiclass image detection. The studied datasets contain around 6,400 magnetic resonance imaging (MRI) images, each segregated into the severity of Alzheimer's classes: mild dementia, very mild dementia, non-dementia, moderate dementia. These four image specifications were used to classify the dementia stages in each patient applying the convolutional neural network (CNN) algorithm. Employing the CNN-based in silico model, the authors successfully classified and predicted the different AD stages and got around 97.19% accuracy. Again, machine learning (ML) techniques like extreme gradient boosting (XGB), support vector machine (SVM), k-nearest neighbor (KNN), and artificial neural network (ANN) offered accuracy of 96.62%, 96.56%, 94.62, and 89.88%, respectively.

scholarly journals Groundwater Prediction Using Machine-Learning Tools

Algorithms ◽  
2020 ◽  
Vol 13 (11) ◽  
pp. 300
Author(s):  
Eslam A. Hussein ◽  
Christopher Thron ◽  
Mehrdad Ghaziasgar ◽  
Antoine Bagula ◽  
Mattia Vaccari
Keyword(s):  
Machine Learning ◽  
Support Vector Regression ◽  
Gaussian Mixture ◽  
Machine Learning Techniques ◽  
Gradient Boosting ◽  
Support Vector ◽  
Learning Tools ◽  
Global Features ◽  
Groundwater Availability ◽  
Extreme Gradient Boosting

Predicting groundwater availability is important to water sustainability and drought mitigation. Machine-learning tools have the potential to improve groundwater prediction, thus enabling resource planners to: (1) anticipate water quality in unsampled areas or depth zones; (2) design targeted monitoring programs; (3) inform groundwater protection strategies; and (4) evaluate the sustainability of groundwater sources of drinking water. This paper proposes a machine-learning approach to groundwater prediction with the following characteristics: (i) the use of a regression-based approach to predict full groundwater images based on sequences of monthly groundwater maps; (ii) strategic automatic feature selection (both local and global features) using extreme gradient boosting; and (iii) the use of a multiplicity of machine-learning techniques (extreme gradient boosting, multivariate linear regression, random forests, multilayer perceptron and support vector regression). Of these techniques, support vector regression consistently performed best in terms of minimizing root mean square error and mean absolute error. Furthermore, including a global feature obtained from a Gaussian Mixture Model produced models with lower error than the best which could be obtained with local geographical features.

scholarly journals Computer-Aided Grading of Gliomas Combining Automatic Segmentation and Radiomics

2018 ◽  
Vol 2018 ◽  
pp. 1-11 ◽  
Author(s):  
Wei Chen ◽  
Boqiang Liu ◽  
Suting Peng ◽  
Jiawei Sun ◽  
Xu Qiao
Keyword(s):  
Automatic Segmentation ◽  
Texture Features ◽  
Recursive Feature Elimination ◽  
Gradient Boosting ◽  
Support Vector ◽  
Low Grade ◽  
Cad System ◽  
Wide Range ◽  
Extreme Gradient Boosting ◽  
Computer Aided

Gliomas are the most common primary brain tumors, and the objective grading is of great importance for treatment. This paper presents an automatic computer-aided diagnosis of gliomas that combines automatic segmentation and radiomics, which can improve the diagnostic ability. The MRI data containing 220 high-grade gliomas and 54 low-grade gliomas are used to evaluate our system. A multiscale 3D convolutional neural network is trained to segment whole tumor regions. A wide range of radiomic features including first-order features, shape features, and texture features is extracted. By using support vector machines with recursive feature elimination for feature selection, a CAD system that has an extreme gradient boosting classifier with a 5-fold cross-validation is constructed for the grading of gliomas. Our CAD system is highly effective for the grading of gliomas with an accuracy of 91.27%, a weighted macroprecision of 91.27%, a weighted macrorecall of 91.27%, and a weighted macro-F1 score of 90.64%. This demonstrates that the proposed CAD system can assist radiologists for high accurate grading of gliomas and has the potential for clinical applications.

A Comparative Study of Machine Learning Techniques for Predicting Sepsis for MIMIC-III Patients

2021 ◽  
Author(s):  
Xuze Zhao ◽  
Bo Qu
Keyword(s):  
Machine Learning ◽  
Intensive Care ◽  
Intensive Care Units ◽  
Medical Information ◽  
Machine Learning Techniques ◽  
Gradient Boosting ◽  
Support Vector ◽  
Mortality And Morbidity ◽  
Extreme Gradient Boosting ◽  
Mimic Iii

Abstract Background: Sepsis is one of the dominating causes of mortality and morbidity in-hospital especially in intensive care units (ICU) patients. Therefore, a reliable decision-making model for predicting sepsis is of great importance. The purpose of this study was to develop an eXtreme Gradient Boosting (XGBoost) based model and explore whether it performs better in predicting sepsis from the time of admission in intensive care units (ICU) than other machine learning (ML) methods. Methods: The source data used for model establishment in this study were from a retrospective medical information mart for intensive care (MIMIC) III dataset, restricted to intensive care units (ICUs) patients aged between 18 and 89. Model performance of the XGBoost model was compared to logistic regression (LR), recursive neural network (RNN), and support vector machine (SVM). Then, the performances of the models were evaluated and compared by the area under the curve (AUC) of the receiver operating characteristic (ROC) curves.Results: A total of 6430 MIMIC-III cases are included in this article, in which, 3021 cases have encountered sepsis while 3409 cases have not, respectively. As for the AUC (0.808 (95% CI): 0.767-0.848,DT), 0.802 (95%CI: 0.762-0.842，RNN), 0.790 (95%CI: 0.751-0.830,SVM), 0.775 (95%CI: 0.736-0.813,LR) , results of the models, XGBoost performs best in predicting sepsis.Conclusions: By using the DT algorithm, a more accurate prediction model can be established. Amongst other ML methods, the XGBoost model demonstrated the best ability in detecting the sepsis of the patients in ICU.

scholarly journals Intercomparing the robustness of machine learning models in simulation and forecasting of streamflow

2020 ◽  
Author(s):  
Parthiban Loganathan ◽  
Amit Baburao Mahindrakar
Keyword(s):  
Machine Learning ◽  
Large Scale ◽  
Resource Planning ◽  
Machine Learning Techniques ◽  
Low Flow ◽  
Gradient Boosting ◽  
Daily Streamflow ◽  
Learning Techniques ◽  
Extreme Gradient Boosting ◽  
Hydrological Indices

Abstract The intercomparison of streamflow simulation and the prediction of discharge using various renowned machine learning techniques were performed. The daily streamflow discharge model was developed for 35 observation stations located in a large-scale river basin named Cauvery. Various hydrological indices were calculated for observed and predicted discharges for comparing and evaluating the replicability of local hydrological conditions. The model variance and bias observed from the proposed extreme gradient boosting decision tree model were less than 15%, which is compared with other machine learning techniques considered in this study. The model Nash–Sutcliffe efficiency and coefficient of determination values are above 0.7 for both the training and testing phases which demonstrate the effectiveness of model performance. The comparison of monthly observed and model-predicted discharges during the validation period illustrates the model's ability in representing the peaks and fall in high-, medium-, and low-flow zones. The assessment and comparison of hydrological indices between observed and predicted discharges illustrate the model's ability in representing the baseflow, high-spell, and low-spell statistics. Simulating streamflow and predicting discharge are essential for water resource planning and management, especially in large-scale river basins. The proposed machine learning technique demonstrates significant improvement in model efficiency by dropping variance and bias which, in turn, improves the replicability of local-scale hydrology.

Sign in / Sign up

Export Citation Format

Share Document