scholarly journals Hyperparameter Importance Analysis based on N-RReliefF Algorithm

2019 ◽  
Vol 14 (4) ◽  
pp. 557-573 ◽  
Author(s):  
Yunlei Sun ◽  
Huiquan Gong ◽  
Yucong Li ◽  
Dalin Zhang
Keyword(s):  
Machine Learning ◽  
Bayesian Optimization ◽  
Great Success ◽  
Random Forest Algorithm ◽  
Bayesian Optimization Algorithm ◽  
Svm Algorithm ◽  
Importance Analysis ◽  
Hyperparameter Selection ◽  
And Performance ◽  
Influence Degree

Hyperparameter selection has always been the key to machine learning. The Bayesian optimization algorithm has recently achieved great success, but it has certain constraints and limitations in selecting hyperparameters. In response to these constraints and limitations, this paper proposed the N-RReliefF algorithm, which can evaluate the importance of hyperparameters and the importance weights between hyperparameters. The N-RReliefF algorithm estimates the contribution of a single hyperparameter to the performance according to the influence degree of each hyperparameter on the performance and calculates the weight of importance between the hyperparameters according to the improved normalization formula. The N-RReliefF algorithm analyses the hyperparameter configuration and performance set generated by Bayesian optimization, and obtains the important hyperparameters in random forest algorithm and SVM algorithm. The experimental results verify the effectiveness of the N-RReliefF algorithm.

scholarly journals An Approach to Hyperparameter Optimization for the Objective Function in Machine Learning

Electronics ◽  
2019 ◽  
Vol 8 (11) ◽  
pp. 1267 ◽  
Author(s):  
Yonghoon Kim ◽  
and Mokdong Chung
Keyword(s):  
Machine Learning ◽  
Learning Process ◽  
Learning Rate ◽  
Bayesian Optimization ◽  
Learning Performance ◽  
Batch Size ◽  
Critical Problem ◽  
Hyperparameter Optimization ◽  
Training Performance ◽  
And Performance

In machine learning, performance is of great value. However, each learning process requires much time and effort in setting each parameter. The critical problem in machine learning is determining the hyperparameters, such as the learning rate, mini-batch size, and regularization coefficient. In particular, we focus on the learning rate, which is directly related to learning efficiency and performance. Bayesian optimization using a Gaussian Process is common for this purpose. In this paper, based on Bayesian optimization, we attempt to optimize the hyperparameters automatically by utilizing a Gamma distribution, instead of a Gaussian distribution, to improve the training performance of predicting image discrimination. As a result, our proposed method proves to be more reasonable and efficient in the estimation of learning rate when training the data, and can be useful in machine learning.

scholarly journals Prediction of tubular solar still performance by machine learning integrated with Bayesian optimization algorithm

2020 ◽  
pp. 116233
Author(s):  
Yunpeng Wang ◽  
A.W. Kandeal ◽  
Ahmed Swidan ◽  
Swellam W. Sharshir ◽  
Gamal B. Abdelaziz ◽  
...  
Keyword(s):  
Machine Learning ◽  
Optimization Algorithm ◽  
Bayesian Optimization ◽  
Solar Still ◽  
Bayesian Optimization Algorithm

scholarly journals A time-dependent parameter estimation framework for crop modeling

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Faezeh Akhavizadegan ◽  
Javad Ansarifar ◽  
Lizhi Wang ◽  
Isaiah Huber ◽  
Sotirios V. Archontoulis
Keyword(s):  
Machine Learning ◽  
Parameter Estimation ◽  
Estimation Method ◽  
Time Dependent ◽  
Bayesian Optimization ◽  
Parameter Estimates ◽  
Crop Modeling ◽  
Bayesian Optimization Algorithm ◽  
Crop Models ◽  
Dependent Parameter

AbstractThe performance of crop models in simulating various aspects of the cropping system is sensitive to parameter calibration. Parameter estimation is challenging, especially for time-dependent parameters such as cultivar parameters with 2–3 years of lifespan. Manual calibration of the parameters is time-consuming, requires expertise, and is prone to error. This research develops a new automated framework to estimate time-dependent parameters for crop models using a parallel Bayesian optimization algorithm. This approach integrates the power of optimization and machine learning with prior agronomic knowledge. To test the proposed time-dependent parameter estimation method, we simulated historical yield increase (from 1985 to 2018) in 25 environments in the US Corn Belt with APSIM. Then we compared yield simulation results and nine parameter estimates from our proposed parallel Bayesian framework, with Bayesian optimization and manual calibration. Results indicated that parameters calibrated using the proposed framework achieved an 11.6% reduction in the prediction error over Bayesian optimization and a 52.1% reduction over manual calibration. We also trained nine machine learning models for yield prediction and found that none of them was able to outperform the proposed method in terms of root mean square error and R2. The most significant contribution of the new automated framework for time-dependent parameter estimation is its capability to find close-to-optimal parameters for the crop model. The proposed approach also produced explainable insight into cultivar traits’ trends over 34 years (1985–2018).

scholarly journals Early Prediction of Seven-Day Mortality in Intensive Care Unit Using a Machine Learning Model: Results from the SPIN-UTI Project

2021 ◽  
Vol 10 (5) ◽  
pp. 992
Author(s):  
Martina Barchitta ◽  
Andrea Maugeri ◽  
Giuliana Favara ◽  
Paolo Marco Riela ◽  
Giovanni Gallo ◽  
...  
Keyword(s):  
Machine Learning ◽  
Intensive Care ◽  
Intensive Care Units ◽  
Learning Algorithm ◽  
Area Under The Curve ◽  
Support Vector ◽  
Icu Admission ◽  
Risk Of Death ◽  
Saps Ii ◽  
Svm Algorithm

Patients in intensive care units (ICUs) were at higher risk of worsen prognosis and mortality. Here, we aimed to evaluate the ability of the Simplified Acute Physiology Score (SAPS II) to predict the risk of 7-day mortality, and to test a machine learning algorithm which combines the SAPS II with additional patients’ characteristics at ICU admission. We used data from the “Italian Nosocomial Infections Surveillance in Intensive Care Units” network. Support Vector Machines (SVM) algorithm was used to classify 3782 patients according to sex, patient’s origin, type of ICU admission, non-surgical treatment for acute coronary disease, surgical intervention, SAPS II, presence of invasive devices, trauma, impaired immunity, antibiotic therapy and onset of HAI. The accuracy of SAPS II for predicting patients who died from those who did not was 69.3%, with an Area Under the Curve (AUC) of 0.678. Using the SVM algorithm, instead, we achieved an accuracy of 83.5% and AUC of 0.896. Notably, SAPS II was the variable that weighted more on the model and its removal resulted in an AUC of 0.653 and an accuracy of 68.4%. Overall, these findings suggest the present SVM model as a useful tool to early predict patients at higher risk of death at ICU admission.

scholarly journals An Integrated Approach of Mechanistic-Modeling and Machine-Learning for Thickness Optimization of Frozen Microwaveable Foods

Foods ◽  
2021 ◽  
Vol 10 (4) ◽  
pp. 763
Author(s):  
Ran Yang ◽  
Zhenbo Wang ◽  
Jiajia Chen
Keyword(s):  
Machine Learning ◽  
Learning Algorithm ◽  
Food Product ◽  
Integrated Approach ◽  
Mechanistic Modeling ◽  
Bayesian Optimization ◽  
Initial Training ◽  
Thickness Optimization ◽  
Heating Uniformity ◽  
Food Design

Mechanistic-modeling has been a useful tool to help food scientists in understanding complicated microwave-food interactions, but it cannot be directly used by the food developers for food design due to its resource-intensive characteristic. This study developed and validated an integrated approach that coupled mechanistic-modeling and machine-learning to achieve efficient food product design (thickness optimization) with better heating uniformity. The mechanistic-modeling that incorporated electromagnetics and heat transfer was previously developed and validated extensively and was used directly in this study. A Bayesian optimization machine-learning algorithm was developed and integrated with the mechanistic-modeling. The integrated approach was validated by comparing the optimization performance with a parametric sweep approach, which is solely based on mechanistic-modeling. The results showed that the integrated approach had the capability and robustness to optimize the thickness of different-shape products using different initial training datasets with higher efficiency (45.9% to 62.1% improvement) than the parametric sweep approach. Three rectangular-shape trays with one optimized thickness (1.56 cm) and two non-optimized thicknesses (1.20 and 2.00 cm) were 3-D printed and used in microwave heating experiments, which confirmed the feasibility of the integrated approach in thickness optimization. The integrated approach can be further developed and extended as a platform to efficiently design complicated microwavable foods with multiple-parameter optimization.

scholarly journals New Hybrid Approach for Developing Automated Machine Learning Workflows: A Real Case Application in Evaluation of Marcellus Shale Gas Production

Fuels ◽  
2021 ◽  
Vol 2 (3) ◽  
pp. 286-303
Author(s):  
Vuong Van Pham ◽  
Ebrahim Fathi ◽  
Fatemeh Belyadi
Keyword(s):  
Artificial Intelligence ◽  
Machine Learning ◽  
Shale Gas ◽  
Field Data ◽  
Marcellus Shale ◽  
Hybrid Approach ◽  
Gas Production ◽  
Bayesian Optimization ◽  
Real Field ◽  
Engineering Problems

The success of machine learning (ML) techniques implemented in different industries heavily rely on operator expertise and domain knowledge, which is used in manually choosing an algorithm and setting up the specific algorithm parameters for a problem. Due to the manual nature of model selection and parameter tuning, it is impossible to quantify or evaluate the quality of this manual process, which in turn limits the ability to perform comparison studies between different algorithms. In this study, we propose a new hybrid approach for developing machine learning workflows to help automated algorithm selection and hyperparameter optimization. The proposed approach provides a robust, reproducible, and unbiased workflow that can be quantified and validated using different scoring metrics. We have used the most common workflows implemented in the application of artificial intelligence (AI) and ML in engineering problems including grid/random search, Bayesian search and optimization, genetic programming, and compared that with our new hybrid approach that includes the integration of Tree-based Pipeline Optimization Tool (TPOT) and Bayesian optimization. The performance of each workflow is quantified using different scoring metrics such as Pearson correlation (i.e., R2 correlation) and Mean Square Error (i.e., MSE). For this purpose, actual field data obtained from 1567 gas wells in Marcellus Shale, with 121 features from reservoir, drilling, completion, stimulation, and operation is tested using different proposed workflows. A proposed new hybrid workflow is then used to evaluate the type well used for evaluation of Marcellus shale gas production. In conclusion, our automated hybrid approach showed significant improvement in comparison to other proposed workflows using both scoring matrices. The new hybrid approach provides a practical tool that supports the automated model and hyperparameter selection, which is tested using real field data that can be implemented in solving different engineering problems using artificial intelligence and machine learning. The new hybrid model is tested in a real field and compared with conventional type wells developed by field engineers. It is found that the type well of the field is very close to P50 predictions of the field, which shows great success in the completion design of the field performed by field engineers. It also shows that the field average production could have been improved by 8% if shorter cluster spacing and higher proppant loading per cluster were used during the frac jobs.

Classification and photometric redshift estimation of quasars in photometric surveys

2020 ◽  
Vol 15 (S359) ◽  
pp. 40-41
Author(s):  
L. M. Izuti Nakazono ◽  
C. Mendes de Oliveira ◽  
N. S. T. Hirata ◽  
S. Jeram ◽  
A. Gonzalez ◽  
...  
Keyword(s):  
Machine Learning ◽  
Random Forest ◽  
Nearest Neighbour ◽  
Random Forest Algorithm ◽  
Photometric Redshift ◽  
Using Data

AbstractWe present a machine learning methodology to separate quasars from galaxies and stars using data from S-PLUS in the Stripe-82 region. In terms of quasar classification, we achieved 95.49% for precision and 95.26% for recall using a Random Forest algorithm. For photometric redshift estimation, we obtained a precision of 6% using k-Nearest Neighbour.

Sign in / Sign up

Export Citation Format

Share Document