Automated Machine Learning and Bayesian Optimization

2019 ◽  
pp. 1-18
Author(s):  
Francesco Archetti ◽  
Antonio Candelieri
Keyword(s):  
Machine Learning ◽  
Bayesian Optimization ◽  
Automated Machine Learning

scholarly journals Bayesian Optimization for Categorical and Category-Specific Continuous Inputs

2020 ◽  
Vol 34 (04) ◽  
pp. 5256-5263 ◽  
Author(s):  
Dang Nguyen ◽  
Sunil Gupta ◽  
Santu Rana ◽  
Alistair Shilton ◽  
Svetha Venkatesh
Keyword(s):  
Machine Learning ◽  
Parameter Tuning ◽  
Bayesian Optimization ◽  
Parallel Optimization ◽  
Continuous Variables ◽  
Unified Framework ◽  
Multiple Resources ◽  
Thompson Sampling ◽  
Machine Learning Model ◽  
Automated Machine Learning

Many real-world functions are defined over both categorical and category-specific continuous variables and thus cannot be optimized by traditional Bayesian optimization (BO) methods. To optimize such functions, we propose a new method that formulates the problem as a multi-armed bandit problem, wherein each category corresponds to an arm with its reward distribution centered around the optimum of the objective function in continuous variables. Our goal is to identify the best arm and the maximizer of the corresponding continuous function simultaneously. Our algorithm uses a Thompson sampling scheme that helps connecting both multi-arm bandit and BO in a unified framework. We extend our method to batch BO to allow parallel optimization when multiple resources are available. We theoretically analyze our method for convergence and prove sub-linear regret bounds. We perform a variety of experiments: optimization of several benchmark functions, hyper-parameter tuning of a neural network, and automatic selection of the best machine learning model along with its optimal hyper-parameters (a.k.a automated machine learning). Comparisons with other methods demonstrate the effectiveness of our proposed method.

Bandit-Based Automated Machine Learning

2018 ◽  
Author(s):  
Silvia Cristina Nunes das Dores ◽  
Carlos Soares ◽  
Duncan Ruiz
Keyword(s):  
Machine Learning ◽  
Automated Machine Learning

scholarly journals An Integrated Approach of Mechanistic-Modeling and Machine-Learning for Thickness Optimization of Frozen Microwaveable Foods

Foods ◽  
2021 ◽  
Vol 10 (4) ◽  
pp. 763
Author(s):  
Ran Yang ◽  
Zhenbo Wang ◽  
Jiajia Chen
Keyword(s):  
Machine Learning ◽  
Learning Algorithm ◽  
Food Product ◽  
Integrated Approach ◽  
Mechanistic Modeling ◽  
Bayesian Optimization ◽  
Initial Training ◽  
Thickness Optimization ◽  
Heating Uniformity ◽  
Food Design

Mechanistic-modeling has been a useful tool to help food scientists in understanding complicated microwave-food interactions, but it cannot be directly used by the food developers for food design due to its resource-intensive characteristic. This study developed and validated an integrated approach that coupled mechanistic-modeling and machine-learning to achieve efficient food product design (thickness optimization) with better heating uniformity. The mechanistic-modeling that incorporated electromagnetics and heat transfer was previously developed and validated extensively and was used directly in this study. A Bayesian optimization machine-learning algorithm was developed and integrated with the mechanistic-modeling. The integrated approach was validated by comparing the optimization performance with a parametric sweep approach, which is solely based on mechanistic-modeling. The results showed that the integrated approach had the capability and robustness to optimize the thickness of different-shape products using different initial training datasets with higher efficiency (45.9% to 62.1% improvement) than the parametric sweep approach. Three rectangular-shape trays with one optimized thickness (1.56 cm) and two non-optimized thicknesses (1.20 and 2.00 cm) were 3-D printed and used in microwave heating experiments, which confirmed the feasibility of the integrated approach in thickness optimization. The integrated approach can be further developed and extended as a platform to efficiently design complicated microwavable foods with multiple-parameter optimization.

scholarly journals New Hybrid Approach for Developing Automated Machine Learning Workflows: A Real Case Application in Evaluation of Marcellus Shale Gas Production

Fuels ◽  
2021 ◽  
Vol 2 (3) ◽  
pp. 286-303
Author(s):  
Vuong Van Pham ◽  
Ebrahim Fathi ◽  
Fatemeh Belyadi
Keyword(s):  
Artificial Intelligence ◽  
Machine Learning ◽  
Shale Gas ◽  
Field Data ◽  
Marcellus Shale ◽  
Hybrid Approach ◽  
Gas Production ◽  
Bayesian Optimization ◽  
Real Field ◽  
Engineering Problems

The success of machine learning (ML) techniques implemented in different industries heavily rely on operator expertise and domain knowledge, which is used in manually choosing an algorithm and setting up the specific algorithm parameters for a problem. Due to the manual nature of model selection and parameter tuning, it is impossible to quantify or evaluate the quality of this manual process, which in turn limits the ability to perform comparison studies between different algorithms. In this study, we propose a new hybrid approach for developing machine learning workflows to help automated algorithm selection and hyperparameter optimization. The proposed approach provides a robust, reproducible, and unbiased workflow that can be quantified and validated using different scoring metrics. We have used the most common workflows implemented in the application of artificial intelligence (AI) and ML in engineering problems including grid/random search, Bayesian search and optimization, genetic programming, and compared that with our new hybrid approach that includes the integration of Tree-based Pipeline Optimization Tool (TPOT) and Bayesian optimization. The performance of each workflow is quantified using different scoring metrics such as Pearson correlation (i.e., R2 correlation) and Mean Square Error (i.e., MSE). For this purpose, actual field data obtained from 1567 gas wells in Marcellus Shale, with 121 features from reservoir, drilling, completion, stimulation, and operation is tested using different proposed workflows. A proposed new hybrid workflow is then used to evaluate the type well used for evaluation of Marcellus shale gas production. In conclusion, our automated hybrid approach showed significant improvement in comparison to other proposed workflows using both scoring matrices. The new hybrid approach provides a practical tool that supports the automated model and hyperparameter selection, which is tested using real field data that can be implemented in solving different engineering problems using artificial intelligence and machine learning. The new hybrid model is tested in a real field and compared with conventional type wells developed by field engineers. It is found that the type well of the field is very close to P50 predictions of the field, which shows great success in the completion design of the field performed by field engineers. It also shows that the field average production could have been improved by 8% if shorter cluster spacing and higher proppant loading per cluster were used during the frac jobs.

scholarly journals Automated Machine-Learning Radiotherapy Planning for Pediatric and Adult Brain Tumours

2021 ◽  
Vol 52 (2) ◽  
pp. S3
Author(s):  
Grace Tsui ◽  
Derek S. Tsang ◽  
Chris McIntosh ◽  
Thomas G. Purdie ◽  
Glenn Bauman ◽  
...  
Keyword(s):  
Machine Learning ◽  
Brain Tumours ◽  
Adult Brain ◽  
Radiotherapy Planning ◽  
Automated Machine Learning

scholarly journals Testing the Suitability of Automated Machine Learning for Weeds Identification

AI ◽  
2021 ◽  
Vol 2 (1) ◽  
pp. 34-47
Author(s):  
Borja Espejo-Garcia ◽  
Ioannis Malounas ◽  
Eleanna Vali ◽  
Spyros Fountas
Keyword(s):  
Machine Learning ◽  
Plant Protection ◽  
Crop Protection ◽  
Identification Problem ◽  
Learning System ◽  
Classifier Ensembles ◽  
Automated Machine Learning ◽  
A New Technique ◽  
Plant Seedlings ◽  
And Training

In the past years, several machine-learning-based techniques have arisen for providing effective crop protection. For instance, deep neural networks have been used to identify different types of weeds under different real-world conditions. However, these techniques usually require extensive involvement of experts working iteratively in the development of the most suitable machine learning system. To support this task and save resources, a new technique called Automated Machine Learning has started being studied. In this work, a complete open-source Automated Machine Learning system was evaluated with two different datasets, (i) The Early Crop Weeds dataset and (ii) the Plant Seedlings dataset, covering the weeds identification problem. Different configurations, such as the use of plant segmentation, the use of classifier ensembles instead of Softmax and training with noisy data, have been compared. The results showed promising performances of 93.8% and 90.74% F1 score depending on the dataset used. These performances were aligned with other related works in AutoML, but they are far from machine-learning-based systems manually fine-tuned by human experts. From these results, it can be concluded that finding a balance between manual expert work and Automated Machine Learning will be an interesting path to work in order to increase the efficiency in plant protection.

Sign in / Sign up

Export Citation Format

Share Document