Evaluating Optimization Strategies for Engine Simulations Using Machine Learning Emulators

2019 ◽  
Vol 141 (9) ◽  
Author(s):  
Daniel M. Probst ◽  
Mandhapati Raju ◽  
Peter K. Senecal ◽  
Janardhan Kodavasal ◽  
Pinaki Pal ◽  
...  
Keyword(s):  
Machine Learning ◽  
Random Search ◽  
Optimization Algorithms ◽  
Optimization Methods ◽  
Machine Learning Algorithms ◽  
Cfd Simulations ◽  
Engine Combustion ◽  
Computational Fluid Dynamics Cfd ◽  
Micro Genetic Algorithm ◽  
Time Required

This work evaluates different optimization algorithms for computational fluid dynamics (CFD) simulations of engine combustion. Due to the computational expense of CFD simulations, emulators built with machine learning algorithms were used as surrogates for the optimizers. Two types of emulators were used: a Gaussian process (GP) and a weighted variety of machine learning methods called SuperLearner (SL). The emulators were trained using a dataset of 2048 CFD simulations that were run concurrently on a supercomputer. The design of experiments (DOE) for the CFD runs was obtained by perturbing nine input parameters using a Monte-Carlo method. The CFD simulations were of a heavy duty engine running with a low octane gasoline-like fuel at a partially premixed compression ignition mode. Ten optimization algorithms were tested, including types typically used in research applications. Each optimizer was allowed 800 function evaluations and was randomly tested 100 times. The optimizers were evaluated for the median, minimum, and maximum merits obtained in the 100 attempts. Some optimizers required more sequential evaluations, thereby resulting in longer wall clock times to reach an optimum. The best performing optimization methods were particle swarm optimization (PSO), differential evolution (DE), GENOUD (an evolutionary algorithm), and micro-genetic algorithm (GA). These methods found a high median optimum as well as a reasonable minimum optimum of the 100 trials. Moreover, all of these methods were able to operate with less than 100 successive iterations, which reduced the wall clock time required in practice. Two methods were found to be effective but required a much larger number of successive iterations: the DIRECT and MALSCHAINS algorithms. A random search method that completed in a single iteration performed poorly in finding optimum designs but was included to illustrate the limitation of highly concurrent search methods. The last three methods, Nelder–Mead, bound optimization by quadratic approximation (BOBYQA), and constrained optimization by linear approximation (COBYLA), did not perform as well.

Evaluating Optimization Strategies for Engine Simulations Using Machine Learning Emulators

2018 ◽  
Author(s):  
Daniel M. Probst ◽  
Mandhapati Raju ◽  
Peter K. Senecal ◽  
Janardhan Kodavasal ◽  
Pinaki Pal ◽  
...  
Keyword(s):  
Machine Learning ◽  
San Antonio ◽  
Random Search ◽  
Optimization Algorithms ◽  
Optimization Methods ◽  
Machine Learning Algorithms ◽  
Cfd Simulations ◽  
Southwest Research Institute ◽  
Computational Fluid Dynamics Cfd ◽  
Time Required

This work evaluates different optimization algorithms for Computational Fluid Dynamics (CFD) simulations of engine combustion. Due to the computational expense of CFD simulations, emulators built with machine learning algorithms were used as surrogates for the optimizers. Two types of emulators were used: a Gaussian Process (GP) and a weighted variety of machine learning methods called SuperLearner (SL). The emulators were trained using a dataset of 2048 CFD simulations that were run concurrently on a supercomputer. The Design of Experiments (DOE) for the CFD runs was obtained by perturbing nine input parameters using a Monte Carlo method. The CFD simulations were of a heavy duty engine running with a low octane gasoline-like fuel at a partially premixed compression ignition mode. Ten optimization algorithms were tested, including types typically used in research applications. Each optimizer was allowed 800 function evaluations and was randomly tested 100 times. The optimizers were evaluated for the median, minimum, and maximum merits obtained in the 100 attempts. Some optimizers required more sequential evaluations, thereby resulting in longer wall clock times to reach an optimum. The best performing optimization methods were particle swarm optimization (PSO), differential evolution (DE), GENOUD (an evolutionary algorithm), and Micro-Genetic Algorithm (GA). These methods found a high median optimum as well as a reasonable minimum optimum of the 100 trials. Moreover, all of these methods were able to operate with less than 100 successive iterations, which reduced the wall clock time required in practice. Two methods were found to be effective but required a much larger number of successive iterations: the DIRECT and MALSCHAINS algorithms. A random search method that completed in a single iteration performed poorly in finding 1 Currently at Southwest Research Institute, San Antonio, Texas optimum designs, but was included to illustrate the limitation of highly concurrent search methods. The last three methods, Nelder-Mead, BOBYQA, and COBYLA, did not perform as well.

scholarly journals Machine Learning for Design Optimization of Electromagnetic Devices: Recent Developments and Future Directions

2021 ◽  
Vol 11 (4) ◽  
pp. 1627
Author(s):  
Yanbin Li ◽  
Gang Lei ◽  
Gerd Bramerdorfer ◽  
Sheng Peng ◽  
Xiaodong Sun ◽  
...  
Keyword(s):  
Machine Learning ◽  
Design Optimization ◽  
Optimization Methods ◽  
Machine Learning Algorithms ◽  
Cloud Services ◽  
Robust Design Optimization ◽  
Support Vector ◽  
Future Directions ◽  
Electromagnetic Devices ◽  
Recent Developments

This paper reviews the recent developments of design optimization methods for electromagnetic devices, with a focus on machine learning methods. First, the recent advances in multi-objective, multidisciplinary, multilevel, topology, fuzzy, and robust design optimization of electromagnetic devices are overviewed. Second, a review is presented to the performance prediction and design optimization of electromagnetic devices based on the machine learning algorithms, including artificial neural network, support vector machine, extreme learning machine, random forest, and deep learning. Last, to meet modern requirements of high manufacturing/production quality and lifetime reliability, several promising topics, including the application of cloud services and digital twin, are discussed as future directions for design optimization of electromagnetic devices.

Systematic Uncertainty Considerations in the Comparison of Experimental and Computed Cylinder Pressure and Heat Release Histories

2018 ◽  
Author(s):  
K. R. Partridge ◽  
P. R. Jha ◽  
H. Mahabadipour ◽  
K. K. Srinivasan ◽  
S. R. Krishnan
Keyword(s):  
Heat Release ◽  
Cylinder Pressure ◽  
Computational Simulations ◽  
Cfd Simulations ◽  
Engine Combustion ◽  
Computational Fluid Dynamics Cfd ◽  
Ic Engines ◽  
Dynamics Simulations ◽  
Combustion Processes ◽  
Experimental Pressure

Computational simulations of engine combustion processes are increasingly relied upon to lead the design of advanced IC engines. Both computational fluid dynamics (CFD) simulations as well as thermodynamics-based phenomenological 0D or 1D gas dynamics simulations are examples of current simulation strategies. Before simulations can be utilized to guide the design process, they must be validated with experimental results. Typically, the experimental data used for validation of computational simulations include in-cylinder pressure and apparent heat release rate (AHRR) histories. However, the process of comparison of experimental and simulated pressure and AHRR curves is largely qualitative; therefore, the validation process is mostly visual. In the present work, the authors introduce a framework for quantifying uncertainties in experimental pressure data, as well as uncertainties in the “average” AHRR curve that is derived from ensemble-averaged cylinder pressure histories. Predicted AHRR curves from CFD simulations are also quantitatively compared with the experimental AHRR bounded by “uncertainty bands” in the present work.

scholarly journals Evaluation and calibration of a low-cost particle sensor in ambient conditions using machine-learning methods

2020 ◽  
Vol 13 (4) ◽  
pp. 1693-1707 ◽  
Author(s):  
Minxing Si ◽  
Ying Xiong ◽  
Shan Du ◽  
Ke Du
Keyword(s):  
Machine Learning ◽  
Linear Regression ◽  
Random Search ◽  
Low Cost ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Ambient Conditions ◽  
Test Dataset ◽  
Compact Size ◽  
Significant Difference

Abstract. Particle sensing technology has shown great potential for monitoring particulate matter (PM) with very few temporal and spatial restrictions because of its low cost, compact size, and easy operation. However, the performance of low-cost sensors for PM monitoring in ambient conditions has not been thoroughly evaluated. Monitoring results by low-cost sensors are often questionable. In this study, a low-cost fine particle monitor (Plantower PMS 5003) was colocated with a reference instrument, the Synchronized Hybrid Ambient Real-time Particulate (SHARP) monitor, at the Calgary Varsity air monitoring station from December 2018 to April 2019. The study evaluated the performance of this low-cost PM sensor in ambient conditions and calibrated its readings using simple linear regression (SLR), multiple linear regression (MLR), and two more powerful machine-learning algorithms using random search techniques for the best model architectures. The two machine-learning algorithms are XGBoost and a feedforward neural network (NN). Field evaluation showed that the Pearson correlation (r) between the low-cost sensor and the SHARP instrument was 0.78. The Fligner and Killeen (F–K) test indicated a statistically significant difference between the variances of the PM2.5 values by the low-cost sensor and the SHARP instrument. Large overestimations by the low-cost sensor before calibration were observed in the field and were believed to be caused by the variation of ambient relative humidity. The root mean square error (RMSE) was 9.93 when comparing the low-cost sensor with the SHARP instrument. The calibration by the feedforward NN had the smallest RMSE of 3.91 in the test dataset compared to the calibrations by SLR (4.91), MLR (4.65), and XGBoost (4.19). After calibrations, the F–K test using the test dataset showed that the variances of the PM2.5 values by the NN, XGBoost, and the reference method were not statistically significantly different. From this study, we conclude that a feedforward NN is a promising method to address the poor performance of low-cost sensors for PM2.5 monitoring. In addition, the random search method for hyperparameters was demonstrated to be an efficient approach for selecting the best model structure.

scholarly journals Weighted Random Search for Hyperparameter Optimization

2019 ◽  
Vol 14 (2) ◽  
pp. 154-169 ◽  
Author(s):  
Adrian-Catalin Florea ◽  
Razvan Andonie
Keyword(s):  
Machine Learning ◽  
Optimization Problem ◽  
Random Search ◽  
Machine Learning Algorithms ◽  
New Combinations ◽  
Hyperparameter Optimization ◽  
A Value ◽  
Computational Budget ◽  
Theoretical Results ◽  
Discrete Domain

We introduce an improved version of Random Search (RS), used here for hyperparameter optimization of machine learning algorithms. Unlike the standard RS, which generates for each trial new values for all hyperparameters, we generate new values for each hyperparameter with a probability of change. The intuition behind our approach is that a value that already triggered a good result is a good candidate for the next step, and should be tested in new combinations of hyperparameter values. Within the same computational budget, our method yields better results than the standard RS. Our theoretical results prove this statement. We test our method on a variation of one of the most commonly used objective function for this class of problems (the Grievank function) and for the hyperparameter optimization of a deep learning CNN architecture. Our results can be generalized to any optimization problem dened on a discrete domain.

scholarly journals Utilizing High Performance Computing to Improve the Application of Machine Learning for Time-Efficient Prediction of Buildings’ Daylighting Performance

2021 ◽  
Vol 2069 (1) ◽  
pp. 012153
Author(s):  
Rania Labib
Keyword(s):  
Machine Learning ◽  
High Performance Computing ◽  
High Performance ◽  
Learning Algorithm ◽  
Energy Performance ◽  
Machine Learning Algorithms ◽  
Machine Learning Algorithm ◽  
Validation Data ◽  
Time Required ◽  
Performance Computing

Abstract Architects often investigate the daylighting performance of hundreds of design solutions and configurations to ensure an energy-efficient solution for their designs. To shorten the time required for daylighting simulations, architects usually reduce the number of variables or parameters of the building and facade design. This practice usually results in the elimination of design variables that could contribute to an energy-optimized design configuration. Therefore, recent research has focused on incorporating machine learning algorithms that require the execution of only a relatively small subset of the simulations to predict the daylighting and energy performance of buildings. Although machine learning has been shown to be accurate, it still becomes a time-consuming process due to the time required to execute a set of simulations to be used as training and validation data. Furthermore, to save time, designers often decide to use a small simulation subset, which leads to a poorly designed machine learning algorithm that produces inaccurate results. Therefore, this study aims to introduce an automated framework that utilizes high performance computing (HPC) to execute the simulations necessary for the machine learning algorithm while saving time and effort. High performance computing facilitates the execution of thousands of tasks simultaneously for a time-efficient simulation process, therefore allowing designers to increase the size of the simulation’s subset. Pairing high performance computing with machine learning allows for accurate and nearly instantaneous building performance predictions.

Field Development Optimization Using Machine Learning Methods to Identify the Optimal Water Flooding Regime

2021 ◽  
Author(s):  
Alexey Vasilievich Timonov ◽  
Arturas Rimo Shabonas ◽  
Sergey Alexandrovich Schmidt
Keyword(s):  
Machine Learning ◽  
Optimization Algorithms ◽  
Machine Learning Algorithms ◽  
Water Flooding ◽  
Learning Models ◽  
Learning Methods ◽  
Injection Wells ◽  
Field Development ◽  
Machine Learning Methods ◽  
Machine Learning Models

Abstract The main technology used to optimize field development is hydrodynamic modeling, which is very costly in terms of computing resources and expert time to configure the model. And in the case of brownfields, the complexity increases exponentially. The paper describes the stages of developing a hybrid geological-physical-mathematical proxy model using machine learning methods, which allows performing multivariate calculations and predicting production including various injection well operating regimes. Based on the calculations, we search for the optimal ratio of injection volume distribution to injection wells under given infrastructural constraints. The approach implemented in this work takes into account many factors (some features of the geological structure, history of field development, mutual influence of wells, etc.) and can offer optimal options for distribution of injection volumes of injection wells without performing full-scale or sector hydrodynamic simulation. To predict production, we use machine learning methods (based on decision trees and neural networks) and methods for optimizing the target functions. As a result of this research, a unified algorithm for data verification and preprocessing has been developed for feature extraction tasks and the use of deep machine learning models as input data. Various machine learning algorithms were tested and it was determined that the highest prediction accuracy is achieved by building machine learning models based on Temporal Convolutional Networks (TCN) and gradient boosting. Developed and tested an algorithm for finding the optimal allocation of injection volumes, taking into account the existing infrastructure constraints. Different optimization algorithms are tested. It is determined that the choice and setting of boundary conditions is critical for optimization algorithms in this problem. An integrated approach was tested on terrigenous formations of the West Siberian field, where the developed algorithm showed effectiveness.

scholarly journals Evaluation and Calibration of a Low-cost Particle Sensor in Ambient Conditions Using Machine Learning Technologies

2019 ◽  
Author(s):  
Minxing Si ◽  
Ying Xiong ◽  
Shan Du ◽  
Ke Du
Keyword(s):  
Machine Learning ◽  
Linear Regression ◽  
Random Search ◽  
Low Cost ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Ambient Conditions ◽  
Test Dataset ◽  
Compact Size ◽  
Significant Difference

Abstract. Particle sensing technology has shown great potential for monitoring particulate matter (PM) with very few temporal and spatial restrictions because of low-cost, compact size, and easy operation. However, the performance of low-cost sensors for PM monitoring in ambient conditions has not been thoroughly evaluated. Monitoring results by low-cost sensors are often questionable. In this study, a low-cost fine particle monitor (Plantower PMS 5003) was co-located with a reference instrument, named Synchronized Hybrid Ambient Real-time Particulate (SHARP) monitor, in Calgary Varsity air monitoring station from December 2018 to April 2019. The study evaluated the performance of this low-cost PM sensor in ambient conditions and calibrated its readings using simple linear regression (SLR), multiple linear regression (MLR), and two more powerful machine learning algorithms using random search techniques for the best model architectures. The two machine learning algorithms are XGBoost and feedforward neural network (NN). Field evaluation showed that the Pearson r between the low-cost sensor and the SHAPR instrument was 0.78. Fligner and Killeen (F-K) test indicated a statistically significant difference between the variances of the PM2.5 values by the low-cost sensor and by the SHARP instrument. Large overestimations by the low-cost sensor before calibration were observed in the field and were believed to be caused by the variation of ambient relative humidity. The root mean square error (RMSE) was 9.93 when comparing the low-cost sensor with the SHARP instrument. The calibration by the feedforward NN had the smallest RMSE of 3.91 in the test dataset, compared to the calibrations by SLR (4.91), MLR (4.65), and XGBoost (4.19). After calibrations, the F-K test using the test dataset showed that the variances of the PM2.5 values by the NN and the XGBoost and by the reference method were not statistically significantly different. From this study, we conclude that feedforward NN is a promising method to address the poor performance of the low-cost sensors for PM2.5 monitoring. In addition, the random search method for hyperparameters was demonstrated to be an efficient approach for selecting the best model structure.

Workforce Analytics: A Data-Driven Machine Learning Approach to Predict Job Change of Data Scientists

2021 ◽  
Vol 9 (2) ◽  
pp. 335-349
Author(s):  
Sohini Sengupta ◽  
Sareeta Mugde ◽  
Renuka Deshpande ◽  
Kimaya Potdar
Keyword(s):  
Machine Learning ◽  
Data Science ◽  
Machine Learning Algorithms ◽  
Support Vector ◽  
Job Change ◽  
Nearest Neighbours ◽  
Machine Learning Approach ◽  
Reducing Costs ◽  
Time Required ◽  
New Jobs

Today the total amount of data created, captured, and consumed in the world is increasing at a rapid rate, as digitally driven organizations continue to contribute to the ever- growing global data sphere. (Holst, Statista Report 2020). This data brings with it a plethora of opportunities for organizations across different sectors. Hence, their hiring outlook is shifting towards candidates who possess the abilities to decode data and generate actionable insights to gain a competitive advantage. A career in data science offers great scope and the demand for such candidates is expected to rise steeply. When companies hire for big data and data science roles, they often provide training. From an HR perspective, it is important to understand how many of them would work for the company in the future or how many look at the training with an upskilling perspective for new jobs. HR has the aim of reducing costs and time required to conduct trainings by designing courses aligning to the candidate’s interest and needs. In this paper, we explored the data based on features including demographics, education and prior experience of the candidates. We made use of machine learning algorithms, viz. Logistic Regression, Naive Bayes, K Nearest-Neighbours Classifier, Decision Trees, Random Forest, Support Vector Machine, Gradient Descent Boosting, and XGBoost to predict whether a candidate will look for a new job or will stay and work for the company. 

scholarly journals The Use of Machine Learning Algorithms in the Evaluation of the Effectiveness of Resynchronization Therapy

2022 ◽  
Vol 9 (1) ◽  
pp. 17
Author(s):  
Bartosz Krzowski ◽  
Jakub Rokicki ◽  
Renata Główczyńska ◽  
Nikola Fajkis-Zajączkowska ◽  
Katarzyna Barczewska ◽  
...  
Keyword(s):  
Artificial Intelligence ◽  
Machine Learning ◽  
Cardiac Resynchronization Therapy ◽  
Machine Learning Algorithms ◽  
Detection Sensitivity ◽  
Cardiac Resynchronization ◽  
The European Union ◽  
Resynchronization Therapy ◽  
Number Of Patients ◽  
Time Required

Background: Cardiovascular disease remains the leading cause of death in the European Union and worldwide. Constant improvement in cardiac care is leading to an increased number of patients with heart failure, which is a challenging condition in terms of clinical management. Cardiac resynchronization therapy is becoming more popular because of its grounded position in guidelines and clinical practice. However, some patients do not respond to treatment as expected. One way of assessing cardiac resynchronization therapy is with ECG analysis. Artificial intelligence is increasing in terms of everyday usability due to the possibility of everyday workflow improvement and, as a result, shortens the time required for diagnosis. A special area of artificial intelligence is machine learning. AI algorithms learn on their own based on implemented data. The aim of this study was to evaluate using artificial intelligence algorithms for detecting inadequate resynchronization therapy. Methods: A total of 1241 ECG tracings were collected from 547 cardiac department patients. All ECG signals were analyzed by three independent cardiologists. Every signal event (QRS-complex) and rhythm was manually classified by the medical team and fully reviewed by additional cardiologists. The results were divided into two parts: 80% of the results were used to train the algorithm, and 20% were used for the test (Cardiomatics, Cracow, Poland). Results: The required level of detection sensitivity of effective cardiac resynchronization therapy stimulation was achieved: 99.2% with a precision of 92.4%. Conclusions: Artificial intelligence algorithms can be a useful tool in assessing the effectiveness of resynchronization therapy.

Sign in / Sign up

Export Citation Format

Share Document