scholarly journals The Tabu_Genetic Algorithm: A Novel Method for Hyper-Parameter Optimization of Learning Algorithms

Electronics ◽  
2019 ◽  
Vol 8 (5) ◽  
pp. 579 ◽  
Author(s):  
Baosu Guo ◽  
Jingwen Hu ◽  
Wenwen Wu ◽  
Qingjin Peng ◽  
Fenghe Wu
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Parameter Optimization ◽  
Random Search ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Parameters Optimization ◽  
Bayesian Optimization ◽  
Learning Models ◽  
Machine Learning Models

Machine learning algorithms have been widely used to deal with a variety of practical problems such as computer vision and speech processing. But the performance of machine learning algorithms is primarily affected by their hyper-parameters, as without good hyper-parameter values the performance of these algorithms will be very poor. Unfortunately, for complex machine learning models like deep neural networks, it is very difficult to determine their hyper-parameters. Therefore, it is of great significance to develop an efficient algorithm for hyper-parameter automatic optimization. In this paper, a novel hyper-parameter optimization methodology is presented to combine the advantages of a Genetic Algorithm and Tabu Search to achieve the efficient search for hyper-parameters of learning algorithms. This method is defined as the Tabu_Genetic Algorithm. In order to verify the performance of the proposed algorithm, two sets of contrast experiments are conducted. The Tabu_Genetic Algorithm and other four methods are simultaneously used to search for good values of hyper-parameters of deep convolutional neural networks. Experimental results show that, compared to Random Search and Bayesian optimization methods, the proposed Tabu_Genetic Algorithm finds a better model in less time. Whether in a low-dimensional or high-dimensional space, the Tabu_Genetic Algorithm has better search capabilities as an effective method for finding the hyper-parameters of learning algorithms. The presented method in this paper provides a new solution for solving the hyper-parameters optimization problem of complex machine learning models, which will provide machine learning algorithms with better performance when solving practical problems.

scholarly journals Assessing Surface Water Flood Risks in Urban Areas Using Machine Learning

Water ◽  
2021 ◽  
Vol 13 (24) ◽  
pp. 3520
Author(s):  
Zhufeng Li ◽  
Haixing Liu ◽  
Chunbo Luo ◽  
Guangtao Fu
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Surface Water ◽  
Flood Risk ◽  
Urban Areas ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Learning Models ◽  
Machine Learning Models

Urban flooding is a devastating natural hazard for cities around the world. Flood risk mapping is a key tool in flood management. However, it is computationally expensive to produce flood risk maps using hydrodynamic models. To this end, this paper investigates the use of machine learning for the assessment of surface water flood risks in urban areas. The factors that are considered in machine learning models include coordinates, elevation, slope gradient, imperviousness, land use, land cover, soil type, substrate, distance to river, distance to road, and normalized difference vegetation index. The machine learning models are tested using the case study of Exeter, UK. The performance of machine learning algorithms, including naïve Bayes, perceptron, artificial neural networks (ANNs), and convolutional neural networks (CNNs), is compared based on a spectrum of indicators, e.g., accuracy, F-beta score, and receiver operating characteristic curve. The results obtained from the case study show that the flood risk maps can be accurately generated by the machine learning models. The performance of models on the 30-year flood event is better than 100-year and 1000-year flood events. The CNNs and ANNs outperform the other machine learning algorithms tested. This study shows that machine learning can help provide rapid flood mapping, and contribute to urban flood risk assessment and management.

scholarly journals Hyperparameter Tuning for Machine Learning Algorithms Used for Arabic Sentiment Analysis

Informatics ◽  
2021 ◽  
Vol 8 (4) ◽  
pp. 79
Author(s):  
Enas Elgeldawi ◽  
Awny Sayed ◽  
Ahmed R. Galal ◽  
Alaa M. Zaki
Keyword(s):  
Machine Learning ◽  
Sentiment Analysis ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Bayesian Optimization ◽  
Support Vector ◽  
Learning Models ◽  
Before And After ◽  
Tuning Process ◽  
Machine Learning Models

Machine learning models are used today to solve problems within a broad span of disciplines. If the proper hyperparameter tuning of a machine learning classifier is performed, significantly higher accuracy can be obtained. In this paper, a comprehensive comparative analysis of various hyperparameter tuning techniques is performed; these are Grid Search, Random Search, Bayesian Optimization, Particle Swarm Optimization (PSO), and Genetic Algorithm (GA). They are used to optimize the accuracy of six machine learning algorithms, namely, Logistic Regression (LR), Ridge Classifier (RC), Support Vector Machine Classifier (SVC), Decision Tree (DT), Random Forest (RF), and Naive Bayes (NB) classifiers. To test the performance of each hyperparameter tuning technique, the machine learning models are used to solve an Arabic sentiment classification problem. Sentiment analysis is the process of detecting whether a text carries a positive, negative, or neutral sentiment. However, extracting such sentiment from a complex derivational morphology language such as Arabic has been always very challenging. The performance of all classifiers is tested using our constructed dataset both before and after the hyperparameter tuning process. A detailed analysis is described, along with the strengths and limitations of each hyperparameter tuning technique. The results show that the highest accuracy was given by SVC both before and after the hyperparameter tuning process, with a score of 95.6208 obtained when using Bayesian Optimization.

Machine Learning Models for Finger Bend Evaluation using Implemented Low cost Flex Sensor

2021 ◽  
Vol 9 (VI) ◽  
pp. 3605-3611
Author(s):  
Pratyush Kaware
Keyword(s):  
Machine Learning ◽  
Support Vector Machine ◽  
Low Cost ◽  
Learning Algorithms ◽  
Cost Effective ◽  
Machine Learning Algorithms ◽  
Support Vector ◽  
Learning Models ◽  
Machine Learning Models

In this paper a cost-effective sensor has been implemented to read finger bend signals, by attaching the sensor to a finger, so as to classify them based on the degree of bent as well as the joint about which the finger was being bent. This was done by testing with various machine learning algorithms to get the most accurate and consistent classifier. Finally, we found that Support Vector Machine was the best algorithm suited to classify our data, using we were able predict live state of a finger, i.e., the degree of bent and the joints involved. The live voltage values from the sensor were transmitted using a NodeMCU micro-controller which were converted to digital and uploaded on a database for analysis.

scholarly journals Imputation of PaO2 from SpO2 values from the MIMIC-III Critical Care Database Using Machine-Learning Based Algorithms

2021 ◽  
Author(s):  
Shuangxia Ren ◽  
Jill Zupetic ◽  
Mehdi Nouraie ◽  
Xinghua Lu ◽  
Richard D. Boyce ◽  
...  
Keyword(s):  
Machine Learning ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Superior Performance ◽  
Learning Models ◽  
Mechanically Ventilated ◽  
Mechanically Ventilated Patients ◽  
Mimic Iii ◽  
Ventilated Patients ◽  
Machine Learning Models

AbstractBackgroundThe partial pressure of oxygen (PaO2)/fraction of oxygen delivered (FIO2) ratio is the reference standard for assessment of hypoxemia in mechanically ventilated patients. Non-invasive monitoring with the peripheral saturation of oxygen (SpO2) is increasingly utilized to estimate PaO2 because it does not require invasive sampling. Several equations have been reported to impute PaO2/FIO2 from SpO2 /FIO2. However, machine-learning algorithms to impute the PaO2 from the SpO2 has not been compared to published equations.Research QuestionHow do machine learning algorithms perform at predicting the PaO2 from SpO2 compared to previously published equations?MethodsThree machine learning algorithms (neural network, regression, and kernel-based methods) were developed using 7 clinical variable features (n=9,900 ICU events) and subsequently 3 features (n=20,198 ICU events) as input into the models from data available in mechanically ventilated patients from the Medical Information Mart for Intensive Care (MIMIC) III database. As a regression task, the machine learning models were used to impute PaO2 values. As a classification task, the models were used to predict patients with moderate-to-severe hypoxemic respiratory failure based on a clinically relevant cut-off of PaO2/FIO2 ≤ 150. The accuracy of the machine learning models was compared to published log-linear and non-linear equations. An online imputation calculator was created.ResultsCompared to seven features, three features (SpO2, FiO2 and PEEP) were sufficient to impute PaO2/FIO2 ratio using a large dataset. Any of the tested machine learning models enabled imputation of PaO2/FIO2 from the SpO2/FIO2 with lower error and had greater accuracy in predicting PaO2/FIO2 ≤ 150 compared to published equations. Using three features, the machine learning models showed superior performance in imputing PaO2 across the entire span of SpO2 values, including those ≥ 97%.InterpretationThe improved performance shown for the machine learning algorithms suggests a promising framework for future use in large datasets.

scholarly journals Energy demand prediction with machine learning supported by auto-tuning: a case study

2021 ◽  
Vol 2069 (1) ◽  
pp. 012143
Author(s):  
Sorana Ozaki ◽  
Ryozo Ooka ◽  
Shintaro Ikeda
Keyword(s):  
Machine Learning ◽  
Energy Demand ◽  
Random Search ◽  
Bayesian Optimization ◽  
Learning Models ◽  
Time Step ◽  
Efficient Operation ◽  
Tuning Methods ◽  
Machine Learning Models

Abstract The operational energy of buildings is making up one of the highest proportions of life-cycle carbon emissions. A more efficient operation of facilities would result in significant energy savings but necessitates computational models to predict a building’s future energy demands with high precision. To this end, various machine learning models have been proposed in recent years. These models’ prediction accuracies, however, strongly depend on their internal structure and hyperparameters. The time demand and expertise required for their finetuning call for a more efficient solution. In the context of a case study, this paper describes the relationship between a machine learning model’s prediction accuracy and its hyperparameters. Based on time-stamped recordings of outdoor temperatures and electricity demands of a hospital in Japan, recorded every 30 minutes for more than four years, using a deep neural network (DNN) ensemble model, electricity demands were predicted for sixty time steps to follow. Specifically, we used automatic hyperparameter tuning methods, such as grid search, random search, and Bayesian optimization. A single time step ahead, all tuning methods reduced the RSME to less than 50%, compared to non-optimized tuning. The results attest to machine learning models’ reliance on hyperparameters and the effectiveness of their automatic tuning.

Machine Learning Reactivity in the Chemical Space Surrounding Vaska's Complex

2019 ◽  
Author(s):  
Pascal Friederich ◽  
Gabriel dos Passos Gomes ◽  
Riccardo De Bin ◽  
Alan Aspuru-Guzik ◽  
David Balcells
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Chemical Space ◽  
Bayesian Optimization ◽  
Gradient Boosting ◽  
Learning Models ◽  
Hydrogen Activation ◽  
Activation Barriers ◽  
Machine Learning Models ◽  
Vaska's Complex

Machine learning models, including neural networks, Bayesian optimization, gradient boosting and Gaussian processes, were trained with DFT data for the accurate, affordable and explainable prediction of hydrogen activation barriers in the chemical space surrounding Vaska's complex.

scholarly journals Using Big Data-machine learning models for diabetes prediction and flight delays analytics

2020 ◽  
Vol 7 (1) ◽  
Author(s):  
Thérence Nibareke ◽  
Jalal Laassiri
Keyword(s):  
Machine Learning ◽  
Big Data ◽  
Linear Regression ◽  
Decision Tree ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Smart Devices ◽  
Learning Models ◽  
Flight Delays ◽  
Machine Learning Models

Abstract Introduction Nowadays large data volumes are daily generated at a high rate. Data from health system, social network, financial, government, marketing, bank transactions as well as the censors and smart devices are increasing. The tools and models have to be optimized. In this paper we applied and compared Machine Learning algorithms (Linear Regression, Naïve bayes, Decision Tree) to predict diabetes. Further more, we performed analytics on flight delays. The main contribution of this paper is to give an overview of Big Data tools and machine learning models. We highlight some metrics that allow us to choose a more accurate model. We predict diabetes disease using three machine learning models and then compared their performance. Further more we analyzed flight delay and produced a dashboard which can help managers of flight companies to have a 360° view of their flights and take strategic decisions. Case description We applied three Machine Learning algorithms for predicting diabetes and we compared the performance to see what model give the best results. We performed analytics on flights datasets to help decision making and predict flight delays. Discussion and evaluation The experiment shows that the Linear Regression, Naive Bayesian and Decision Tree give the same accuracy (0.766) but Decision Tree outperforms the two other models with the greatest score (1) and the smallest error (0). For the flight delays analytics, the model could show for example the airport that recorded the most flight delays. Conclusions Several tools and machine learning models to deal with big data analytics have been discussed in this paper. We concluded that for the same datasets, we have to carefully choose the model to use in prediction. In our future works, we will test different models in other fields (climate, banking, insurance.).

scholarly journals SCADA System Testbed for Cybersecurity Research Using Machine Learning Approach

2018 ◽  
Vol 10 (8) ◽  
pp. 76 ◽  
Author(s):  
Marcio Teixeira ◽  
Tara Salman ◽  
Maede Zolanvari ◽  
Raj Jain ◽  
Nader Meskin ◽  
...  
Keyword(s):  
Machine Learning ◽  
Supervisory Control ◽  
Network Traffic ◽  
Learning Algorithms ◽  
Cyber Attacks ◽  
Machine Learning Algorithms ◽  
Learning Models ◽  
Scada System ◽  
Machine Learning Approach ◽  
Machine Learning Models

This paper presents the development of a Supervisory Control and Data Acquisition (SCADA) system testbed used for cybersecurity research. The testbed consists of a water storage tank’s control system, which is a stage in the process of water treatment and distribution. Sophisticated cyber-attacks were conducted against the testbed. During the attacks, the network traffic was captured, and features were extracted from the traffic to build a dataset for training and testing different machine learning algorithms. Five traditional machine learning algorithms were trained to detect the attacks: Random Forest, Decision Tree, Logistic Regression, Naïve Bayes and KNN. Then, the trained machine learning models were built and deployed in the network, where new tests were made using online network traffic. The performance obtained during the training and testing of the machine learning models was compared to the performance obtained during the online deployment of these models in the network. The results show the efficiency of the machine learning models in detecting the attacks in real time. The testbed provides a good understanding of the effects and consequences of attacks on real SCADA environments.

scholarly journals Software Engineering for Machine Learning in Health Informatics

2019 ◽  
Author(s):  
Mohammed Moreb ◽  
Oguz Ata
Keyword(s):  
Machine Learning ◽  
Software Engineering ◽  
Health Informatics ◽  
Learning Algorithm ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Prototype System ◽  
Machine Learning Algorithm ◽  
Learning Models ◽  
Machine Learning Models

Abstract Background We propose a novel framework for health Informatics: framework and methodology of Software Engineering for machine learning in Health Informatics (SEMLHI). This framework shed light on its features, that allow users to study and analyze the requirements, determine the function of objects related to the system and determine the machine learning algorithms that will be used for the dataset.Methods Based on original data that collected from the hospital in Palestine government in the past three years, first the data validated and all outlier removed, analyzed using develop framework in order to compare ML provide patients with real-time. Our proposed module comparison with three Systems Engineering Methods Vee, agile and SEMLHI. The result used by implement prototype system, which require machine learning algorithm, after development phase, questionnaire deliver to developer to indicate the result using three methodology. SEMLHI framework, is composed into four components: software, machine learning model, machine learning algorithms, and health informatics data, Machine learning Algorithm component used five algorithms use to evaluate the accuracy for machine learning models on component.Results we compare our approach with the previously published systems in terms of performance to evaluate the accuracy for machine learning models, the results of accuracy with different algorithms applied for 750 case, linear SVG have about 0.57 value compared with KNeighbors classifier, logistic regression, multinomial NB, random forest classifier. This research investigates the interaction between SE, and ML within the context of health informatics, our proposed framework define the methodology for developers to analyzing and developing software for the health informatic model, and create a space, in which software engineering, and ML experts could work on the ML model lifecycle, on the disease level and the subtype level.Conclusions This article is an ongoing effort towards defining and translating an existing research pipeline into four integrated modules, as framework system using the dataset from healthcare to reduce cost estimation by using a new suggested methodology. The framework is available as open source software, licensed under GNU General Public License Version 3 to encourage others to contribute to the future development of the SEMLHI framework.

scholarly journals Integrating Feature Selection and Multiclass Classification for Sport Result Prediction

2019 ◽  
Vol 9 (1) ◽  
pp. 1339-1342
Keyword(s):  
Machine Learning ◽  
Feature Selection ◽  
Learning Algorithms ◽  
Multiclass Classification ◽  
Machine Learning Algorithms ◽  
Classification Algorithms ◽  
Learning Models ◽  
Sports Fans ◽  
Player Performance ◽  
Machine Learning Models

Machine learning (ML) has become the most predominant methodology that shows good results in the classification and prediction domains. Predictive systems are being employed to predict events and its results in almost every walk of life. The field of prediction in sports is gaining importance as there is a huge community of betters and sports fans. Moreover team owners and club managers are struggling for Machine learning models that could be used for formulating strategies to win matches. Numerous factors such as results of previous matches, indicators of player performance and opponent information are required to build these models. This paper provides an analysis of such key models focusing on application of machine learning algorithms to sport result prediction. The results obtained helped us to elucidate the best combination of feature selection and classification algorithms that render maximum accuracy in sport result prediction.

Sign in / Sign up

Export Citation Format

Share Document