Statistical Study of Machine Learning Algorithms Using Parametric and Non-Parametric Tests

2020 ◽  
Vol 11 (3) ◽  
pp. 80-105 ◽  
Author(s):  
Vijay M. Khadse ◽  
Parikshit Narendra Mahalle ◽  
Gitanjali R. Shinde
Keyword(s):  
Machine Learning ◽  
Optimal Algorithm ◽  
Smart Cities ◽  
Learning Algorithms ◽  
Performance Measure ◽  
Machine Learning Algorithms ◽  
Training Data ◽  
Statistical Validation ◽  
The Difference ◽  
Validation Of Results

The emerging area of the internet of things (IoT) generates a large amount of data from IoT applications such as health care, smart cities, etc. This data needs to be analyzed in order to derive useful inferences. Machine learning (ML) plays a significant role in analyzing such data. It becomes difficult to select optimal algorithm from the available set of algorithms/classifiers to obtain best results. The performance of algorithms differs when applied to datasets from different application domains. In learning, it is difficult to understand if the difference in performance is real or due to random variation in test data, training data, or internal randomness of the learning algorithms. This study takes into account these issues during a comparison of ML algorithms for binary and multivariate classification. It helps in providing guidelines for statistical validation of results. The results obtained show that the performance measure of accuracy for one algorithm differs by critical difference (CD) than others over binary and multivariate datasets obtained from different application domains.

scholarly journals Utilizing Statistical Tests for Comparing Machine Learning Algorithms

2021 ◽  
pp. 69-74
Author(s):  
Hozan Khalid Hamarashid
Keyword(s):  
Machine Learning ◽  
Prediction Models ◽  
Hypothesis Test ◽  
Statistical Tests ◽  
Learning Algorithms ◽  
Performance Measure ◽  
Machine Learning Algorithms ◽  
Statistical Hypothesis ◽  
The Mean ◽  
The Difference

The mean result of machine learning models is determined by utilizing k-fold cross-validation. The algorithm with the best average performance should surpass those with the poorest. But what if the difference in average outcomes is the consequence of a statistical anomaly? To conduct whether or not the mean result differences between two algorithms is genuine then statistical hypothesis test is utilized. Using statistical hypothesis testing, this study will demonstrate how to compare machine learning algorithms. The output of several machine learning algorithms or simulation pipelines is compared during model selection. The model that performs the best based on your performance measure becomes the last model, which can be utilized to make predictions on new data. With classification and regression prediction models it can be conducted by utilizing traditional machine learning and deep learning methods. The difficulty is to identify whether or not the difference between two models is accurate.

Optimization of Diabetes Training DATA using Machine Learning Algorithms

2018 ◽  
Vol 6 (2) ◽  
pp. 283-286
Author(s):  
M. Samba Siva Rao ◽  
◽  
M.Yaswanth . ◽  
K. Raghavendra Swamy ◽  
◽  
...  
Keyword(s):  
Machine Learning ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Training Data

scholarly journals Machine Learning Model of Dimensionless Numbers to Predict Flow Patterns and Droplet Characteristics for Two-Phase Digital Flows

2021 ◽  
Vol 11 (9) ◽  
pp. 4251
Author(s):  
Jinsong Zhang ◽  
Shuai Zhang ◽  
Jianhua Zhang ◽  
Zhiliang Wang
Keyword(s):  
Machine Learning ◽  
Learning Algorithms ◽  
Digital Microfluidics ◽  
Flow Patterns ◽  
Machine Learning Algorithms ◽  
Dimensionless Numbers ◽  
Two Phase ◽  
The Difference ◽  
Input Variables ◽  
Digital Microfluidic

In the digital microfluidic experiments, the droplet characteristics and flow patterns are generally identified and predicted by the empirical methods, which are difficult to process a large amount of data mining. In addition, due to the existence of inevitable human invention, the inconsistent judgment standards make the comparison between different experiments cumbersome and almost impossible. In this paper, we tried to use machine learning to build algorithms that could automatically identify, judge, and predict flow patterns and droplet characteristics, so that the empirical judgment was transferred to be an intelligent process. The difference on the usual machine learning algorithms, a generalized variable system was introduced to describe the different geometry configurations of the digital microfluidics. Specifically, Buckingham’s theorem had been adopted to obtain multiple groups of dimensionless numbers as the input variables of machine learning algorithms. Through the verification of the algorithms, the SVM and BPNN algorithms had classified and predicted the different flow patterns and droplet characteristics (the length and frequency) successfully. By comparing with the primitive parameters system, the dimensionless numbers system was superior in the predictive capability. The traditional dimensionless numbers selected for the machine learning algorithms should have physical meanings strongly rather than mathematical meanings. The machine learning algorithms applying the dimensionless numbers had declined the dimensionality of the system and the amount of computation and not lose the information of primitive parameters.

Tebyan: Fake News Detection System (Preprint)

2021 ◽  
Author(s):  
Lamya Alderywsh ◽  
Aseel Aldawood ◽  
Ashwag Alasmari ◽  
Farah Aldeijy ◽  
Ghadah Alqubisy ◽  
...  
Keyword(s):  
Machine Learning ◽  
Arab World ◽  
Detection System ◽  
Learning Algorithms ◽  
Performance Measure ◽  
Machine Learning Algorithms ◽  
Svm Classifier ◽  
Fake News ◽  
Typical Type ◽  
Performance Results

BACKGROUND There is a serious threat from fake news spreading in technologically advanced societies, including those in the Arab world, via deceptive machine-generated text. In the last decade, Arabic fake news identification has gained increased attention, and numerous detection approaches have revealed some ability to find fake news throughout various data sources. Nevertheless, many existing approaches overlook recent advancements in fake news detection, explicitly to incorporate machine learning algorithms system. OBJECTIVE Tebyan project aims to address the problem of fake news by developing a fake news detection system that employs machine learning algorithms to detect whether the news is fake or real in the context of Arab world. METHODS The project went through numerous phases using an iterative methodology to develop the system. This study analysis incorporated numerous stages using an iterative method to develop the system of misinformation and contextualize fake news regarding society's information. It consists of implementing the machine learning algorithms system using Python to collect genuine and fake news datasets. The study also assesses how information-exchanging behaviors can minimize and find the optimal source of authentication of the emergent news through system testing approaches. RESULTS The study revealed that the main deliverable of this project is the Tebyan system in the community, which allows the user to ensure the credibility of news in Arabic newspapers. It showed that the SVM classifier, on average, exhibited the highest performance results, resulting in 90% in every performance measure of sources. Moreover, the results indicate the second-best algorithm is the linear SVC since it resulted in 90% in performance measure with the societies' typical type of fake information. CONCLUSIONS The study concludes that conducting a system with machine learning algorithms using Python programming language allows the rapid measures of the users' perception to comment and rate the credibility result and subscribing to news email services.

PSIX-15 Assessment of machine learning algorithms for prediction of Aleutian disease in American mink

2021 ◽  
Vol 99 (Supplement_3) ◽  
pp. 264-265
Author(s):  
Duy Ngoc Do ◽  
Guoyu Hu ◽  
Younes Miar
Keyword(s):  
Machine Learning ◽  
Random Forest ◽  
Linear Models ◽  
American Mink ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Training Data ◽  
Enzyme Linked Immunosorbent Assay ◽  
Linear Discriminant ◽  
Machine Learning Classification

Abstract American mink (Neovison vison) is the major source of fur for the fur industries worldwide and Aleutian disease (AD) is causing severe financial losses to the mink industry. Different methods have been used to diagnose the AD in mink, but the combination of several methods can be the most appropriate approach for the selection of AD resilient mink. Iodine agglutination test (IAT) and counterimmunoelectrophoresis (CIEP) methods are commonly employed in test-and-remove strategy; meanwhile, enzyme-linked immunosorbent assay (ELISA) and packed-cell volume (PCV) methods are complementary. However, using multiple methods are expensive; and therefore, hindering the corrected use of AD tests in selection. This research presented the assessments of the AD classification based on machine learning algorithms. The Aleutian disease was tested on 1,830 individuals using these tests in an AD positive mink farm (Canadian Centre for Fur Animal Research, NS, Canada). The accuracy of classification for CIEP was evaluated based on the sex information, and IAT, ELISA and PCV test results implemented in seven machine learning classification algorithms (Random Forest, Artificial Neural Networks, C50Tree, Naive Bayes, Generalized Linear Models, Boost, and Linear Discriminant Analysis) using the Caret package in R. The accuracy of prediction varied among the methods. Overall, the Random Forest was the best-performing algorithm for the current dataset with an accuracy of 0.89 in the training data and 0.94 in the testing data. Our work demonstrated the utility and relative ease of using machine learning algorithms to assess the CIEP information, and consequently reducing the cost of AD tests. However, further works require the inclusion of production and reproduction information in the models and extension of phenotypic collection to increase the accuracy of current methods.

Comparing Machine Learning Models for the Predictions of Speed in Smart Transportation Systems

2022 ◽  
pp. 34-46
Author(s):  
Amtul Waheed ◽  
Jana Shafi ◽  
Saritha V.
Keyword(s):  
Machine Learning ◽  
Learning Algorithm ◽  
Smart Cities ◽  
Learning Algorithms ◽  
High Capacity ◽  
Transportation Systems ◽  
Machine Learning Algorithms ◽  
Smart Transportation ◽  
Data Propagation ◽  
Machine Learning Models

In today's world of advanced technologies in IoT and ITS in smart cities scenarios, there are many different projections such as improved data propagation in smart roads and cooperative transportation networks, autonomous and continuously connected vehicles, and low latency applications in high capacity environments and heterogeneous connectivity and speed. This chapter presents the performance of the speed of vehicles on roadways employing machine learning methods. Input variable for each learning algorithm is the density that is measured as vehicle per mile and volume that is measured as vehicle per hour. And the result shows that the output variable is the speed that is measured as miles per hour represent the performance of each algorithm. The performance of machine learning algorithms is calculated by comparing the result of predictions made by different machine learning algorithms with true speed using the histogram. A result recommends that speed is varying according to the histogram.

scholarly journals Identification of Leukemia Subtypes from Microscopic Images Using Convolutional Neural Network

Diagnostics ◽  
2019 ◽  
Vol 9 (3) ◽  
pp. 104 ◽  
Author(s):  
Ahmed ◽  
Yigit ◽  
Isik ◽  
Alpkocak
Keyword(s):  
Machine Learning ◽  
Data Augmentation ◽  
Nearest Neighbor ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Training Data ◽  
Support Vector ◽  
K Nearest Neighbor ◽  
Data Set ◽  
Leukemia Data

Leukemia is a fatal cancer and has two main types: Acute and chronic. Each type has two more subtypes: Lymphoid and myeloid. Hence, in total, there are four subtypes of leukemia. This study proposes a new approach for diagnosis of all subtypes of leukemia from microscopic blood cell images using convolutional neural networks (CNN), which requires a large training data set. Therefore, we also investigated the effects of data augmentation for an increasing number of training samples synthetically. We used two publicly available leukemia data sources: ALL-IDB and ASH Image Bank. Next, we applied seven different image transformation techniques as data augmentation. We designed a CNN architecture capable of recognizing all subtypes of leukemia. Besides, we also explored other well-known machine learning algorithms such as naive Bayes, support vector machine, k-nearest neighbor, and decision tree. To evaluate our approach, we set up a set of experiments and used 5-fold cross-validation. The results we obtained from experiments showed that our CNN model performance has 88.25% and 81.74% accuracy, in leukemia versus healthy and multiclass classification of all subtypes, respectively. Finally, we also showed that the CNN model has a better performance than other wellknown machine learning algorithms.

Mapping (un)certainty of machine learning-based spatial prediction models based on predictor space distances

2020 ◽  
Author(s):  
Hanna Meyer ◽  
Edzer Pebesma
Keyword(s):  
Machine Learning ◽  
Spatial Patterns ◽  
Environmental Science ◽  
Prediction Models ◽  
Learning Algorithms ◽  
Predictor Variable ◽  
Spatial Prediction ◽  
Machine Learning Algorithms ◽  
Training Data ◽  
Field Samples

<p>Spatial mapping is an important task in environmental science to reveal spatial patterns and changes of the environment. In this context predictive modelling using flexible machine learning algorithms has become very popular. However, looking at the diversity of modelled (global) maps of environmental variables, there might be increasingly the impression that machine learning is a magic tool to map everything. Recently, the reliability of such maps have been increasingly questioned, calling for a reliable quantification of uncertainties.</p><p>Though spatial (cross-)validation allows giving a general error estimate for the predictions, models are usually applied to make predictions for a much larger area or might even be transferred to make predictions for an area where they were not trained on. But by making predictions on heterogeneous landscapes, there will be areas that feature environmental properties that have not been observed in the training data and hence not learned by the algorithm. This is problematic as most machine learning algorithms are weak in extrapolations and can only make reliable predictions for environments with conditions the model has knowledge about. Hence predictions for environmental conditions that differ significantly from the training data have to be considered as uncertain.</p><p>To approach this problem, we suggest a measure of uncertainty that allows identifying locations where predictions should be regarded with care. The proposed uncertainty measure is based on distances to the training data in the multidimensional predictor variable space. However, distances are not equally relevant within the feature space but some variables are more important than others in the machine learning model and hence are mainly responsible for prediction patterns. Therefore, we weight the distances by the model-derived importance of the predictors. </p><p>As a case study we use a simulated area-wide response variable for Europe, bio-climatic variables as predictors, as well as simulated field samples. Random Forest is applied as algorithm to predict the simulated response. The model is then used to make predictions for entire Europe. We then calculate the corresponding uncertainty and compare it to the area-wide true prediction error. The results show that the uncertainty map reflects the patterns in the true error very well and considerably outperforms ensemble-based standard deviations of predictions as indicator for uncertainty.</p><p>The resulting map of uncertainty gives valuable insights into spatial patterns of prediction uncertainty which is important when the predictions are used as a baseline for decision making or subsequent environmental modelling. Hence, we suggest that a map of distance-based uncertainty should be given in addition to prediction maps.</p>

An Assessment of Machine Learning Techniques for Predicting Turbine Airfoil Component Temperatures, Using FEA Simulations for Training Data

2019 ◽  
Author(s):  
James A. Tallman ◽  
Michal Osusky ◽  
Nick Magina ◽  
Evan Sewall
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Artificial Neural Network ◽  
Surrogate Model ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Training Data ◽  
Machine Learning Techniques ◽  
Learning Techniques ◽  
Artificial Neural

Abstract This paper provides an assessment of three different machine learning techniques for accurately reproducing a distributed temperature prediction of a high-pressure turbine airfoil. A three-dimensional Finite Element Analysis thermal model of a cooled turbine airfoil was solved repeatedly (200 instances) for various operating point settings of the corresponding gas turbine engine. The response surface created by the repeated solutions was fed into three machine learning algorithms and surrogate model representations of the FEA model’s response were generated. The machine learning algorithms investigated were a Gaussian Process, a Boosted Decision Tree, and an Artificial Neural Network. Additionally, a simple Linear Regression surrogate model was created for comparative purposes. The Artificial Neural Network model proved to be the most successful at reproducing the FEA model over the range of operating points. The mean and standard deviation differences between the FEA and the Neural Network models were 15% and 14% of a desired accuracy threshold, respectively. The Digital Thread for Design (DT4D) was used to expedite all model execution and machine learning training. A description of DT4D is also provided.

Creep Rupture Forecasting

2014 ◽  
Vol 2 (2) ◽  
pp. 1-25 ◽  
Author(s):  
Stylianos Chatzidakis ◽  
Miltiadis Alamaniotis ◽  
Lefteri H. Tsoukalas
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Process Model ◽  
Production Systems ◽  
Learning Algorithms ◽  
Difficult Problem ◽  
Network Size ◽  
Creep Rupture ◽  
Machine Learning Algorithms ◽  
Training Data

Creep rupture is becoming increasingly one of the most important problems affecting behavior and performance of power production systems operating in high temperature environments and potentially under irradiation as is the case of nuclear reactors. Creep rupture forecasting and estimation of the useful life is required to avoid unanticipated component failure and cost ineffective operation. Despite the rigorous investigations of creep mechanisms and their effect on component lifetime, experimental data are sparse rendering the time to rupture prediction a rather difficult problem. An approach for performing creep rupture forecasting that exploits the unique characteristics of machine learning algorithms is proposed herein. The approach seeks to introduce a mechanism that will synergistically exploit recent findings in creep rupture with the state-of-the-art computational paradigm of machine learning. In this study, three machine learning algorithms, namely General Regression Neural Networks, Artificial Neural Networks and Gaussian Processes, were employed to capture the underlying trends and provide creep rupture forecasting. The current implementation is demonstrated and evaluated on actual experimental creep rupture data. Results show that the Gaussian process model based on the Matérn kernel achieved the best overall prediction performance (56.38%). Significant dependencies exist on the number of training data, neural network size, kernel selection and whether interpolation or extrapolation is performed.

Sign in / Sign up

Export Citation Format

Share Document