scholarly journals Overfitting, Model Tuning, and Evaluation of Prediction Performance

2022 ◽  
pp. 109-139
Author(s):  
Osval Antonio Montesinos López ◽  
Abelardo Montesinos López ◽  
Jose Crossa
Keyword(s):  
Machine Learning ◽  
Predictive Modeling ◽  
Learning Model ◽  
Prediction Performance ◽  
Training Data ◽  
Statistical Machine Learning ◽  
Machine Learning Model ◽  
Data Points ◽  
The Difference ◽  
Model Tuning

AbstractThe overfitting phenomenon happens when a statistical machine learning model learns very well about the noise as well as the signal that is present in the training data. On the other hand, an underfitted phenomenon occurs when only a few predictors are included in the statistical machine learning model that represents the complete structure of the data pattern poorly. This problem also arises when the training data set is too small and thus an underfitted model does a poor job of fitting the training data and unsatisfactorily predicts new data points. This chapter describes the importance of the trade-off between prediction accuracy and model interpretability, as well as the difference between explanatory and predictive modeling: Explanatory modeling minimizes bias, whereas predictive modeling seeks to minimize the combination of bias and estimation variance. We assess the importance and different methods of cross-validation as well as the importance and strategies of tuning that are key to the successful use of some statistical machine learning methods. We explain the most important metrics for evaluating the prediction performance for continuous, binary, categorical, and count response variables.

Building a predictive model for warfarin dosing via machine learning

2020 ◽  
Vol 41 (Supplement_2) ◽  
Author(s):  
O Tan ◽  
J.Z Wu ◽  
K.K Yeo ◽  
S.Y.A Lee ◽  
J.S Hon ◽  
...  
Keyword(s):  
Machine Learning ◽  
Missing Values ◽  
Warfarin Dose ◽  
Learning Model ◽  
Training Data ◽  
Multiple Drug ◽  
Data Set ◽  
Machine Learning Model ◽  
Data Points ◽  
Test Sets

Abstract Background Warfarin titration via International Normalised Ratio (INR) monitoring can be a challenge for patients on long term anticoagulation due to its multiple drug interactions, long half life and patient's individual reponse, yet critically important due to its narrow therapeutic index. Machine learning may have a role in learning and replicating existing warfarin prescribing practices, to be potentially incorporated into an automated warfarin titration model. Purpose We aim to explore the feasibility of using machine learning to develop a model that can learn and predict actual warfarin titration practices. Methods A retrospective dataset of 4,247 patients with 48,895 data points of INR values were obtained from our institutional database. Patients who had less than 5 visits recorded, invalid or missing values were excluded. Variables studied included age, warfarin indication, warfarin dose, target INR range, actual INR values, time between titration and time in therapeutic range (TTR as defined by the Rosenthal formula). The machine learning model was developed on an unbiased training data set (1,805 patients), further refined on a handpicked balanced validation set (400 patients), before being evaluated on two balanced test sets of 100 patients each. The test sets were handpicked based on the criteria of TTR (“in vs out of range”) and stability of INR results (“low vs high fluctuation”) (Table 1). Given the time series nature of the data, a Recurrent Neural Network (RNN) was chosen to learn warfarin prescription practices. Long-short term memory (LSTM) cells were further employed to address the problem of time gaps between warfarin titration visits which could result in vanishing gradients. Results A total of 2,163 patients with 42,622 data points were studied (mean age 65±11.7 years, 54.7% male). The mean TTR was 65.4%. The total warfarin dose per week as predicted by the RNN was compared with actual total warfarin dose per week prescribed for each patient in the test sets. The coefficient of determination for the RNN in the “in vs out of range” and “low vs high fluctuation” test sets were 0.941 and 0.962 respectively (Figure 1). Conclusion This proof of concept study demonstrated that a RNN based machine learning model was able to learn and predict warfarin dosage changes with reasonable accuracy. The findings merit further evaluation of the potential use of machine learning in an automated warfarin titration model. Funding Acknowledgement Type of funding source: None

scholarly journals A Novel XGBoost Method to Infer the Primary Lesion of 20 Solid Tumor Types From Gene Expression Data

2021 ◽  
Vol 12 ◽  
Author(s):  
Sijie Chen ◽  
Wenjing Zhou ◽  
Jinghui Tu ◽  
Jian Li ◽  
Bo Wang ◽  
...  
Keyword(s):  
Machine Learning ◽  
Learning Model ◽  
Training Data ◽  
Diagnostic Efficiency ◽  
Metastatic Tumors ◽  
Pathological Conditions ◽  
Machine Learning Model ◽  
Independent Test ◽  
Tumor Types ◽  
Fold Cross Validation

PurposeEstablish a suitable machine learning model to identify its primary lesions for primary metastatic tumors in an integrated learning approach, making it more accurate to improve primary lesions’ diagnostic efficiency.MethodsAfter deleting the features whose expression level is lower than the threshold, we use two methods to perform feature selection and use XGBoost for classification. After the optimal model is selected through 10-fold cross-validation, it is verified on an independent test set.ResultsSelecting features with around 800 genes for training, theR2-score of a 10-fold CV of training data can reach 96.38%, and theR2-score of test data can reach 83.3%.ConclusionThese findings suggest that by combining tumor data with machine learning methods, each cancer has its corresponding classification accuracy, which can be used to predict primary metastatic tumors’ location. The machine-learning-based method can be used as an orthogonal diagnostic method to judge the machine learning model processing and clinical actual pathological conditions.

scholarly journals Predicting venous thromboembolism in hospitalized trauma patients: a combination of the Caprini score and data-driven machine learning model

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Lingxiao He ◽  
Lei Luo ◽  
Xiaoling Hou ◽  
Dengbin Liao ◽  
Ran Liu ◽  
...  
Keyword(s):  
Machine Learning ◽  
Venous Thromboembolism ◽  
Prediction Model ◽  
Learning Model ◽  
Prediction Performance ◽  
Machine Learning Algorithms ◽  
Screening Methods ◽  
Trauma Patients ◽  
Machine Learning Model ◽  
Positive Rate

Abstract Background Venous thromboembolism (VTE) is a common complication of hospitalized trauma patients and has an adverse impact on patient outcomes. However, there is still a lack of appropriate tools for effectively predicting VTE for trauma patients. We try to verify the accuracy of the Caprini score for predicting VTE in trauma patients, and further improve the prediction through machine learning algorithms. Methods We retrospectively reviewed emergency trauma patients who were admitted to a trauma center in a tertiary hospital from September 2019 to March 2020. The data in the patient’s electronic health record (EHR) and the Caprini score were extracted, combined with multiple feature screening methods and the random forest (RF) algorithm to constructs the VTE prediction model, and compares the prediction performance of (1) using only Caprini score; (2) using EHR data to build a machine learning model; (3) using EHR data and Caprini score to build a machine learning model. True Positive Rate (TPR), False Positive Rate (FPR), Area Under Curve (AUC), accuracy, and precision were reported. Results The Caprini score shows a good VTE prediction effect on the trauma hospitalized population when the cut-off point is 11 (TPR = 0.667, FPR = 0.227, AUC = 0.773), The best prediction model is LASSO+RF model combined with Caprini Score and other five features extracted from EHR data (TPR = 0.757, FPR = 0.290, AUC = 0.799). Conclusion The Caprini score has good VTE prediction performance in trauma patients, and the use of machine learning methods can further improve the prediction performance.

Mud-Gas Breakthrough Equinor Develops Real-Time Reservoir-Fluid Identification

2021 ◽  
Vol 73 (02) ◽  
pp. 37-39
Author(s):  
Trent Jacobs
Keyword(s):  
Machine Learning ◽  
Real Time ◽  
Oil And Gas ◽  
Partial Information ◽  
Learning Model ◽  
Identification System ◽  
Fluid Identification ◽  
Reservoir Fluid ◽  
Machine Learning Model ◽  
The Difference

For all that logging-while-drilling has provided since its wide-spread adoption in the 1980s, there is one thing on the industry’s wish list that it could never offer: an accurate way to tell the difference between oil and gas. A new technology created by petrotechnicals at Equinor, however, has made this possible. The innovation could be thought of as a pseudo-log, but Equinor is describing it as a reservoir-fluid-identification system. Using an internally developed machine-learning model, it compares a database of more than 4,000 reservoir samples against the real-time analysis of the mud gas that flows up a well as it is drilled. Crunched out of the technology’s various hardware and software components is a prediction on the gas/oil ratio (GOR) that the rock being drilled through will have once it is producing. Since this happens in real time, it boils down to an alert system for when drillers are tapping into uneconomic pay zones. “This is something people have tried to do for 30 years - using partial information to predict entire oil and gas properties,” said Tao Yang. He added that “the data acquisition is rather cheap compared with all the downhole tools, and it doesn’t cost you rig time,” highlighting that the mud-gas analyzer critical to the process sits on a rig or platform without interfering with drilling operations. Yang is a reservoir technology specialist at Equinor and one of the authors of a technical paper (SPE 201323) about the new digital technology that was presented at the SPE Annual Technical Conference and Exhibition in October. He and his colleagues spent more than 3 years building the system which began in the Norwegian oil company’s Houston office as a project to improve pressure/volume/temperature (PVT) analysis in tight-oil wells in North America. It has since found a home in the company’s much larger offshore business unit in Stavanger. Offshore projects designed around certain oil-production targets can face harsh realities when they end up producing more associated gas than expected. It is the difference between drilling an underperforming well full of headaches and one that will pay out hundreds of millions of dollars over its lifetime. By introducing real-time fluid identification, Equinor is trying to enforce a new control on that risk by giving drillers the information they need to pull the bit back and start drilling a side-track deeper into the formation where the odds are better of finding higher proportions of oil or condensates. At the conference, Yang shared details about some of the first field implementations, saying that in most cases the GOR predictions made by the fluid-identification system were confirmed by traditional PVT analysis from the trial wells. Unlike other advancements made on this front, he also said the new approach is the first of its kind to combine such a large database of PVT data with a machine-learning model “that is common to any well.” That means “we do not need to know where this well is located” to make a GOR prediction, said Yang.

scholarly journals A New Satellite-Based Retrieval of Low-Cloud Liquid-Water Path Using Machine Learning and Meteosat SEVIRI Data

2020 ◽  
Vol 12 (21) ◽  
pp. 3475
Author(s):  
Miae Kim ◽  
Jan Cermak ◽  
Hendrik Andersen ◽  
Julia Fuchs ◽  
Roland Stirnberg
Keyword(s):  
Machine Learning ◽  
Liquid Water ◽  
Ground Truth ◽  
Learning Model ◽  
Statistical Machine Learning ◽  
Liquid Water Path ◽  
Ground Truth Data ◽  
Water Path ◽  
Machine Learning Model ◽  
Low Cloud

Clouds are one of the major uncertainties of the climate system. The study of cloud processes requires information on cloud physical properties, in particular liquid water path (LWP). This parameter is commonly retrieved from satellite data using look-up table approaches. However, existing LWP retrievals come with uncertainties related to assumptions inherent in physical retrievals. Here, we present a new retrieval technique for cloud LWP based on a statistical machine learning model. The approach utilizes spectral information from geostationary satellite channels of Meteosat Spinning-Enhanced Visible and Infrared Imager (SEVIRI), as well as satellite viewing geometry. As ground truth, data from CloudNet stations were used to train the model. We found that LWP predicted by the machine-learning model agrees substantially better with CloudNet observations than a current physics-based product, the Climate Monitoring Satellite Application Facility (CM SAF) CLoud property dAtAset using SEVIRI, edition 2 (CLAAS-2), highlighting the potential of such approaches for future retrieval developments.

scholarly journals Numerical Investigations on the Shape Optimization of Stainless-Steel Ring Joint with Machine Learning

2020 ◽  
Vol 11 (1) ◽  
pp. 223
Author(s):  
Minsoo Kim ◽  
Sarang Yi ◽  
Seokmoo Hong
Keyword(s):  
Machine Learning ◽  
Finite Element ◽  
Learning Model ◽  
Training Data ◽  
Metal Ring ◽  
Rubber Ring ◽  
Water Pipes ◽  
Metal Rings ◽  
Machine Learning Model ◽  
Joint Method

Since pipes used for water pipes are thin and difficult to fasten using welding or screws, they are fastened by a crimping joint method using a metal ring and a rubber ring. In the conventional crimping joint method, the metal ring and the rubber ring are arranged side by side. However, if water leaks from the rubber ring, there is a problem that the adjacent metal ring is rapidly corroded. In this study, to delay and minimize the corrosion of connected water pipes, we propose a spaced crimping joint method in which metal rings and rubber rings are separated at appropriate intervals. This not only improves the contact performance between the connected water pipes but also minimizes the load applied to the crimping jig during crimping to prevent damage to the jig. For this, finite element analyses were performed for the crimp tool and process analysis, and the design parameters were set as the curling length at the top of the joint, the distance between the metal rings and rubber rings, and the crimp jig radius. Through FEA of 100 cases, data to be trained in machine learning were acquired. After that, training data were trained on a machine learning model and compared with a regression model to verify the model’s performance. If the number of training data is small, the two methods are similar. However, the greater the number of training data, the higher the accuracy predicted by the machine learning model. Finally, the spaced crimping joint to which the derived optimal shape was applied was manufactured, and the maximum pressure and pressure distribution applied during compression were obtained using a pressure film. This is almost similar to the value obtained by finite element analysis under the same conditions, and through this, the validity of the approach proposed in this study was verified.

scholarly journals Machine Learning Model for IRIS Flower Classification using Tensor Flow and PyTorch

2021 ◽  
pp. 15-23
Author(s):  
Dr. Kalaivazhi Vijayaragavan ◽  
S. Prakathi ◽  
S. Rajalakshmi ◽  
M Sandhiya
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Classification Accuracy ◽  
Learning Model ◽  
Training Data ◽  
Data Set ◽  
Machine Learning Model ◽  
Learning Frameworks ◽  
Learning Concept ◽  
Iris Data

Machine learning is a subfield of artificial intelligence, which is learning algorithms to make decision-based on data and try to behave like a human being. Classification is one of the most fundamental concepts in machine learning. It is a process of recognizing, understanding, and grouping ideas and objects into pre-set categories or sub-populations. Using precategorized training datasets, machine learning concept use variety of algorithms to classify the future datasets into categories. Classification algorithms use input training data in machine learning to predict the subsequent data that fall into one of the predetermined categories. To improve the classification accuracy design of neural network is regarded as effective model to obtain better accuracy. However, design of neural network is usually consider scaling layer, perceptron layers and probabilistic layer. In this paper, an enhanced model selection can be evaluated with training and testing strategy. Further, the classification accuracy can be predicted. Finally by using two popular machine learning frameworks: PyTorch and Tensor Flow the prediction of classification accuracy is compared. Results demonstrate that the proposed method can predict with more accuracy. After the deployment of our machine learning model the performance of the model has been evaluated with the help of iris data set.

scholarly journals An Explainable Machine Learning Model for Early Prediction of Sepsis Using ICU Data

2021 ◽  
Author(s):  
Naimahmed Nesaragi ◽  
Shivnarayan Patidar
Keyword(s):  
Machine Learning ◽  
Utility Score ◽  
Learning Model ◽  
Training Data ◽  
Early Prediction ◽  
Clinical Variables ◽  
Machine Learning Model ◽  
Hospital Systems ◽  
Prediction Risk ◽  
The Given

Early identification of individuals with sepsis is very useful in assisting clinical triage and decision-making, resulting in early intervention and improved outcomes. This study aims to develop an explainable machine learning model with the clinical interpretability to predict sepsis onset before 6 hours and validate with improved prediction risk power for every time interval since admission to the ICU. The retrospective observational cohort study is carried out using PhysioNet Challenge 2019 ICU data from three distinct hospital systems, viz. A, B, and C. Data from A and B were shared publicly for training and validation while sequestered data from all three cohorts were used for scoring. However, this study is limited only to publicly available training data. Training data contains 15,52,210 patient records of 40,336 ICU patients with up to 40 clinical variables (sourced for each hour of their ICU stay) divided into two datasets, based on hospital systems A and B. The clinical feature exploration and interpretation for early prediction of sepsis is achieved using the proposed framework, viz. the explainable Machine Learning model for Early Prediction of Sepsis (xMLEPS). A total of 85 features comprising the given 40 clinical variables augmented with 10 derived physiological features and 35 time-lag difference features are fed to xMLEPS for the said prediction task of sepsis onset. A ten-fold cross-validation scheme is employed wherein an optimal prediction risk threshold is searched for each of the 10 LightGBM models. These optimum threshold values are later used by the corresponding models to refine the predictive power in terms of utility score for the prediction of labels in each fold. The entire framework is designed via Bayesian optimization and trained with the resultant feature set of 85 features, yielding an average normalized utility score of 0.4214 and area under receiver operating characteristic curve of 0.8591 on publicly available training data. This study establish a practical and explainable sepsis onset prediction model for ICU data using applied ML approach, mainly gradient boosting. The study highlights the clinical significance of physiological inter-relations among the given and proposed clinical signs via feature importance and SHapley Additive exPlanations (SHAP) plots for visualized interpretation.

scholarly journals E-Gadget to Detect Food Freshness using IoT and ML

2021 ◽  
Vol 9 (12) ◽  
pp. 2072-2076
Author(s):  
Dr. M. P. Borawake
Keyword(s):  
Machine Learning ◽  
Food Item ◽  
Good Health ◽  
Learning Model ◽  
Training Data ◽  
Data Set ◽  
Health And Nutrition ◽  
Machine Learning Model ◽  
Spoilage Bacteria ◽  
Food Freshness

Abstract: The food we consume plays an important role in our daily life. It provides us energy which is needed to work, grow, be active, and to learn and think. The healthy food is essential for good health and nutrition. Light, oxygen, heat, humidity, temperature and spoilage bacteria can all affect both safety and quality of perishable foods. Food kept at room temperature undergoes some chemical reactions after certain period of time, which affects the taste, texture and smell of a food. Consuming spoiled food is harmful for consumers as it can lead to foodborne diseases. This project aims at detecting spoiled food using appropriate sensors and monitoring gases released by the particular food item. Sensors will measure the different parameters of food such as pH, ammonia gas, oxygen level, moisture, etc. The microcontroller takes the readings from sensors and these readings then given as an input to a machine learning model which can decide whether the food is spoilt or not based on training data set. Also, we plan to implement a machine learning model which can calculate the lifespan of that food item. Index Terms: Arduino Uno, Food spoilage, IoT, Machine Learning, Sensors.

scholarly journals Infectivity Upsurge by COVID-19 Viral Variants in Japan: Evidence from Deep Learning Modeling

2021 ◽  
Vol 18 (15) ◽  
pp. 7799
Author(s):  
Essam A. Rashed ◽  
Akimasa Hirata
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Learning Model ◽  
Economic Effects ◽  
Training Data ◽  
Healthcare Facilities ◽  
Forecasting Models ◽  
Machine Learning Model ◽  
Recent Emergence ◽  
Alpha Variant

The significant health and economic effects of COVID-19 emphasize the requirement for reliable forecasting models to avoid the sudden collapse of healthcare facilities with overloaded hospitals. Several forecasting models have been developed based on the data acquired within the early stages of the virus spread. However, with the recent emergence of new virus variants, it is unclear how the new strains could influence the efficiency of forecasting using models adopted using earlier data. In this study, we analyzed daily positive cases (DPC) data using a machine learning model to understand the effect of new viral variants on morbidity rates. A deep learning model that considers several environmental and mobility factors was used to forecast DPC in six districts of Japan. From machine learning predictions with training data since the early days of COVID-19, high-quality estimation has been achieved for data obtained earlier than March 2021. However, a significant upsurge was observed in some districts after the discovery of the new COVID-19 variant B.1.1.7 (Alpha). An average increase of 20–40% in DPC was observed after the emergence of the Alpha variant and an increase of up to 20% has been recognized in the effective reproduction number. Approximately four weeks was needed for the machine learning model to adjust the forecasting error caused by the new variants. The comparison between machine-learning predictions and reported values demonstrated that the emergence of new virus variants should be considered within COVID-19 forecasting models. This study presents an easy yet efficient way to quantify the change caused by new viral variants with potential usefulness for global data analysis.

Sign in / Sign up

Export Citation Format

Share Document