scholarly journals Assessing Continuous Operator Workload With a Hybrid Scaffolded Neuroergonomic Modeling Approach

2017 ◽  
Vol 59 (1) ◽  
pp. 134-146 ◽  
Author(s):  
Brett J. Borghetti ◽  
Joseph J. Giametta ◽  
Christina F. Rusnock
Keyword(s):  
Machine Learning ◽  
Adaptive Systems ◽  
Model Performance ◽  
Machine Learning Algorithms ◽  
Training Data ◽  
State Assessments ◽  
Learning Models ◽  
Dynamic Task ◽  
Operator Workload ◽  
Machine Learning Models

Objective: We aimed to predict operator workload from neurological data using statistical learning methods to fit neurological-to-state-assessment models. Background: Adaptive systems require real-time mental workload assessment to perform dynamic task allocations or operator augmentation as workload issues arise. Neuroergonomic measures have great potential for informing adaptive systems, and we combine these measures with models of task demand as well as information about critical events and performance to clarify the inherent ambiguity of interpretation. Method: We use machine learning algorithms on electroencephalogram (EEG) input to infer operator workload based upon Improved Performance Research Integration Tool workload model estimates. Results: Cross-participant models predict workload of other participants, statistically distinguishing between 62% of the workload changes. Machine learning models trained from Monte Carlo resampled workload profiles can be used in place of deterministic workload profiles for cross-participant modeling without incurring a significant decrease in machine learning model performance, suggesting that stochastic models can be used when limited training data are available. Conclusion: We employed a novel temporary scaffold of simulation-generated workload profile truth data during the model-fitting process. A continuous workload profile serves as the target to train our statistical machine learning models. Once trained, the workload profile scaffolding is removed and the trained model is used directly on neurophysiological data in future operator state assessments. Application: These modeling techniques demonstrate how to use neuroergonomic methods to develop operator state assessments, which can be employed in adaptive systems.

scholarly journals Effectiveness, Explainability and Reliability of Machine Meta-Learning Methods for Predicting Mortality in Patients with COVID-19: Results of the Brazilian COVID-19 Registry

2021 ◽  
Author(s):  
Bruno Barbosa Miranda de Paiva ◽  
Polianna Delfino Pereira ◽  
Claudio Moises Valiense de Andrade ◽  
Virginia Mara Reis Gomes ◽  
Maria Clara Pontello Barbosa Lima ◽  
...  
Keyword(s):  
Machine Learning ◽  
Prediction Models ◽  
State Of The Art ◽  
Laboratory Data ◽  
Machine Learning Algorithms ◽  
Training Data ◽  
Learning Models ◽  
Learning Methods ◽  
Meta Learning ◽  
Machine Learning Models

Objective: To provide a thorough comparative study among state ofthe art machine learning methods and statistical methods for determining in-hospital mortality in COVID 19 patients using data upon hospital admission; to study the reliability of the predictions of the most effective methods by correlating the probability of the outcome and the accuracy of the methods; to investigate how explainable are the predictions produced by the most effective methods. Materials and Methods: De-identified data were obtained from COVID 19 positive patients in 36 participating hospitals, from March 1 to September 30, 2020. Demographic, comorbidity, clinical presentation and laboratory data were used as training data to develop COVID 19 mortality prediction models. Multiple machine learning and traditional statistics models were trained on this prediction task using a folded cross validation procedure, from which we assessed performance and interpretability metrics. Results: The Stacking of machine learning models improved over the previous state of the art results by more than 26% in predicting the class of interest (death), achieving 87.1% of AUROC and macroF1 of 73.9%. We also show that some machine learning models can be very interpretable and reliable, yielding more accurate predictions while providing a good explanation for the why. Conclusion: The best results were obtained using the meta learning ensemble model Stacking. State of the art explainability techniques such as SHAP values can be used to draw useful insights into the patterns learned by machine-learning algorithms. Machine learning models can be more explainable than traditional statistics models while also yielding highly reliable predictions. Key words: COVID-19; prognosis; prediction model; machine learning

scholarly journals Design and Manufacture of a Multiband Rectangular Spiral-Shaped Microstrip Antenna Using EM-Driven and Machine Learning

2021 ◽  
Vol 27 (1) ◽  
pp. 29-40
Author(s):  
Ashrf Aoad
Keyword(s):  
Machine Learning ◽  
Microstrip Antenna ◽  
Machine Learning Algorithms ◽  
Mobile Systems ◽  
Training Data ◽  
Learning Models ◽  
Prediction Ability ◽  
Antenna Structure ◽  
Wide Range ◽  
Machine Learning Models

This paper presents a multiband rectangular microstrip antenna using spiral-shaped configurations. The antenna has been designed by combining two configurations of microstrip and spiral with consideration of careful selection of the substrate material, the dimension of the rectangular microstrip, the distance between the turned spiral, and the number of turns of the spiral. The efficiency and accuracy have been improved using machine learning algorithms as well. Machine learning has been studied to model the proposed antenna based on the performance requirements, which requires a sufficient training data to improve the accuracy. Three different machine learning models are applied to improve the accuracy and generalization performance and compared to simulation and measurement results. Simulation, measurement, and machine learning results confirm that the proposed antenna is a new electrically small and operating over a wide range of high-frequency bands between 1 GHz–4 GHz. Machine learning models have the best prediction ability with a mean square error (MSE) of 0.03, and 0.05. The antenna structure and size are compatible and suitable for several multi-band wireless mobile systems operating in L-band and S-band. The results, such as directivity, Half-Power Beamwidth, Voltage Standing Wave Ratio (VSWR), and S-parameter curves, are analysed and compared with the numerical formulation for both spiral and microstrip antennas.

scholarly journals Three-level Sleep Stage Classification Based on Wrist-worn Accelerometry Data Alone

2021 ◽  
Author(s):  
Jian Hu ◽  
Haochang Shou
Keyword(s):  
Machine Learning ◽  
Sequential Analysis ◽  
Short Term Memory ◽  
Sleep Stage ◽  
Model Performance ◽  
Training Data ◽  
Sleep Stages ◽  
Learning Models ◽  
Local Variability ◽  
Machine Learning Models

Objective: The use of wearable sensor devices on daily basis to track real-time movements during wake and sleep has provided opportunities for automatic sleep quantification using such data. Existing algorithms for classifying sleep stages often require large training data and multiple input signals including heart rate and respiratory data. We aimed to examine the capability of classifying sleep stages using sensible features directly from accelerometers only with the aid of advanced recurrent neural networks. Materials and Methods: We analyzed a publicly available dataset with accelerometry data in 5s epoch length and polysomnography assessments. We developed long short-term memory (LSTM) models that take the 3-axis accelerations, angles, and temperatures from concurrent and historic observation windows to predict wake, REM and non-REM sleep. Leave-one-subject-out experiments were conducted to compare and evaluate the model performance with conventional nonsequential machine learning models using metrics such as multiclass training and testing accuracy, weighted precision, F1 score and area-under-the-curve (AUC). Results: Our sequential analysis framework outperforms traditional non-sequential models in all aspects of model evaluation metrics. We achieved an average of 65% and a maximum of 81% validation accuracy for classifying three sleep labels even with a relatively small training sample of clinical visitors. The presence of two additional derived variables, local variability and range, have shown to strongly improve the model performance. Discussion : Results indicate that it is crucial to account for deep temporal dependency and assess local variability of the features. The post-hoc analysis of individual model performances on subjects' demographic characteristics also suggest the need of including pathological samples in the training data in order to develop robust machine learning models that are capable of capturing normal and anomaly sleep patterns in the population.

scholarly journals An Adaptive Deep Ensemble Learning Method for Dynamic Evolving Diagnostic Task Scenarios

Diagnostics ◽  
2021 ◽  
Vol 11 (12) ◽  
pp. 2288
Author(s):  
Kaixiang Su ◽  
Jiao Wu ◽  
Dongxiao Gu ◽  
Shanlin Yang ◽  
Shuyuan Deng ◽  
...  
Keyword(s):  
Machine Learning ◽  
Ensemble Learning ◽  
Model Performance ◽  
Optimal Number ◽  
Training Data ◽  
Learning Method ◽  
Learning Models ◽  
Proposed Model ◽  
Public Datasets ◽  
Machine Learning Models

Increasingly, machine learning methods have been applied to aid in diagnosis with good results. However, some complex models can confuse physicians because they are difficult to understand, while data differences across diagnostic tasks and institutions can cause model performance fluctuations. To address this challenge, we combined the Deep Ensemble Model (DEM) and tree-structured Parzen Estimator (TPE) and proposed an adaptive deep ensemble learning method (TPE-DEM) for dynamic evolving diagnostic task scenarios. Different from previous research that focuses on achieving better performance with a fixed structure model, our proposed model uses TPE to efficiently aggregate simple models more easily understood by physicians and require less training data. In addition, our proposed model can choose the optimal number of layers for the model and the type and number of basic learners to achieve the best performance in different diagnostic task scenarios based on the data distribution and characteristics of the current diagnostic task. We tested our model on one dataset constructed with a partner hospital and five UCI public datasets with different characteristics and volumes based on various diagnostic tasks. Our performance evaluation results show that our proposed model outperforms other baseline models on different datasets. Our study provides a novel approach for simple and understandable machine learning models in tasks with variable datasets and feature sets, and the findings have important implications for the application of machine learning models in computer-aided diagnosis.

scholarly journals Quantitative Interpretation Explains Machine Learning Models for Chemical Reaction Prediction and Uncovers Bias

2020 ◽  
Author(s):  
David Peter Kovacs ◽  
William McCorkindale ◽  
Alpha Lee
Keyword(s):  
Machine Learning ◽  
Prediction Models ◽  
Model Performance ◽  
Training Data ◽  
Correct Prediction ◽  
Learning Models ◽  
Reaction Prediction ◽  
Wrong Reason ◽  
Realistic Assessment ◽  
Machine Learning Models

<div><div><div><p>Organic synthesis remains a stumbling block in drug discovery. Although a plethora of machine learning models have been proposed as solutions in the literature, they suffer from being opaque black-boxes. It is neither clear if the models are making correct predictions because they inferred the salient chemistry, nor is it clear which training data they are relying on to reach a prediction. This opaqueness hinders both model developers and users. In this paper, we quantitatively interpret the Molecular Transformer, the state-of-the-art model for reaction prediction. We develop a framework to attribute predicted reaction outcomes both to specific parts of reactants, and to reactions in the training set. Furthermore, we demonstrate how to retrieve evidence for predicted reaction outcomes, and understand counterintuitive predictions by scrutinising the data. Additionally, we identify ”Clever Hans” predictions where the correct prediction is reached for the wrong reason due to dataset bias. We present a new debiased dataset that provides a more realistic assessment of model performance, which we propose as the new standard benchmark for comparing reaction prediction models.</p></div></div></div>

scholarly journals Quantitative interpretation explains machine learning models for chemical reaction prediction and uncovers bias

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Dávid Péter Kovács ◽  
William McCorkindale ◽  
Alpha A. Lee
Keyword(s):  
Machine Learning ◽  
Prediction Models ◽  
Model Performance ◽  
Training Data ◽  
Correct Prediction ◽  
Learning Models ◽  
Reaction Prediction ◽  
Wrong Reason ◽  
Realistic Assessment ◽  
Machine Learning Models

AbstractOrganic synthesis remains a major challenge in drug discovery. Although a plethora of machine learning models have been proposed as solutions in the literature, they suffer from being opaque black-boxes. It is neither clear if the models are making correct predictions because they inferred the salient chemistry, nor is it clear which training data they are relying on to reach a prediction. This opaqueness hinders both model developers and users. In this paper, we quantitatively interpret the Molecular Transformer, the state-of-the-art model for reaction prediction. We develop a framework to attribute predicted reaction outcomes both to specific parts of reactants, and to reactions in the training set. Furthermore, we demonstrate how to retrieve evidence for predicted reaction outcomes, and understand counterintuitive predictions by scrutinising the data. Additionally, we identify Clever Hans predictions where the correct prediction is reached for the wrong reason due to dataset bias. We present a new debiased dataset that provides a more realistic assessment of model performance, which we propose as the new standard benchmark for comparing reaction prediction models.

scholarly journals Quantitative Interpretation Explains Machine Learning Models for Chemical Reaction Prediction and Uncovers Bias

2020 ◽  
Author(s):  
David Peter Kovacs ◽  
William McCorkindale ◽  
Alpha Lee
Keyword(s):  
Machine Learning ◽  
Prediction Models ◽  
Model Performance ◽  
Training Data ◽  
Correct Prediction ◽  
Learning Models ◽  
Reaction Prediction ◽  
Wrong Reason ◽  
Realistic Assessment ◽  
Machine Learning Models

<div><div><div><p>Organic synthesis remains a stumbling block in drug discovery. Although a plethora of machine learning models have been proposed as solutions in the literature, they suffer from being opaque black-boxes. It is neither clear if the models are making correct predictions because they inferred the salient chemistry, nor is it clear which training data they are relying on to reach a prediction. This opaqueness hinders both model developers and users. In this paper, we quantitatively interpret the Molecular Transformer, the state-of-the-art model for reaction prediction. We develop a framework to attribute predicted reaction outcomes both to specific parts of reactants, and to reactions in the training set. Furthermore, we demonstrate how to retrieve evidence for predicted reaction outcomes, and understand counterintuitive predictions by scrutinising the data. Additionally, we identify ”Clever Hans” predictions where the correct prediction is reached for the wrong reason due to dataset bias. We present a new debiased dataset that provides a more realistic assessment of model performance, which we propose as the new standard benchmark for comparing reaction prediction models.</p></div></div></div>

Prediction of aircraft estimated time of arrival using machine learning methods

2021 ◽  
pp. 1-15
Author(s):  
O. Basturk ◽  
C. Cetek
Keyword(s):  
Machine Learning ◽  
Web Application ◽  
Absolute Error ◽  
Machine Learning Algorithms ◽  
Weather Data ◽  
Time Of Arrival ◽  
Learning Models ◽  
Trajectory Data ◽  
Different Sources ◽  
Machine Learning Models

ABSTRACT In this study, prediction of aircraft Estimated Time of Arrival (ETA) is proposed using machine learning algorithms. Accurate prediction of ETA is important for management of delay and air traffic flow, runway assignment, gate assignment, collaborative decision making (CDM), coordination of ground personnel and equipment, and optimisation of arrival sequence etc. Machine learning is able to learn from experience and make predictions with weak assumptions or no assumptions at all. In the proposed approach, general flight information, trajectory data and weather data were obtained from different sources in various formats. Raw data were converted to tidy data and inserted into a relational database. To obtain the features for training the machine learning models, the data were explored, cleaned and transformed into convenient features. New features were also derived from the available data. Random forests and deep neural networks were used to train the machine learning models. Both models can predict the ETA with a mean absolute error (MAE) less than 6min after departure, and less than 3min after terminal manoeuvring area (TMA) entrance. Additionally, a web application was developed to dynamically predict the ETA using proposed models.

scholarly journals Characterizing and Evaluating the Zoonotic Potential of Novel Viruses Discovered in Vampire Bats

Viruses ◽  
2021 ◽  
Vol 13 (2) ◽  
pp. 252
Author(s):  
Laura M. Bergner ◽  
Nardus Mollentze ◽  
Richard J. Orton ◽  
Carlos Tello ◽  
Alice Broos ◽  
...  
Keyword(s):  
Machine Learning ◽  
Phylogenetic Analyses ◽  
Human Infection ◽  
Machine Learning Algorithms ◽  
Zoonotic Potential ◽  
Metagenomic Sequencing ◽  
Learning Models ◽  
Sequencing Data ◽  
Vampire Bats ◽  
Machine Learning Models

The contemporary surge in metagenomic sequencing has transformed knowledge of viral diversity in wildlife. However, evaluating which newly discovered viruses pose sufficient risk of infecting humans to merit detailed laboratory characterization and surveillance remains largely speculative. Machine learning algorithms have been developed to address this imbalance by ranking the relative likelihood of human infection based on viral genome sequences, but are not yet routinely applied to viruses at the time of their discovery. Here, we characterized viral genomes detected through metagenomic sequencing of feces and saliva from common vampire bats (Desmodus rotundus) and used these data as a case study in evaluating zoonotic potential using molecular sequencing data. Of 58 detected viral families, including 17 which infect mammals, the only known zoonosis detected was rabies virus; however, additional genomes were detected from the families Hepeviridae, Coronaviridae, Reoviridae, Astroviridae and Picornaviridae, all of which contain human-infecting species. In phylogenetic analyses, novel vampire bat viruses most frequently grouped with other bat viruses that are not currently known to infect humans. In agreement, machine learning models built from only phylogenetic information ranked all novel viruses similarly, yielding little insight into zoonotic potential. In contrast, genome composition-based machine learning models estimated different levels of zoonotic potential, even for closely related viruses, categorizing one out of four detected hepeviruses and two out of three picornaviruses as having high priority for further research. We highlight the value of evaluating zoonotic potential beyond ad hoc consideration of phylogeny and provide surveillance recommendations for novel viruses in a wildlife host which has frequent contact with humans and domestic animals.

Model Comparison for Esp Run-Life Prediction: Classic Statistics Vs. Machine Learning

2021 ◽  
Author(s):  
Alejandro Celemín ◽  
Diego A. Estupiñan ◽  
Ricardo Nieto
Keyword(s):  
Machine Learning ◽  
Model Comparison ◽  
Proportional Hazards ◽  
Proportional Hazards Model ◽  
Machine Learning Algorithms ◽  
Slight Reduction ◽  
Learning Models ◽  
Validation Data ◽  
Operational Conditions ◽  
Machine Learning Models

Abstract Electrical Submersible Pumps reliability and run-life analysis has been extensively studied since its development. Current machine learning algorithms allow to correlate operational conditions to ESP run-life in order to generate predictions for active and new wells. Four machine learning models are compared to a linear proportional hazards model, used as a baseline for comparison purposes. Proper accuracy metrics for survival analysis problems are calculated on run-life predictions vs. actual values over training and validation data subsets. Results demonstrate that the baseline model is able to produce more consistent predictions with a slight reduction in its accuracy, compared to current machine learning models for small datasets. This study demonstrates that the quality of the date and it pre-processing supports the current shift from model-centric to data-centric approach to machine and deep learning problems.

Sign in / Sign up

Export Citation Format

Share Document