The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizations

2020 ◽  
Vol 39 (3) ◽  
pp. 713-756 ◽  
Author(s):  
A. Chatzimparmpas ◽  
R. M. Martins ◽  
I. Jusufi ◽  
K. Kucher ◽  
F. Rossi ◽  
...  
Keyword(s):  
Machine Learning ◽  
State Of The Art ◽  
The State ◽  
Learning Models ◽  
Machine Learning Models

scholarly journals Effectiveness, Explainability and Reliability of Machine Meta-Learning Methods for Predicting Mortality in Patients with COVID-19: Results of the Brazilian COVID-19 Registry

2021 ◽  
Author(s):  
Bruno Barbosa Miranda de Paiva ◽  
Polianna Delfino Pereira ◽  
Claudio Moises Valiense de Andrade ◽  
Virginia Mara Reis Gomes ◽  
Maria Clara Pontello Barbosa Lima ◽  
...  
Keyword(s):  
Machine Learning ◽  
Prediction Models ◽  
State Of The Art ◽  
Laboratory Data ◽  
Machine Learning Algorithms ◽  
Training Data ◽  
Learning Models ◽  
Learning Methods ◽  
Meta Learning ◽  
Machine Learning Models

Objective: To provide a thorough comparative study among state ofthe art machine learning methods and statistical methods for determining in-hospital mortality in COVID 19 patients using data upon hospital admission; to study the reliability of the predictions of the most effective methods by correlating the probability of the outcome and the accuracy of the methods; to investigate how explainable are the predictions produced by the most effective methods. Materials and Methods: De-identified data were obtained from COVID 19 positive patients in 36 participating hospitals, from March 1 to September 30, 2020. Demographic, comorbidity, clinical presentation and laboratory data were used as training data to develop COVID 19 mortality prediction models. Multiple machine learning and traditional statistics models were trained on this prediction task using a folded cross validation procedure, from which we assessed performance and interpretability metrics. Results: The Stacking of machine learning models improved over the previous state of the art results by more than 26% in predicting the class of interest (death), achieving 87.1% of AUROC and macroF1 of 73.9%. We also show that some machine learning models can be very interpretable and reliable, yielding more accurate predictions while providing a good explanation for the why. Conclusion: The best results were obtained using the meta learning ensemble model Stacking. State of the art explainability techniques such as SHAP values can be used to draw useful insights into the patterns learned by machine-learning algorithms. Machine learning models can be more explainable than traditional statistics models while also yielding highly reliable predictions. Key words: COVID-19; prognosis; prediction model; machine learning

scholarly journals Decoding defect statistics from diffractograms via machine learning

2021 ◽  
Vol 7 (1) ◽  
Author(s):  
Cody Kunka ◽  
Apaar Shanker ◽  
Elton Y. Chen ◽  
Surya R. Kalidindi ◽  
Rémi Dingreville
Keyword(s):  
Machine Learning ◽  
Spatial Distribution ◽  
State Of The Art ◽  
Structural Information ◽  
Atomistic Simulations ◽  
Learning Models ◽  
Machine Learning Model ◽  
Electron Diffractograms ◽  
Material Information ◽  
Machine Learning Models

AbstractDiffraction techniques can powerfully and nondestructively probe materials while maintaining high resolution in both space and time. Unfortunately, these characterizations have been limited and sometimes even erroneous due to the difficulty of decoding the desired material information from features of the diffractograms. Currently, these features are identified non-comprehensively via human intuition, so the resulting models can only predict a subset of the available structural information. In the present work we show (i) how to compute machine-identified features that fully summarize a diffractogram and (ii) how to employ machine learning to reliably connect these features to an expanded set of structural statistics. To exemplify this framework, we assessed virtual electron diffractograms generated from atomistic simulations of irradiated copper. When based on machine-identified features rather than human-identified features, our machine-learning model not only predicted one-point statistics (i.e. density) but also a two-point statistic (i.e. spatial distribution) of the defect population. Hence, this work demonstrates that machine-learning models that input machine-identified features significantly advance the state of the art for accurately and robustly decoding diffractograms.

scholarly journals A Human-AI Loop Approach for Joint Keyword Discovery and Expectation Estimation in Micropost Event Detection

2020 ◽  
Vol 34 (03) ◽  
pp. 2451-2458
Author(s):  
Akansha Bhardwaj ◽  
Jie Yang ◽  
Philippe Cudré-Mauroux
Keyword(s):  
Machine Learning ◽  
Real World ◽  
Event Detection ◽  
State Of The Art ◽  
Regularization Parameter ◽  
Learning Models ◽  
Training Process ◽  
Model Training ◽  
Real World Datasets ◽  
Machine Learning Models

Microblogging platforms such as Twitter are increasingly being used in event detection. Existing approaches mainly use machine learning models and rely on event-related keywords to collect the data for model training. These approaches make strong assumptions on the distribution of the relevant microposts containing the keyword – referred to as the expectation of the distribution – and use it as a posterior regularization parameter during model training. Such approaches are, however, limited as they fail to reliably estimate the informativeness of a keyword and its expectation for model training. This paper introduces a Human-AI loop approach to jointly discover informative keywords for model training while estimating their expectation. Our approach iteratively leverages the crowd to estimate both keyword-specific expectation and the disagreement between the crowd and the model in order to discover new keywords that are most beneficial for model training. These keywords and their expectation not only improve the resulting performance but also make the model training process more transparent. We empirically demonstrate the merits of our approach, both in terms of accuracy and interpretability, on multiple real-world datasets and show that our approach improves the state of the art by 24.3%.

scholarly journals A general method for estimating the prevalence of influenza-like-symptoms with Wikipedia data

PLoS ONE ◽  
2021 ◽  
Vol 16 (8) ◽  
pp. e0256858
Author(s):  
Giovanni De Toni ◽  
Cristian Consonni ◽  
Alberto Montresor
Keyword(s):  
Machine Learning ◽  
The Netherlands ◽  
State Of The Art ◽  
Data Sources ◽  
Learning Models ◽  
Personalized Pagerank ◽  
Web Searches ◽  
The Impact ◽  
Machine Learning Models ◽  
General Method

Influenza is an acute respiratory seasonal disease that affects millions of people worldwide and causes thousands of deaths in Europe alone. Estimating in a fast and reliable way the impact of an illness on a given country is essential to plan and organize effective countermeasures, which is now possible by leveraging unconventional data sources like web searches and visits. In this study, we show the feasibility of exploiting machine learning models and information about Wikipedia’s page views of a selected group of articles to obtain accurate estimates of influenza-like illnesses incidence in four European countries: Italy, Germany, Belgium, and the Netherlands. We propose a novel language-agnostic method, based on two algorithms, Personalized PageRank and CycleRank, to automatically select the most relevant Wikipedia pages to be monitored without the need for expert supervision. We then show how our model can reach state-of-the-art results by comparing it with previous solutions.

scholarly journals What if Social Robots Look for Productive Engagement?

2021 ◽  
Author(s):  
Jauwairia Nasir ◽  
Barbara Bruno ◽  
Mohamed Chetouani ◽  
Pierre Dillenbourg
Keyword(s):  
Machine Learning ◽  
Learning Outcomes ◽  
State Of The Art ◽  
Learning Models ◽  
Collaborative Activity ◽  
Data Set ◽  
Performance Metric ◽  
The Relationship ◽  
Machine Learning Models ◽  
Productive Engagement

AbstractIn educational HRI, it is generally believed that a robots behavior has a direct effect on the engagement of a user with the robot, the task at hand and also their partner in case of a collaborative activity. Increasing this engagement is then held responsible for increased learning and productivity. The state of the art usually investigates the relationship between the behaviors of the robot and the engagement state of the user while assuming a linear relationship between engagement and the end goal: learning. However, is it correct to assume that to maximise learning, one needs to maximise engagement? Furthermore, conventional supervised models of engagement require human annotators to get labels. This is not only laborious but also introduces further subjectivity in an already subjective construct of engagement. Can we have machine-learning models for engagement detection where annotations do not rely on human annotators? Looking deeper at the behavioral patterns and the learning outcomes and a performance metric in a multi-modal data set collected in an educational human–human–robot setup with 68 students, we observe a hidden link that we term as Productive Engagement. We theorize a robot incorporating this knowledge will (1) distinguish teams based on engagement that is conducive of learning; and (2) adopt behaviors that eventually lead the users to increased learning by means of being productively engaged. Furthermore, this seminal link paves way for machine-learning models in educational HRI with automatic labelling based on the data.

scholarly journals Aachen-Heerlen annotated steel microstructure dataset

2021 ◽  
Vol 8 (1) ◽  
Author(s):  
Deniz Iren ◽  
Marc Ackermann ◽  
Julian Gorfer ◽  
Gaurav Pujar ◽  
Sebastian Wesselmecking ◽  
...  
Keyword(s):  
Machine Learning ◽  
Mechanical Characteristics ◽  
State Of The Art ◽  
Quality Data ◽  
Bainitic Steel ◽  
Learning Models ◽  
Steel Microstructure ◽  
Art Object ◽  
Abstract Shapes ◽  
Machine Learning Models

AbstractStudying steel microstructures yields important insights regarding its mechanical characteristics. Within steel, microstructures transform based on a multitude of factors including chemical composition, transformation temperatures, and cooling rates. Martensite-austenite (MA) islands in bainitic steel appear as blocky structures with abstract shapes that are difficult to identify and differentiate from other types of microstructures. In this regard, material science may benefit from machine learning models that are able to automatically and accurately detect these structures. However, the training process of the state-of-the-art machine learning models requires a large amount of high-quality data. In this dataset, we provide 1.705 scanning electron microscopy images along with a set of 8.909 expert-annotated polygons to describe the geometry of the MA islands that appear on the images. We envision that this dataset will be useful for material scientists to explore the relationship between the morphology of bainitic steel and mechanical characteristics. Moreover, computer vision researchers and practitioners may use this data for training state-of-the-art object segmentation models for abstract geometries such as MA islands.

Sign in / Sign up

Export Citation Format

Share Document