scholarly journals Data Synthesis for Testing Black-Box Machine Learning Models

2022 ◽  
Author(s):  
Diptikalyan Saha ◽  
Aniya Aggarwal ◽  
Sandeep Hans
Keyword(s):  
Machine Learning ◽  
Black Box ◽  
Learning Models ◽  
Data Synthesis ◽  
Machine Learning Models

scholarly journals Explainable AI: A Review of Machine Learning Interpretability Methods

Entropy ◽  
2020 ◽  
Vol 23 (1) ◽  
pp. 18
Author(s):  
Pantelis Linardatos ◽  
Vasilis Papastefanopoulos ◽  
Sotiris Kotsiantis
Keyword(s):  
Artificial Intelligence ◽  
Machine Learning ◽  
Black Box ◽  
Learning Systems ◽  
Model Complexity ◽  
Learning Models ◽  
New Methods ◽  
Industrial Adoption ◽  
Machine Learning Models ◽  
The Way

Recent advances in artificial intelligence (AI) have led to its widespread industrial adoption, with machine learning systems demonstrating superhuman performance in a significant number of tasks. However, this surge in performance, has often been achieved through increased model complexity, turning such systems into “black box” approaches and causing uncertainty regarding the way they operate and, ultimately, the way that they come to decisions. This ambiguity has made it problematic for machine learning systems to be adopted in sensitive yet critical domains, where their value could be immense, such as healthcare. As a result, scientific interest in the field of Explainable Artificial Intelligence (XAI), a field that is concerned with the development of new methods that explain and interpret machine learning models, has been tremendously reignited over recent years. This study focuses on machine learning interpretability methods; more specifically, a literature review and taxonomy of these methods are presented, as well as links to their programming implementations, in the hope that this survey would serve as a reference point for both theorists and practitioners.

scholarly journals Uncovering and Correcting Shortcut Learning in Machine Learning Models for Skin Cancer Diagnosis

Diagnostics ◽  
2021 ◽  
Vol 12 (1) ◽  
pp. 40
Author(s):  
Meike Nauta ◽  
Ricky Walsh ◽  
Adam Dubowski ◽  
Christin Seifert
Keyword(s):  
Machine Learning ◽  
Clinical Practice ◽  
Skin Cancer ◽  
Cancer Diagnosis ◽  
Image Inpainting ◽  
Relevant Information ◽  
Black Box ◽  
Training Dataset ◽  
Learning Models ◽  
Machine Learning Models

Machine learning models have been successfully applied for analysis of skin images. However, due to the black box nature of such deep learning models, it is difficult to understand their underlying reasoning. This prevents a human from validating whether the model is right for the right reasons. Spurious correlations and other biases in data can cause a model to base its predictions on such artefacts rather than on the true relevant information. These learned shortcuts can in turn cause incorrect performance estimates and can result in unexpected outcomes when the model is applied in clinical practice. This study presents a method to detect and quantify this shortcut learning in trained classifiers for skin cancer diagnosis, since it is known that dermoscopy images can contain artefacts. Specifically, we train a standard VGG16-based skin cancer classifier on the public ISIC dataset, for which colour calibration charts (elliptical, coloured patches) occur only in benign images and not in malignant ones. Our methodology artificially inserts those patches and uses inpainting to automatically remove patches from images to assess the changes in predictions. We find that our standard classifier partly bases its predictions of benign images on the presence of such a coloured patch. More importantly, by artificially inserting coloured patches into malignant images, we show that shortcut learning results in a significant increase in misdiagnoses, making the classifier unreliable when used in clinical practice. With our results, we, therefore, want to increase awareness of the risks of using black box machine learning models trained on potentially biased datasets. Finally, we present a model-agnostic method to neutralise shortcut learning by removing the bias in the training dataset by exchanging coloured patches with benign skin tissue using image inpainting and re-training the classifier on this de-biased dataset.

Query-efficient label-only attacks against black-box machine learning models

2020 ◽  
Vol 90 ◽  
pp. 101698
Author(s):  
Yizhi Ren ◽  
Qi Zhou ◽  
Zhen Wang ◽  
Ting Wu ◽  
Guohua Wu ◽  
...  
Keyword(s):  
Machine Learning ◽  
Black Box ◽  
Learning Models ◽  
Machine Learning Models

Interpretable machine learning with reject option

2018 ◽  
Vol 66 (4) ◽  
pp. 283-290 ◽  
Author(s):  
Johannes Brinkrolf ◽  
Barbara Hammer
Keyword(s):  
Machine Learning ◽  
Vector Quantization ◽  
Random Forests ◽  
Black Box ◽  
Learning Models ◽  
Process Automation ◽  
Reject Option ◽  
Interpretable Machine Learning ◽  
Adversarial Examples ◽  
Machine Learning Models

Abstract Classification by means of machine learning models constitutes one relevant technology in process automation and predictive maintenance. However, common techniques such as deep networks or random forests suffer from their black box characteristics and possible adversarial examples. In this contribution, we give an overview about a popular alternative technology from machine learning, namely modern variants of learning vector quantization, which, due to their combined discriminative and generative nature, incorporate interpretability and the possibility of explicit reject options for irregular samples. We give an explicit bound on minimum changes required for a change of the classification in case of LVQ networks with reject option, and we demonstrate the efficiency of reject options in two examples.

Sign in / Sign up

Export Citation Format

Share Document