Fractal Set Theoretic Analysis of Performance Losses for Tuning Training Data in Learning Systems

1993 ◽  
pp. 157-176 ◽  
Author(s):  
Aydan M. Erkmen
Keyword(s):  
Learning Systems ◽  
Training Data ◽  
Theoretic Analysis ◽  
Fractal Set ◽  
Analysis Of Performance

scholarly journals Zero-Shot Feature Selection via Transferring Supervised Knowledge

2021 ◽  
Vol 17 (2) ◽  
pp. 1-20
Author(s):  
Zheng Wang ◽  
Qiao Wang ◽  
Tingzhang Zhao ◽  
Chaokun Wang ◽  
Xiaojun Ye
Keyword(s):  
Machine Learning ◽  
Feature Selection ◽  
Dimensionality Reduction ◽  
Real World ◽  
Rapid Growth ◽  
Learning Systems ◽  
Training Data ◽  
Effective Technique ◽  
Supervised Methods ◽  
Real World Datasets

Feature selection, an effective technique for dimensionality reduction, plays an important role in many machine learning systems. Supervised knowledge can significantly improve the performance. However, faced with the rapid growth of newly emerging concepts, existing supervised methods might easily suffer from the scarcity and validity of labeled data for training. In this paper, the authors study the problem of zero-shot feature selection (i.e., building a feature selection model that generalizes well to “unseen” concepts with limited training data of “seen” concepts). Specifically, they adopt class-semantic descriptions (i.e., attributes) as supervision for feature selection, so as to utilize the supervised knowledge transferred from the seen concepts. For more reliable discriminative features, they further propose the center-characteristic loss which encourages the selected features to capture the central characteristics of seen concepts. Extensive experiments conducted on various real-world datasets demonstrate the effectiveness of the method.

scholarly journals Human-Like Sketch Object Recognition via Analogical Learning

2019 ◽  
Vol 33 ◽  
pp. 1336-1343
Author(s):  
Kezhen Chen ◽  
Irina Rabkina ◽  
Matthew D. McLure ◽  
Kenneth D. Forbus
Keyword(s):  
Computer Vision ◽  
Deep Learning ◽  
Object Recognition ◽  
Image Recognition ◽  
Visual Representations ◽  
Learning Systems ◽  
Training Data ◽  
Adversarial Examples ◽  
Analogical Learning

Deep learning systems can perform well on some image recognition tasks. However, they have serious limitations, including requiring far more training data than humans do and being fooled by adversarial examples. By contrast, analogical learning over relational representations tends to be far more data-efficient, requiring only human-like amounts of training data. This paper introduces an approach that combines automatically constructed qualitative visual representations with analogical learning to tackle a hard computer vision problem, object recognition from sketches. Results from the MNIST dataset and a novel dataset, the Coloring Book Objects dataset, are provided. Comparison to existing approaches indicates that analogical generalization can be used to identify sketched objects from these datasets with several orders of magnitude fewer examples than deep learning systems require.

scholarly journals Rationale Discovery and Explainable AI

2021 ◽  
Author(s):  
Cor Steging ◽  
Silja Renooij ◽  
Bart Verheij
Keyword(s):  
Machine Learning ◽  
Feature Detection ◽  
State Of The Art ◽  
High Accuracy ◽  
Learning Systems ◽  
Training Data ◽  
Relevant Feature ◽  
Explainable Ai ◽  
The Right ◽  
The Impact

The justification of an algorithm’s outcomes is important in many domains, and in particular in the law. However, previous research has shown that machine learning systems can make the right decisions for the wrong reasons: despite high accuracies, not all of the conditions that define the domain of the training data are learned. In this study, we investigate what the system does learn, using state-of-the-art explainable AI techniques. With the use of SHAP and LIME, we are able to show which features impact the decision making process and how the impact changes with different distributions of the training data. However, our results also show that even high accuracy and good relevant feature detection are no guarantee for a sound rationale. Hence these state-of-the-art explainable AI techniques cannot be used to fully expose unsound rationales, further advocating the need for a separate method for rationale evaluation.

scholarly journals On the relationship between research parasites and fairness in machine learning: challenges and opportunities

GigaScience ◽  
2021 ◽  
Vol 10 (12) ◽  
Author(s):  
Nicolás Nieto ◽  
Agostina Larrazabal ◽  
Victoria Peterson ◽  
Diego H Milone ◽  
Enzo Ferrante
Keyword(s):  
Machine Learning ◽  
Data Model ◽  
Learning Systems ◽  
Training Data ◽  
Model Construction ◽  
Daily Lives ◽  
The Past ◽  
Learning Challenges ◽  
Challenges And Opportunities ◽  
The Relationship

Abstract Machine learning systems influence our daily lives in many different ways. Hence, it is crucial to ensure that the decisions and recommendations made by these systems are fair, equitable, and free of unintended biases. Over the past few years, the field of fairness in machine learning has grown rapidly, investigating how, when, and why these models capture, and even potentiate, biases that are deeply rooted not only in the training data but also in our society. In this Commentary, we discuss challenges and opportunities for rigorous posterior analyses of publicly available data to build fair and equitable machine learning systems, focusing on the importance of training data, model construction, and diversity in the team of developers. The thoughts presented here have grown out of the work we did, which resulted in our winning the annual Research Parasite Award that GigaSciencesponsors.

scholarly journals Integration of New Information in Memory: New Insights from a Complementary Learning Systems Perspective

2020 ◽  
Author(s):  
James L. McClelland ◽  
Bruce L. McNaughton ◽  
Andrew K. Lampinen
Keyword(s):  
Prior Knowledge ◽  
Time Course ◽  
Empirical Studies ◽  
Learning Systems ◽  
Training Data ◽  
Theoretical Understanding ◽  
Linear Networks ◽  
New Information ◽  
The Brain ◽  
New Item

According to complementary learning systems theory, integrating new memories into the neocortex of the brain without interfering with what is already known depends on a gradual learning process, interleaving new items with previously learned items. However, empirical studies show that information consistent with prior knowledge can be integrated very quickly. We use artificial neural networks with properties like those we attribute to the neocortex to develop a theoretical understanding of the role of consistency with prior knowledge in putatively neocortex-like learning systems, providing new insights into when integration will be fast or slow and how integration might be made more efficient when the items to be learned are hierarchically structured. The work relies on deep linear networks that capture the qualitative aspects of the learning dynamics of the more complex non-linear networks used in previous work. The time course of learning in these networks can be linked to the hierarchical structure in the training data, captured mathematically as a set of dimensions that correspond to the branches in the hierarchy. In this context, a new item to be learned can be characterized as having aspects that project onto previously known dimensions, and others that require adding a new branch/dimension. The projection onto the known dimensions can be learned rapidly without interleaving, but learning the new dimension requires gradual interleaved learning. When a new item only overlaps with items within one branch of a hierarchy, interleaving can focus on the previously-known items within this branch, resulting in faster integration with less inter-leaving overall. The discussion considers how the brain might exploit these facts to make learning more efficient and highlights predictions about what aspects of new information might be hard or easy to learn.

scholarly journals Generating Video From Images using GAN and CVAE

2020 ◽  
Vol 8 (5) ◽  
pp. 1401-1404
Keyword(s):  
Machine Learning ◽  
Supervised Learning ◽  
Generative Models ◽  
Generative Model ◽  
Learning Systems ◽  
Training Data ◽  
Variational Autoencoder ◽  
Discriminative Model ◽  
To Come

In a given scene, people can often easily predict a lot of quick future occasions that may occur. However generalized pixel-level expectation in Machine Learning systems is difficult in light of the fact that it struggles with the ambiguity inherent in predicting what's to come. However, the objective of the paper is to concentrate on predicting the dense direction of pixels in a scene — what will move in the scene, where it will travel, and how it will deform through the span of one second for which we propose a conditional variational autoencoder as a solution for this issue. We likewise propose another structure for assessing generative models through an adversarial procedure, wherein we simultaneously train two models, a generative model G that catches the information appropriation, and a discriminative model D that gauges the likelihood that an example originated from the training data instead of G. We focus on two uses of GANs semi-supervised learning, and the age of pictures that human's find visually realistic. We present the Moments in Time Dataset, an enormous scale human-clarified assortment of one million short recordings relating to dynamic situations unfolding within three seconds.

scholarly journals AI for radiographic COVID-19 detection selects shortcuts over signal

2020 ◽  
Author(s):  
Alex J. DeGrave ◽  
Joseph D. Janizek ◽  
Su-In Lee
Keyword(s):  
Artificial Intelligence ◽  
Medical Imaging ◽  
State Of The Art ◽  
Learning Systems ◽  
Chest Radiographs ◽  
Training Data ◽  
Confounding Factors ◽  
External Data ◽  
Computed Tomography Scans ◽  
Explainable Ai

AbstractArtificial intelligence (AI) researchers and radiologists have recently reported AI systems that accurately detect COVID-19 in chest radiographs. However, the robustness of these systems remains unclear. Using state-of-the-art techniques in explainable AI, we demonstrate that recent deep learning systems to detect COVID-19 from chest radiographs rely on confounding factors rather than medical pathology, creating an alarming situation in which the systems appear accurate, but fail when tested in new hospitals. We observe that the approach to obtain training data for these AI systems introduces a nearly ideal scenario for AI to learn these spurious “shortcuts.” Because this approach to data collection has also been used to obtain training data for detection of COVID-19 in computed tomography scans and for medical imaging tasks related to other diseases, our study reveals a far-reaching problem in medical imaging AI. In addition, we show that evaluation of a model on external data is insufficient to ensure AI systems rely on medically relevant pathology, since the undesired “shortcuts” learned by AI systems may not impair performance in new hospitals. These findings demonstrate that explainable AI should be seen as a prerequisite to clinical deployment of ML healthcare models.

scholarly journals Integration of new information in memory: new insights from a complementary learning systems perspective

2020 ◽  
Vol 375 (1799) ◽  
pp. 20190637 ◽  
Author(s):  
James L. McClelland ◽  
Bruce L. McNaughton ◽  
Andrew K. Lampinen
Keyword(s):  
Prior Knowledge ◽  
Time Course ◽  
Empirical Studies ◽  
Learning Systems ◽  
Training Data ◽  
Nonlinear Networks ◽  
New Information ◽  
The Hierarchical Structure ◽  
The Brain ◽  
New Item

According to complementary learning systems theory, integrating new memories into the neocortex of the brain without interfering with what is already known depends on a gradual learning process, interleaving new items with previously learned items. However, empirical studies show that information consistent with prior knowledge can sometimes be integrated very quickly. We use artificial neural networks with properties like those we attribute to the neocortex to develop an understanding of the role of consistency with prior knowledge in putatively neocortex-like learning systems, providing new insights into when integration will be fast or slow and how integration might be made more efficient when the items to be learned are hierarchically structured. The work relies on deep linear networks that capture the qualitative aspects of the learning dynamics of the more complex nonlinear networks used in previous work. The time course of learning in these networks can be linked to the hierarchical structure in the training data, captured mathematically as a set of dimensions that correspond to the branches in the hierarchy. In this context, a new item to be learned can be characterized as having aspects that project onto previously known dimensions, and others that require adding a new branch/dimension. The projection onto the known dimensions can be learned rapidly without interleaving, but learning the new dimension requires gradual interleaved learning. When a new item only overlaps with items within one branch of a hierarchy, interleaving can focus on the previously known items within this branch, resulting in faster integration with less interleaving overall. The discussion considers how the brain might exploit these facts to make learning more efficient and highlights predictions about what aspects of new information might be hard or easy to learn. This article is part of the Theo Murphy meeting issue ‘Memory reactivation: replaying events past, present and future’.

scholarly journals The Conditional Entropy Bottleneck

Entropy ◽  
2020 ◽  
Vol 22 (9) ◽  
pp. 999 ◽  
Author(s):  
Ian Fischer
Keyword(s):  
Machine Learning ◽  
Objective Function ◽  
Failure Modes ◽  
Conditional Entropy ◽  
Learning Systems ◽  
Training Data ◽  
Deterministic Models ◽  
Information Bottleneck ◽  
Adversarial Examples

Much of the field of Machine Learning exhibits a prominent set of failure modes, including vulnerability to adversarial examples, poor out-of-distribution (OoD) detection, miscalibration, and willingness to memorize random labelings of datasets. We characterize these as failures of robust generalization, which extends the traditional measure of generalization as accuracy or related metrics on a held-out set. We hypothesize that these failures to robustly generalize are due to the learning systems retaining too much information about the training data. To test this hypothesis, we propose the Minimum Necessary Information (MNI) criterion for evaluating the quality of a model. In order to train models that perform well with respect to the MNI criterion, we present a new objective function, the Conditional Entropy Bottleneck (CEB), which is closely related to the Information Bottleneck (IB). We experimentally test our hypothesis by comparing the performance of CEB models with deterministic models and Variational Information Bottleneck (VIB) models on a variety of different datasets and robustness challenges. We find strong empirical evidence supporting our hypothesis that MNI models improve on these problems of robust generalization.

Sign in / Sign up

Export Citation Format

Share Document