scholarly journals Likelihood-free Out-of-Distribution Detection with Invertible Generative Models

2021 ◽  
Author(s):  
Amirhossein Ahmadian ◽  
Fredrik Lindsten
Keyword(s):  
Score Function ◽  
Generative Models ◽  
Generative Model ◽  
Training Data ◽  
Detection Methods ◽  
Representation Space ◽  
New Space ◽  
Low Dimensional ◽  
And Training ◽  
Low Dimensional Features

Likelihood of generative models has been used traditionally as a score to detect atypical (Out-of-Distribution, OOD) inputs. However, several recent studies have found this approach to be highly unreliable, even with invertible generative models, where computing the likelihood is feasible. In this paper, we present a different framework for generative model--based OOD detection that employs the model in constructing a new representation space, instead of using it directly in computing typicality scores, where it is emphasized that the score function should be interpretable as the similarity between the input and training data in the new space. In practice, with a focus on invertible models, we propose to extract low-dimensional features (statistics) based on the model encoder and complexity of input images, and then use a One-Class SVM to score the data. Contrary to recently proposed OOD detection methods for generative models, our method does not require computing likelihood values. Consequently, it is much faster when using invertible models with iteratively approximated likelihood (e.g. iResNet), while it still has a performance competitive with other related methods.

scholarly journals Monte Carlo and Reconstruction Membership Inference Attacks against Generative Models

2019 ◽  
Vol 2019 (4) ◽  
pp. 232-249 ◽  
Author(s):  
Benjamin Hilprecht ◽  
Martin Härterich ◽  
Daniel Bernau
Keyword(s):  
Data Privacy ◽  
Information Leakage ◽  
Generative Models ◽  
Generative Model ◽  
Training Data ◽  
Generative Adversarial Networks ◽  
Data Sets ◽  
Success Rates ◽  
Model Quality ◽  
Type Formalization

Abstract We present two information leakage attacks that outperform previous work on membership inference against generative models. The first attack allows membership inference without assumptions on the type of the generative model. Contrary to previous evaluation metrics for generative models, like Kernel Density Estimation, it only considers samples of the model which are close to training data records. The second attack specifically targets Variational Autoencoders, achieving high membership inference accuracy. Furthermore, previous work mostly considers membership inference adversaries who perform single record membership inference. We argue for considering regulatory actors who perform set membership inference to identify the use of specific datasets for training. The attacks are evaluated on two generative model architectures, Generative Adversarial Networks (GANs) and Variational Autoen-coders (VAEs), trained on standard image datasets. Our results show that the two attacks yield success rates superior to previous work on most data sets while at the same time having only very mild assumptions. We envision the two attacks in combination with the membership inference attack type formalization as especially useful. For example, to enforce data privacy standards and automatically assessing model quality in machine learning as a service setups. In practice, our work motivates the use of GANs since they prove less vulnerable against information leakage attacks while producing detailed samples.

scholarly journals Generative Adversarial Network Performance in Low-Dimensional Settings

2021 ◽  
Vol 126 ◽  
Author(s):  
Felix Jimenez ◽  
Amanda Koepke ◽  
Mary Gregg ◽  
Michael Frey
Keyword(s):  
Neural Network ◽  
Network Performance ◽  
Training Data ◽  
High Dimensional ◽  
Generative Adversarial Network ◽  
Low Dimensions ◽  
Target Distribution ◽  
Adversarial Network ◽  
Low Dimensional ◽  
And Training

A generative adversarial network (GAN) is an artificial neural network with a distinctive training architecture, designed to createexamples that faithfully reproduce a target distribution. GANs have recently had particular success in applications involvinghigh-dimensional distributions in areas such as image processing. Little work has been reported for low dimensions, where properties of GANs may be better identified and understood. We studied GAN performance in simulated low-dimensional settings, allowing us totransparently assess effects of target distribution complexity and training data sample size on GAN performance in a simpleexperiment. This experiment revealed two important forms of GAN error, tail underfilling and bridge bias, where the latter is analogousto the tunneling observed in high-dimensional GANs.

scholarly journals Generating Video From Images using GAN and CVAE

2020 ◽  
Vol 8 (5) ◽  
pp. 1401-1404
Keyword(s):  
Machine Learning ◽  
Supervised Learning ◽  
Generative Models ◽  
Generative Model ◽  
Learning Systems ◽  
Training Data ◽  
Variational Autoencoder ◽  
Discriminative Model ◽  
To Come

In a given scene, people can often easily predict a lot of quick future occasions that may occur. However generalized pixel-level expectation in Machine Learning systems is difficult in light of the fact that it struggles with the ambiguity inherent in predicting what's to come. However, the objective of the paper is to concentrate on predicting the dense direction of pixels in a scene — what will move in the scene, where it will travel, and how it will deform through the span of one second for which we propose a conditional variational autoencoder as a solution for this issue. We likewise propose another structure for assessing generative models through an adversarial procedure, wherein we simultaneously train two models, a generative model G that catches the information appropriation, and a discriminative model D that gauges the likelihood that an example originated from the training data instead of G. We focus on two uses of GANs semi-supervised learning, and the age of pictures that human's find visually realistic. We present the Moments in Time Dataset, an enormous scale human-clarified assortment of one million short recordings relating to dynamic situations unfolding within three seconds.

scholarly journals Spectral estimation for detecting low-dimensional structure in networks using arbitrary null models

PLoS ONE ◽  
2021 ◽  
Vol 16 (7) ◽  
pp. e0254057
Author(s):  
Mark D. Humphries ◽  
Javier A. Caballero ◽  
Mat Evans ◽  
Silvia Maggi ◽  
Abhinav Singh
Keyword(s):  
Community Structure ◽  
Real World ◽  
Spectral Estimation ◽  
Null Model ◽  
Generative Models ◽  
Detection Methods ◽  
Dimensional Structure ◽  
Spectral Approach ◽  
Low Dimensional ◽  
Noise Nodes

Discovering low-dimensional structure in real-world networks requires a suitable null model that defines the absence of meaningful structure. Here we introduce a spectral approach for detecting a network’s low-dimensional structure, and the nodes that participate in it, using any null model. We use generative models to estimate the expected eigenvalue distribution under a specified null model, and then detect where the data network’s eigenspectra exceed the estimated bounds. On synthetic networks, this spectral estimation approach cleanly detects transitions between random and community structure, recovers the number and membership of communities, and removes noise nodes. On real networks spectral estimation finds either a significant fraction of noise nodes or no departure from a null model, in stark contrast to traditional community detection methods. Across all analyses, we find the choice of null model can strongly alter conclusions about the presence of network structure. Our spectral estimation approach is therefore a promising basis for detecting low-dimensional structure in real-world networks, or lack thereof.

scholarly journals Improving Molecular Design by Stochastic Iterative Target Augmentation

2020 ◽  
Author(s):  
Kevin Yang ◽  
Wengong Jin ◽  
Kyle Swanson ◽  
Regina Barzilay ◽  
Tommi S Jaakkola
Keyword(s):  
Molecular Design ◽  
Generative Models ◽  
Generative Model ◽  
Training Data ◽  
Neural Models ◽  
Training Approach ◽  
Likelihood Model ◽  
Previous State ◽  
Structured Objects ◽  
Stochastic Em

Generative models in molecular design tend to be richly parameterized, data-hungry neural models, as they must create complex structured objects as outputs. Estimating such models from data may be challenging due to the lack of sufficient training data. In this paper, we propose a surprisingly effective self-training approach for iteratively creating additional molecular targets. We first pre-train the generative model together with a simple property predictor. The property predictor is then used as a likelihood model for filtering candidate structures from the generative model. Additional targets are iteratively produced and used in the course of stochastic EM iterations to maximize the log-likelihood that the candidate structures are accepted. A simple rejection (re-weighting) sampler suffices to draw posterior samples since the generative model is already reasonable after pre-training. We demonstrate significant gains over strong baselines for both unconditional and conditional molecular design. In particular, our approach outperforms the previous state-of-the-art in conditional molecular design by over 10% in absolute gain.

scholarly journals Improving Molecular Design by Stochastic Iterative Target Augmentation

2020 ◽  
Author(s):  
Kevin Yang ◽  
Wengong Jin ◽  
Kyle Swanson ◽  
Regina Barzilay ◽  
Tommi S Jaakkola
Keyword(s):  
Molecular Design ◽  
Generative Models ◽  
Generative Model ◽  
Training Data ◽  
Neural Models ◽  
Training Approach ◽  
Likelihood Model ◽  
Previous State ◽  
Structured Objects ◽  
Stochastic Em

Generative models in molecular design tend to be richly parameterized, data-hungry neural models, as they must create complex structured objects as outputs. Estimating such models from data may be challenging due to the lack of sufficient training data. In this paper, we propose a surprisingly effective self-training approach for iteratively creating additional molecular targets. We first pre-train the generative model together with a simple property predictor. The property predictor is then used as a likelihood model for filtering candidate structures from the generative model. Additional targets are iteratively produced and used in the course of stochastic EM iterations to maximize the log-likelihood that the candidate structures are accepted. A simple rejection (re-weighting) sampler suffices to draw posterior samples since the generative model is already reasonable after pre-training. We demonstrate significant gains over strong baselines for both unconditional and conditional molecular design. In particular, our approach outperforms the previous state-of-the-art in conditional molecular design by over 10% in absolute gain.

scholarly journals Face Verification Using Modeled Eigenspectrum

2008 ◽  
Vol 2 (1) ◽  
pp. 35-45
Author(s):  
Bappaditya Manda ◽  
Xudong Jiang ◽  
Alex Kot
Keyword(s):  
Training Data ◽  
Face Verification ◽  
Face Identification ◽  
Verification Task ◽  
Identification Task ◽  
Linear Discriminant ◽  
Highly Sensitive ◽  
Whole Space ◽  
Low Dimensional ◽  
Low Dimensional Features

Face verification is different from face identification task. Some traditional subspace methods that work well in face identification may suffer from severe over-fitting problem when applied for the verification task. Conventional discriminative methods such as linear discriminant analysis (LDA) and its variants are highly sensitive to the training data, which hinders them from achieving high verification accuracy. This work proposes an eigenspectrum model that alleviates the over-fitting problems by replacing the unreliable small and zero eigenvalues with the model values. It also enables the discriminant evaluation in the whole space to extract the low dimensional features effectively. The proposed approach is evaluated and compared with 8 popular subspace based methods for a face verification task. Experimental results on three face databases show that the proposed method consistently outperforms others.

Cloning Safe Driving Behavior for Self-Driving Cars using Convolutional Neural Networks

2019 ◽  
Vol 12 (2) ◽  
pp. 120-127 ◽  
Author(s):  
Wael Farag
Keyword(s):  
Gradient Descent ◽  
Autonomous Driving ◽  
Driving Behavior ◽  
Training Data ◽  
Stochastic Gradient Descent ◽  
Data Set ◽  
Safe Driving ◽  
Processing Pipeline ◽  
Self Driving Cars ◽  
And Training

Background: In this paper, a Convolutional Neural Network (CNN) to learn safe driving behavior and smooth steering manoeuvring, is proposed as an empowerment of autonomous driving technologies. The training data is collected from a front-facing camera and the steering commands issued by an experienced driver driving in traffic as well as urban roads. Methods: This data is then used to train the proposed CNN to facilitate what it is called “Behavioral Cloning”. The proposed Behavior Cloning CNN is named as “BCNet”, and its deep seventeen-layer architecture has been selected after extensive trials. The BCNet got trained using Adam’s optimization algorithm as a variant of the Stochastic Gradient Descent (SGD) technique. Results: The paper goes through the development and training process in details and shows the image processing pipeline harnessed in the development. Conclusion: The proposed approach proved successful in cloning the driving behavior embedded in the training data set after extensive simulations.

scholarly journals Building Damage Detection from Post-Event Aerial Imagery Using Single Shot Multibox Detector

2019 ◽  
Vol 9 (6) ◽  
pp. 1128 ◽  
Author(s):  
Yundong Li ◽  
Wei Hu ◽  
Han Dong ◽  
Xueyan Zhang
Keyword(s):  
Machine Learning ◽  
Data Augmentation ◽  
Hurricane Sandy ◽  
Training Data ◽  
Aerial Images ◽  
Detection Methods ◽  
Single Shot ◽  
Data Set ◽  
Augmentation Strategies ◽  
Post Disaster

Using aerial cameras, satellite remote sensing or unmanned aerial vehicles (UAV) equipped with cameras can facilitate search and rescue tasks after disasters. The traditional manual interpretation of huge aerial images is inefficient and could be replaced by machine learning-based methods combined with image processing techniques. Given the development of machine learning, researchers find that convolutional neural networks can effectively extract features from images. Some target detection methods based on deep learning, such as the single-shot multibox detector (SSD) algorithm, can achieve better results than traditional methods. However, the impressive performance of machine learning-based methods results from the numerous labeled samples. Given the complexity of post-disaster scenarios, obtaining many samples in the aftermath of disasters is difficult. To address this issue, a damaged building assessment method using SSD with pretraining and data augmentation is proposed in the current study and highlights the following aspects. (1) Objects can be detected and classified into undamaged buildings, damaged buildings, and ruins. (2) A convolution auto-encoder (CAE) that consists of VGG16 is constructed and trained using unlabeled post-disaster images. As a transfer learning strategy, the weights of the SSD model are initialized using the weights of the CAE counterpart. (3) Data augmentation strategies, such as image mirroring, rotation, Gaussian blur, and Gaussian noise processing, are utilized to augment the training data set. As a case study, aerial images of Hurricane Sandy in 2012 were maximized to validate the proposed method’s effectiveness. Experiments show that the pretraining strategy can improve of 10% in terms of overall accuracy compared with the SSD trained from scratch. These experiments also demonstrate that using data augmentation strategies can improve mAP and mF1 by 72% and 20%, respectively. Finally, the experiment is further verified by another dataset of Hurricane Irma, and it is concluded that the paper method is feasible.

Sign in / Sign up

Export Citation Format

Share Document