scholarly journals Stabilizing Adversarial Invariance Induction from Divergence Minimization Perspective

2020 ◽  
Author(s):  
Yusuke Iwasawa ◽  
Kei Akuzawa ◽  
Yutaka Matsuo
Keyword(s):  
Neural Network ◽  
Real World ◽  
Optimization Procedure ◽  
Superior Performance ◽  
Simple Variant ◽  
Invariant Representations ◽  
Real World Datasets ◽  
Divergence Minimization

Adversarial invariance induction (AII) is a generic and powerful framework for enforcing an invariance to nuisance attributes into neural network representations. However, its optimization is often unstable and little is known about its practical behavior. This paper presents an analysis of the reasons for the optimization difficulties and provides a better optimization procedure by rethinking AII from a divergence minimization perspective. Interestingly, this perspective indicates a cause of the optimization difficulties: it does not ensure proper divergence minimization, which is a requirement of the invariant representations. We then propose a simple variant of AII, called invariance induction by discriminator matching, which takes into account the divergence minimization interpretation of the invariant representations. Our method consistently achieves near-optimal invariance in toy datasets with various configurations in which the original AII is catastrophically unstable. Extentive experiments on four real-world datasets also support the superior performance of the proposed method, leading to improved user anonymization and domain generalization.

scholarly journals Attention Enhanced Serial Unet++ Network for Removing Unevenly Distributed Haze

Electronics ◽  
2021 ◽  
Vol 10 (22) ◽  
pp. 2868
Author(s):  
Wenxuan Zhao ◽  
Yaqin Zhao ◽  
Liqi Feng ◽  
Jiaxi Tang
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Real World ◽  
Large Scale ◽  
Learning Strategy ◽  
Contextual Information ◽  
Small Scale ◽  
Image Dehazing ◽  
Atmospheric Scattering ◽  
Real World Datasets

The purpose of image dehazing is the reduction of the image degradation caused by suspended particles for supporting high-level visual tasks. Besides the atmospheric scattering model, convolutional neural network (CNN) has been used for image dehazing. However, the existing image dehazing algorithms are limited in face of unevenly distributed haze and dense haze in real-world scenes. In this paper, we propose a novel end-to-end convolutional neural network called attention enhanced serial Unet++ dehazing network (AESUnet) for single image dehazing. We attempt to build a serial Unet++ structure that adopts a serial strategy of two pruned Unet++ blocks based on residual connection. Compared with the simple Encoder–Decoder structure, the serial Unet++ module can better use the features extracted by encoders and promote contextual information fusion in different resolutions. In addition, we take some improvement measures to the Unet++ module, such as pruning, introducing the convolutional module with ResNet structure, and a residual learning strategy. Thus, the serial Unet++ module can generate more realistic images with less color distortion. Furthermore, following the serial Unet++ blocks, an attention mechanism is introduced to pay different attention to haze regions with different concentrations by learning weights in the spatial domain and channel domain. Experiments are conducted on two representative datasets: the large-scale synthetic dataset RESIDE and the small-scale real-world datasets I-HAZY and O-HAZY. The experimental results show that the proposed dehazing network is not only comparable to state-of-the-art methods for the RESIDE synthetic datasets, but also surpasses them by a very large margin for the I-HAZY and O-HAZY real-world dataset.

scholarly journals Objective-aware Traffic Simulation via Inverse Reinforcement Learning

2021 ◽  
Author(s):  
Guanjie Zheng ◽  
Hanyang Liu ◽  
Kai Xu ◽  
Zhenhui Li
Keyword(s):  
Reinforcement Learning ◽  
Physical Model ◽  
Real World ◽  
Traffic Simulation ◽  
Transportation Systems ◽  
Superior Performance ◽  
Inverse Reinforcement Learning ◽  
Reward Function ◽  
Car Following Model ◽  
Real World Datasets

Traffic simulators act as an essential component in the operating and planning of transportation systems. Conventional traffic simulators usually employ a calibrated physical car-following model to describe vehicles' behaviors and their interactions with traffic environment. However, there is no universal physical model that can accurately predict the pattern of vehicle's behaviors in different situations. A fixed physical model tends to be less effective in a complicated environment given the non-stationary nature of traffic dynamics. In this paper, we formulate traffic simulation as an inverse reinforcement learning problem, and propose a parameter sharing adversarial inverse reinforcement learning model for dynamics-robust simulation learning. Our proposed model is able to imitate a vehicle's trajectories in the real world while simultaneously recovering the reward function that reveals the vehicle's true objective which is invariant to different dynamics. Extensive experiments on synthetic and real-world datasets show the superior performance of our approach compared to state-of-the-art methods and its robustness to variant dynamics of traffic.

scholarly journals Exploiting Heterogeneous Graph Neural Networks with Latent Worker/Task Correlation Information for Label Aggregation in Crowdsourcing

2022 ◽  
Vol 16 (2) ◽  
pp. 1-18
Author(s):  
Hanlu Wu ◽  
Tengfei Ma ◽  
Lingfei Wu ◽  
Fangli Xu ◽  
Shouling Ji
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
State Of The Art ◽  
Superior Performance ◽  
True Label ◽  
Label Aggregation ◽  
Correlation Information ◽  
Real World Datasets ◽  
Graph Neural Networks ◽  
High Level

Crowdsourcing has attracted much attention for its convenience to collect labels from non-expert workers instead of experts. However, due to the high level of noise from the non-experts, a label aggregation model that infers the true label from noisy crowdsourced labels is required. In this article, we propose a novel framework based on graph neural networks for aggregating crowd labels. We construct a heterogeneous graph between workers and tasks and derive a new graph neural network to learn the representations of nodes and the true labels. Besides, we exploit the unknown latent interaction between the same type of nodes (workers or tasks) by adding a homogeneous attention layer in the graph neural networks. Experimental results on 13 real-world datasets show superior performance over state-of-the-art models.

Deep Multi-Instance Multi-Label Learning for Image Annotation

2017 ◽  
Vol 32 (03) ◽  
pp. 1859005 ◽  
Author(s):  
Hai-Feng Guo ◽  
Lixin Han ◽  
Shoubao Su ◽  
Zhou-Bao Sun
Keyword(s):  
Neural Network ◽  
Social Networks ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Real World ◽  
Supervised Classification ◽  
Image Annotation ◽  
Learning Framework ◽  
Recent Advancement ◽  
Real World Datasets

Multi-Instance Multi-Label learning (MIML) is a popular framework for supervised classification where an example is described by multiple instances and associated with multiple labels. Previous MIML approaches have focused on predicting labels for instances. The idea of tackling the problem is to identify its equivalence in the traditional supervised learning framework. Motivated by the recent advancement in deep learning, in this paper, we still consider the problem of predicting labels and attempt to model deep learning in MIML learning framework. The proposed approach enables us to train deep convolutional neural network with images from social networks where images are well labeled, even labeled with several labels or uncorrelated labels. Experiments on real-world datasets demonstrate the effectiveness of our proposed approach.

scholarly journals An Attentional Recurrent Neural Network for Personalized Next Location Recommendation

2020 ◽  
Vol 34 (01) ◽  
pp. 83-90
Author(s):  
Qing Guo ◽  
Zhu Sun ◽  
Jie Zhang ◽  
Yin-Leng Theng
Keyword(s):  
Neural Network ◽  
Random Walk ◽  
Recurrent Neural Network ◽  
Real World ◽  
State Of The Art ◽  
Knowledge Graph ◽  
User Mobility ◽  
Location Recommendation ◽  
Meta Path ◽  
Real World Datasets

Most existing studies on next location recommendation propose to model the sequential regularity of check-in sequences, but suffer from the severe data sparsity issue where most locations have fewer than five following locations. To this end, we propose an Attentional Recurrent Neural Network (ARNN) to jointly model both the sequential regularity and transition regularities of similar locations (neighbors). In particular, we first design a meta-path based random walk over a novel knowledge graph to discover location neighbors based on heterogeneous factors. A recurrent neural network is then adopted to model the sequential regularity by capturing various contexts that govern user mobility. Meanwhile, the transition regularities of the discovered neighbors are integrated via the attention mechanism, which seamlessly cooperates with the sequential regularity as a unified recurrent framework. Experimental results on multiple real-world datasets demonstrate that ARNN outperforms state-of-the-art methods.

Sampling Sparse Representations with Randomized Measurement Langevin Dynamics

2021 ◽  
Vol 15 (2) ◽  
pp. 1-21
Author(s):  
Kafeng Wang ◽  
Haoyi Xiong ◽  
Jiang Bian ◽  
Zhanxing Zhu ◽  
Qian Gao ◽  
...  
Keyword(s):  
Real World ◽  
Kernel Density ◽  
Langevin Dynamics ◽  
Sparse Representations ◽  
Superior Performance ◽  
Algorithm Analysis ◽  
High Dimensional ◽  
Measurement Matrix ◽  
Real World Datasets ◽  
Derivatives Of

Stochastic Gradient Langevin Dynamics (SGLD) have been widely used for Bayesian sampling from certain probability distributions, incorporating derivatives of the log-posterior. With the derivative evaluation of the log-posterior distribution, SGLD methods generate samples from the distribution through performing as a thermostats dynamics that traverses over gradient flows of the log-posterior with certainly controllable perturbation. Even when the density is not known, existing solutions still can first learn the kernel density models from the given datasets, then produce new samples using the SGLD over the kernel density derivatives. In this work, instead of exploring new samples from kernel spaces, a novel SGLD sampler, namely, Randomized Measurement Langevin Dynamics (RMLD) is proposed to sample the high-dimensional sparse representations from the spectral domain of a given dataset. Specifically, given a random measurement matrix for sparse coding, RMLD first derives a novel likelihood evaluator of the probability distribution from the loss function of LASSO, then samples from the high-dimensional distribution using stochastic Langevin dynamics with derivatives of the logarithm likelihood and Metropolis–Hastings sampling. In addition, new samples in low-dimensional measuring spaces can be regenerated using the sampled high-dimensional vectors and the measurement matrix. The algorithm analysis shows that RMLD indeed projects a given dataset into a high-dimensional Gaussian distribution with Laplacian prior, then draw new sparse representation from the dataset through performing SGLD over the distribution. Extensive experiments have been conducted to evaluate the proposed algorithm using real-world datasets. The performance comparisons on three real-world applications demonstrate the superior performance of RMLD beyond baseline methods.

scholarly journals NeuRec: On Nonlinear Transformation for Personalized Ranking

2018 ◽  
Author(s):  
Shuai Zhang ◽  
Lina Yao ◽  
Aixin Sun ◽  
Sen Wang ◽  
Guodong Long ◽  
...  
Keyword(s):  
Neural Network ◽  
Real World ◽  
Real Life ◽  
Interaction Matrix ◽  
Interaction Patterns ◽  
Latent Factors ◽  
Integrated Network ◽  
Ranking Task ◽  
Real World Datasets ◽  
Personalized Ranking

Modeling user-item interaction patterns is an important task for personalized recommendations. Many recommender systems are based on the assumption that there exists a linear relationship between users and items while neglecting the intricacy and non-linearity of real-life historical interactions. In this paper, we propose a neural network based recommendation model (NeuRec) that untangles the complexity of user-item interactions and establish an integrated network to combine non-linear transformation with latent factors. We further design two variants of NeuRec: user-based NeuRec and item-based NeuRec, by focusing on different aspects of the interaction matrix. Extensive experiments on four real-world datasets demonstrated their superior performances on personalized ranking task.

scholarly journals Re-identification risk prediction paradigm using incomplete statistical information and recursive hypergeometric distribution

2021 ◽  
Author(s):  
Zhigang Yang ◽  
Ruyan Wang ◽  
Dapeng Wu ◽  
Boran Yang ◽  
Daizhong Luo
Keyword(s):  
Prediction Model ◽  
Risk Prediction ◽  
Frequency Distribution ◽  
Real World ◽  
Binomial Distribution ◽  
General Rule ◽  
Statistical Information ◽  
Superior Performance ◽  
Real World Datasets ◽  
Huge Challenge

Abstract The dataset anonymization has not eliminated the re-identification risk, the evaluation of which remains a huge challenge, especially given incomplete statistical information. The re-identification risk of individuals depends on their tuple frequency. The paper proposes the recursive hypergeometric (RH) distribution to accurately calculate the tuple frequency and leverages the binomial distribution to approximate the RH distribution and to efficiently predict the re-identification risk of individuals in both generated and real-world datasets. The experimental results show that our tuple frequency based re-identification risk (TFRR) prediction model has a superior performance (average AUC 0.86~0.98) for all types of datasets. Furthermore, we exploit the value dependence knowledge to rectify the prediction result for some subsets (average AUC 0.95~0.98). Our research reveals the general rule of the tuple frequency distribution and enables individuals and regulators to responsively predict the re-identification risk.

OEC-cnn: a simple method for over-exposure correction in photographs

2020 ◽  
Vol 2020 (10) ◽  
pp. 182-1-182-8
Author(s):  
Zhao Gao ◽  
Eran Edirisinghe ◽  
Slava Chesnokov
Keyword(s):  
Neural Network ◽  
Real World ◽  
Dynamic Range ◽  
Daily Life ◽  
Natural Image ◽  
Automatic Correction ◽  
Simple Method ◽  
Real World Datasets ◽  
Imaging Sensors ◽  
Better Than

Over-exposure happens often in daily-life photography due to the range of light far exceeding the capabilities of the limited dynamic range of current imaging sensors. Correcting overexposure aims to recover the fine details from the input. Most of the existing methods are based on manual image pixel manipulation, and therefore are often tedious and time-consuming. In this paper, we present the first convolutional neural network (CNN) capable of inferring the photo-realistic natural image for the single over-exposed photograph. To achieve this, we propose a simple and lightweight Over-Exposure Correction CNN, namely OEC-cnn, and construct a synthesized dataset that covers various scenes and exposure rates to facilitate training. By doing so, we effectively replace the manual fixing operations with an end-toend automatic correction process. Experiments on both synthesized and real-world datasets demonstrate that the proposed approach performs significantly better than existing methods and its simplicity and robustness make it a very useful tool for practical over-exposure correction. Our code and synthesized dataset will be made publicly available.

scholarly journals SpHMC: Spectral Hamiltonian Monte Carlo

2019 ◽  
Vol 33 ◽  
pp. 5516-5524
Author(s):  
Haoyi Xiong ◽  
Kafeng Wang ◽  
Jiang Bian ◽  
Zhanxing Zhu ◽  
Cheng-Zhong Xu ◽  
...  
Keyword(s):  
Monte Carlo ◽  
Probability Distribution ◽  
Real World ◽  
Dimensional Space ◽  
Probability Distributions ◽  
Superior Performance ◽  
High Dimensional ◽  
Hamiltonian Monte Carlo ◽  
Real World Datasets ◽  
Low Dimensional

Stochastic Gradient Hamiltonian Monte Carlo (SGHMC) methods have been widely used to sample from certain probability distributions, incorporating (kernel) density derivatives and/or given datasets. Instead of exploring new samples from kernel spaces, this piece of work proposed a novel SGHMC sampler, namely Spectral Hamiltonian Monte Carlo (SpHMC), that produces the high dimensional sparse representations of given datasets through sparse sensing and SGHMC. Inspired by compressed sensing, we assume all given samples are low-dimensional measurements of certain high-dimensional sparse vectors, while a continuous probability distribution exists in such high-dimensional space. Specifically, given a dictionary for sparse coding, SpHMC first derives a novel likelihood evaluator of the probability distribution from the loss function of LASSO, then samples from the high-dimensional distribution using stochastic Langevin dynamics with derivatives of the logarithm likelihood and Metropolis–Hastings sampling. In addition, new samples in low-dimensional measuring spaces can be regenerated using the sampled high-dimensional vectors and the dictionary. Extensive experiments have been conducted to evaluate the proposed algorithm using real-world datasets. The performance comparisons on three real-world applications demonstrate the superior performance of SpHMC beyond baseline methods.

Sign in / Sign up

Export Citation Format

Share Document