scholarly journals A Neuron Noise-Injection Technique for Privacy Preserving Deep Neural Networks

2020 ◽  
Vol 10 (1) ◽  
pp. 137-152
Author(s):  
Tosin A. Adesuyi ◽  
Byeong Man Kim
Keyword(s):  
Differential Privacy ◽  
Real Life ◽  
Privacy Preserving ◽  
Training Dataset ◽  
Injection Technique ◽  
Sensitive Information ◽  
Contribution Ratio ◽  
Noise Injection ◽  
Real World Datasets ◽  
The Right

AbstractData is the key to information mining that unveils hidden knowledge. The ability to revealed knowledge relies on the extractable features of a dataset and likewise the depth of the mining model. Conversely, several of these datasets embed sensitive information that can engender privacy violation and are subsequently used to build deep neural network (DNN) models. Recent approaches to enact privacy and protect data sensitivity in DNN models does decline accuracy, thus, giving rise to significant accuracy disparity between a non-private DNN and a privacy preserving DNN model. This accuracy gap is due to the enormous uncalculated noise flooding and the inability to quantify the right level of noise required to perturb distinct neurons in the DNN model, hence, a dent in accuracy. Consequently, this has hindered the use of privacy protected DNN models in real life applications. In this paper, we present a neuron noise-injection technique based on layer-wise buffered contribution ratio forwarding and ϵ-differential privacy technique to preserve privacy in a DNN model. We adapt a layer-wise relevance propagation technique to compute contribution ratio for each neuron in our network at the pre-training phase. Based on the proportion of each neuron’s contribution ratio, we generate a noise-tuple via the Laplace mechanism, and this helps to eliminate unwanted noise flooding. The noise-tuple is subsequently injected into the training network through its neurons to preserve privacy of the training dataset in a differentially private manner. Hence, each neuron receives right proportion of noise as estimated via contribution ratio, and as a result, unquantifiable noise that drops accuracy of privacy preserving DNN models is avoided. Extensive experiments were conducted based on three real-world datasets and their results show that our approach was able to narrow down the existing accuracy gap to a close proximity, as well outperforms the state-of-the-art approaches in this context.

scholarly journals A Survey and Experimental Study on Privacy-Preserving Trajectory Data Publishing

2021 ◽  
Author(s):  
Fengmei Jin ◽  
Wen Hua ◽  
Matteo Francia ◽  
Pingfu Chao ◽  
Maria Orlowska ◽  
...  
Keyword(s):  
Privacy Protection ◽  
Traffic Management ◽  
Real Life ◽  
Privacy Preserving ◽  
Location Based Services ◽  
Data Publishing ◽  
Sensitive Information ◽  
Mobility Patterns ◽  
Trajectory Data ◽  
The Right

<div>Trajectory data has become ubiquitous nowadays, which can benefit various real-world applications such as traffic management and location-based services. However, trajectories may disclose highly sensitive information of an individual including mobility patterns, personal profiles and gazetteers, social relationships, etc, making it indispensable to consider privacy protection when releasing trajectory data. Ensuring privacy on trajectories demands more than hiding single locations, since trajectories are intrinsically sparse and high-dimensional, and require to protect multi-scale correlations. To this end, extensive research has been conducted to design effective techniques for privacy-preserving trajectory data publishing. Furthermore, protecting privacy requires carefully balance two metrics: privacy and utility. In other words, it needs to protect as much privacy as possible and meanwhile guarantee the usefulness of the released trajectories for data analysis. In this survey, we provide a comprehensive study and systematic summarization of existing protection models, privacy and utility metrics for trajectories developed in the literature. We also conduct extensive experiments on a real-life public trajectory dataset to evaluate the performance of several representative privacy protection models, demonstrate the trade-off between privacy and utility, and guide the choice of the right privacy model for trajectory publishing given certain privacy and utility desiderata.</div>

scholarly journals A Survey and Experimental Study on Privacy-Preserving Trajectory Data Publishing

2021 ◽  
Author(s):  
Fengmei Jin ◽  
Wen Hua ◽  
Matteo Francia ◽  
Pingfu Chao ◽  
Maria Orlowska ◽  
...  
Keyword(s):  
Privacy Protection ◽  
Traffic Management ◽  
Real Life ◽  
Privacy Preserving ◽  
Location Based Services ◽  
Data Publishing ◽  
Sensitive Information ◽  
Mobility Patterns ◽  
Trajectory Data ◽  
The Right

<div>Trajectory data has become ubiquitous nowadays, which can benefit various real-world applications such as traffic management and location-based services. However, trajectories may disclose highly sensitive information of an individual including mobility patterns, personal profiles and gazetteers, social relationships, etc, making it indispensable to consider privacy protection when releasing trajectory data. Ensuring privacy on trajectories demands more than hiding single locations, since trajectories are intrinsically sparse and high-dimensional, and require to protect multi-scale correlations. To this end, extensive research has been conducted to design effective techniques for privacy-preserving trajectory data publishing. Furthermore, protecting privacy requires carefully balance two metrics: privacy and utility. In other words, it needs to protect as much privacy as possible and meanwhile guarantee the usefulness of the released trajectories for data analysis. In this survey, we provide a comprehensive study and systematic summarization of existing protection models, privacy and utility metrics for trajectories developed in the literature. We also conduct extensive experiments on a real-life public trajectory dataset to evaluate the performance of several representative privacy protection models, demonstrate the trade-off between privacy and utility, and guide the choice of the right privacy model for trajectory publishing given certain privacy and utility desiderata.</div>

scholarly journals Differential Privacy for Stackelberg Games

2020 ◽  
Author(s):  
Ferdinando Fioretto ◽  
Lesia Mitridati ◽  
Pascal Van Hentenryck
Keyword(s):  
Electricity Market ◽  
Original Problem ◽  
Differential Privacy ◽  
Stackelberg Game ◽  
Privacy Preserving ◽  
Coordination Mechanism ◽  
Stackelberg Games ◽  
Sensitive Information ◽  
Near Optimality

This paper introduces a differentially private (DP) mechanism to protect the information exchanged during the coordination of sequential and interdependent markets. This coordination represents a classic Stackelberg game and relies on the exchange of sensitive information between the system agents. The paper is motivated by the observation that the perturbation introduced by traditional DP mechanisms fundamentally changes the underlying optimization problem and even leads to unsatisfiable instances. To remedy such limitation, the paper introduces the Privacy-Preserving Stackelberg Mechanism (PPSM), a framework that enforces the notions of feasibility and fidelity (i.e. near-optimality) of the privacy-preserving information to the original problem objective. PPSM complies with the notion of differential privacy and ensures that the outcomes of the privacy-preserving coordination mechanism are close-to-optimality for each agent. Experimental results on several gas and electricity market benchmarks based on a real case study demonstrate the effectiveness of the proposed approach. A full version of this paper [Fioretto et al., 2020b] contains complete proofs and additional discussion on the motivating application.

scholarly journals Inference attacks against differentially private query results from genomic datasets including dependent tuples

2020 ◽  
Vol 36 (Supplement_1) ◽  
pp. i136-i145
Author(s):  
Nour Almadhoun ◽  
Erman Ayday ◽  
Özgür Ulusoy
Keyword(s):  
Complex Traits ◽  
Differential Privacy ◽  
Clinical Care ◽  
Real Life ◽  
Ratio Test ◽  
Sensitive Information ◽  
Chi Square ◽  
Significant Information ◽  
Inference Attacks ◽  
Inference Attack

Abstract Motivation The rapid decrease in the sequencing technology costs leads to a revolution in medical research and clinical care. Today, researchers have access to large genomic datasets to study associations between variants and complex traits. However, availability of such genomic datasets also results in new privacy concerns about personal information of the participants in genomic studies. Differential privacy (DP) is one of the rigorous privacy concepts, which received widespread interest for sharing summary statistics from genomic datasets while protecting the privacy of participants against inference attacks. However, DP has a known drawback as it does not consider the correlation between dataset tuples. Therefore, privacy guarantees of DP-based mechanisms may degrade if the dataset includes dependent tuples, which is a common situation for genomic datasets due to the inherent correlations between genomes of family members. Results In this article, using two real-life genomic datasets, we show that exploiting the correlation between the dataset participants results in significant information leak from differentially private results of complex queries. We formulate this as an attribute inference attack and show the privacy loss in minor allele frequency (MAF) and chi-square queries. Our results show that using the results of differentially private MAF queries and utilizing the dependency between tuples, an adversary can reveal up to 50% more sensitive information about the genome of a target (compared to original privacy guarantees of standard DP-based mechanisms), while differentially privacy chi-square queries can reveal up to 40% more sensitive information. Furthermore, we show that the adversary can use the inferred genomic data obtained from the attribute inference attack to infer the membership of a target in another genomic dataset (e.g. associated with a sensitive trait). Using a log-likelihood-ratio test, our results also show that the inference power of the adversary can be significantly high in such an attack even using inferred (and hence partially incorrect) genomes. Availability and implementation https://github.com/nourmadhoun/Inference-Attacks-Differential-Privacy

scholarly journals Achieving the Optimal k-Anonymity for Content Privacy in Interactive Cyberphysical Systems

2018 ◽  
Vol 2018 ◽  
pp. 1-15 ◽  
Author(s):  
Jinbao Wang ◽  
Ling Tian ◽  
Yan Huang ◽  
Donghua Yang ◽  
Hong Gao
Keyword(s):  
Health Care ◽  
Differential Privacy ◽  
Daily Life ◽  
Real Life ◽  
Privacy Preserving ◽  
Location Based Services ◽  
Cyberphysical Systems ◽  
Privacy Concerns ◽  
Automatic Driving ◽  
Made In

Modern applications and services leveraged by interactive cyberphysical systems (CPS) are providing significant convenience to our daily life in various aspects at present. Clients submit their requests including query contents to CPS servers to enjoy diverse services such as health care, automatic driving, and location-based services. However, privacy concerns arise at the same time. Content privacy is recognized and a lot of efforts have been made in the literature of privacy preserving in interactive cyberphysical systems such as location-based services. Nevertheless, neither the cloaking based solutions nor existing client based solutions have achieved effective content privacy by optimizing proper content privacy metrics. In this paper we formulate the problem of achieving the optimal content privacy in interactive cyberphysical systems using k-anonymity solutions based on two content privacy metrics, which are defined using the concepts of entropy and differential privacy. Then we propose an algorithm, Multilayer Alignment (MLA), to establish k-anonymity mechanisms for preserving content privacy in interactive cyberphysical systems. Our proposed MLA is theoretically proved to achieve the optimal content privacy in terms of both the entropy based and the differential privacy mannered content privacy metrics. Evaluation based on real-life datasets is conducted, and the evaluation results validate the effectiveness of our proposed algorithm.

scholarly journals A Defense Framework for Privacy Risks in Remote Machine Learning Service

2021 ◽  
Vol 2021 ◽  
pp. 1-13
Author(s):  
Yang Bai ◽  
Yu Li ◽  
Mingchuang Xie ◽  
Mingyu Fan
Keyword(s):  
Machine Learning ◽  
Differential Privacy ◽  
Original Data ◽  
Privacy Preserving ◽  
Training Data ◽  
Sensitive Information ◽  
Learning Approaches ◽  
Local Data ◽  
Sensitive Data ◽  
Privacy Risks

In recent years, machine learning approaches have been widely adopted for many applications, including classification. Machine learning models deal with collective sensitive data usually trained in a remote public cloud server, for instance, machine learning as a service (MLaaS) system. In this scene, users upload their local data and utilize the computation capability to train models, or users directly access models trained by MLaaS. Unfortunately, recent works reveal that the curious server (that trains the model with users’ sensitive local data and is curious to know the information about individuals) and the malicious MLaaS user (who abused to query from the MLaaS system) will cause privacy risks. The adversarial method as one of typical mitigation has been studied by several recent works. However, most of them focus on the privacy-preserving against the malicious user; in other words, they commonly consider the data owner and the model provider as one role. Under this assumption, the privacy leakage risks from the curious server are neglected. Differential privacy methods can defend against privacy threats from both the curious sever and the malicious MLaaS user by directly adding noise to the training data. Nonetheless, the differential privacy method will decrease the classification accuracy of the target model heavily. In this work, we propose a generic privacy-preserving framework based on the adversarial method to defend both the curious server and the malicious MLaaS user. The framework can adapt with several adversarial algorithms to generate adversarial examples directly with data owners’ original data. By doing so, sensitive information about the original data is hidden. Then, we explore the constraint conditions of this framework which help us to find the balance between privacy protection and the model utility. The experiments’ results show that our defense framework with the AdvGAN method is effective against MIA and our defense framework with the FGSM method can protect the sensitive data from direct content exposed attacks. In addition, our method can achieve better privacy and utility balance compared to the existing method.

scholarly journals Variational Bayes In Private Settings (VIPS)

2020 ◽  
Vol 68 ◽  
pp. 109-157
Author(s):  
Mijung Park ◽  
James Foulds ◽  
Kamalika Chaudhuri ◽  
Max Welling
Keyword(s):  
Large Scale ◽  
Latent Dirichlet Allocation ◽  
Probabilistic Models ◽  
Data Augmentation ◽  
Differential Privacy ◽  
Variational Bayes ◽  
Sensitive Information ◽  
Bayesian Data Analysis ◽  
Bayes Algorithm ◽  
Real World Datasets

Many applications of Bayesian data analysis involve sensitive information such as personal documents or medical records, motivating methods which ensure that privacy is protected. We introduce a general privacy-preserving framework for Variational Bayes (VB), a widely used optimization-based Bayesian inference method. Our framework respects differential privacy, the gold-standard privacy criterion, and encompasses a large class of probabilistic models, called the Conjugate Exponential (CE) family. We observe that we can straightforwardly privatise VB’s approximate posterior distributions for models in the CE family, by perturbing the expected sufficient statistics of the complete-data likelihood. For a broadly-used class of non-CE models, those with binomial likelihoods, we show how to bring such models into the CE family, such that inferences in the modified model resemble the private variational Bayes algorithm as closely as possible, using the Pólya-Gamma data augmentation scheme. The iterative nature of variational Bayes presents a further challenge since iterations increase the amount of noise needed. We overcome this by combining: (1) an improved composition method for differential privacy, called the moments accountant, which provides a tight bound on the privacy cost of multiple VB iterations and thus significantly decreases the amount of additive noise; and (2) the privacy amplification effect of subsampling mini-batches from large-scale data in stochastic learning. We empirically demonstrate the effectiveness of our method in CE and non-CE models including latent Dirichlet allocation, Bayesian logistic regression, and sigmoid belief networks, evaluated on real-world datasets.

scholarly journals Investigating Statistical Privacy Frameworks from the Perspective of Hypothesis Testing

2019 ◽  
Vol 2019 (3) ◽  
pp. 233-254 ◽  
Author(s):  
Changchang Liu ◽  
Xi He ◽  
Thee Chanyaswad ◽  
Shiqiang Wang ◽  
Prateek Mittal
Keyword(s):  
Hypothesis Testing ◽  
Gold Standard ◽  
Prior Distribution ◽  
Differential Privacy ◽  
State Of The Art ◽  
Auxiliary Information ◽  
Privacy Preserving ◽  
Sensitive Information ◽  
Statistical Tool ◽  
Practical Guidelines

Abstract Over the last decade, differential privacy (DP) has emerged as the gold standard of a rigorous and provable privacy framework. However, there are very few practical guidelines on how to apply differential privacy in practice, and a key challenge is how to set an appropriate value for the privacy parameter ɛ. In this work, we employ a statistical tool called hypothesis testing for discovering useful and interpretable guidelines for the state-of-the-art privacy-preserving frameworks. We formalize and implement hypothesis testing in terms of an adversary’s capability to infer mutually exclusive sensitive information about the input data (such as whether an individual has participated or not) from the output of the privacy-preserving mechanism. We quantify the success of the hypothesis testing using the precision- recall-relation, which provides an interpretable and natural guideline for practitioners and researchers on selecting ɛ. Our key results include a quantitative analysis of how hypothesis testing can guide the choice of the privacy parameter ɛ in an interpretable manner for a differentially private mechanism and its variants. Importantly, our findings show that an adversary’s auxiliary information - in the form of prior distribution of the database and correlation across records and time - indeed influences the proper choice of ɛ. Finally, we also show how the perspective of hypothesis testing can provide useful insights on the relationships among a broad range of privacy frameworks including differential privacy, Pufferfish privacy, Blowfish privacy, dependent differential privacy, inferential privacy, membership privacy and mutual-information based differential privacy.

scholarly journals Pairwise Learning with Differential Privacy Guarantees

2020 ◽  
Vol 34 (01) ◽  
pp. 694-701
Author(s):  
Mengdi Huai ◽  
Di Wang ◽  
Chenglin Miao ◽  
Jinhui Xu ◽  
Aidong Zhang
Keyword(s):  
Differential Privacy ◽  
Metric Learning ◽  
Loss Functions ◽  
Sensitive Information ◽  
Training Set ◽  
Pairwise Learning ◽  
Auc Maximization ◽  
General Convex ◽  
Convex Loss ◽  
Real World Datasets

Pairwise learning has received much attention recently as it is more capable of modeling the relative relationship between pairs of samples. Many machine learning tasks can be categorized as pairwise learning, such as AUC maximization and metric learning. Existing techniques for pairwise learning all fail to take into consideration a critical issue in their design, i.e., the protection of sensitive information in the training set. Models learned by such algorithms can implicitly memorize the details of sensitive information, which offers opportunity for malicious parties to infer it from the learned models. To address this challenging issue, in this paper, we propose several differentially private pairwise learning algorithms for both online and offline settings. Specifically, for the online setting, we first introduce a differentially private algorithm (called OnPairStrC) for strongly convex loss functions. Then, we extend this algorithm to general convex loss functions and give another differentially private algorithm (called OnPairC). For the offline setting, we also present two differentially private algorithms (called OffPairStrC and OffPairC) for strongly and general convex loss functions, respectively. These proposed algorithms can not only learn the model effectively from the data but also provide strong privacy protection guarantee for sensitive information in the training set. Extensive experiments on real-world datasets are conducted to evaluate the proposed algorithms and the experimental results support our theoretical analysis.

scholarly journals On Privacy Protection of Latent Dirichlet Allocation Model Training

2019 ◽  
Author(s):  
Fangyuan Zhao ◽  
Xuebin Ren ◽  
Shusen Yang ◽  
Xinyu Yang
Keyword(s):  
Machine Learning ◽  
Latent Dirichlet Allocation ◽  
Differential Privacy ◽  
Machine Learning Algorithms ◽  
Sensitive Information ◽  
Training Algorithm ◽  
Allocation Model ◽  
Model Training ◽  
Real World Datasets ◽  
Dirichlet Allocation

Latent Dirichlet Allocation (LDA) is a popular topic modeling technique for discovery of hidden semantic architecture of text datasets, and plays a fundamental role in many machine learning applications. However, like many other machine learning algorithms, the process of training a LDA model may leak the sensitive information of the training datasets and bring significant privacy risks. To mitigate the privacy issues in LDA, we focus on studying privacy-preserving algorithms of LDA model training in this paper. In particular, we first develop a privacy monitoring algorithm to investigate the privacy guarantee obtained from the inherent randomness of the Collapsed Gibbs Sampling (CGS) process in a typical LDA training algorithm on centralized curated datasets. Then, we further propose a locally private LDA training algorithm on crowdsourced data to provide local differential privacy for individual data contributors. The experimental results on real-world datasets demonstrate the effectiveness of our proposed algorithms.

Sign in / Sign up

Export Citation Format

Share Document