Comparing Local and Central Differential Privacy Using Membership Inference Attacks

Abstract A membership inference attack (MIA) against a machine-learning model enables an attacker to determine whether a given data record was part of the model’s training data or not. In this paper, we provide an in-depth study of the phenomenon of disparate vulnerability against MIAs: unequal success rate of MIAs against different population subgroups. We first establish necessary and sufficient conditions for MIAs to be prevented, both on average and for population subgroups, using a notion of distributional generalization. Second, we derive connections of disparate vulnerability to algorithmic fairness and to differential privacy. We show that fairness can only prevent disparate vulnerability against limited classes of adversaries. Differential privacy bounds disparate vulnerability but can significantly reduce the accuracy of the model. We show that estimating disparate vulnerability by naïvely applying existing attacks can lead to overestimation. We then establish which attacks are suitable for estimating disparate vulnerability, and provide a statistical framework for doing so reliably. We conduct experiments on synthetic and real-world data finding significant evidence of disparate vulnerability in realistic settings.

Download Full-text

Inference attacks against differentially private query results from genomic datasets including dependent tuples

Bioinformatics ◽

10.1093/bioinformatics/btaa475 ◽

2020 ◽

Vol 36 (Supplement_1) ◽

pp. i136-i145

Author(s):

Nour Almadhoun ◽

Erman Ayday ◽

Özgür Ulusoy

Keyword(s):

Complex Traits ◽

Differential Privacy ◽

Clinical Care ◽

Real Life ◽

Ratio Test ◽

Sensitive Information ◽

Chi Square ◽

Significant Information ◽

Inference Attacks ◽

Inference Attack

Abstract Motivation The rapid decrease in the sequencing technology costs leads to a revolution in medical research and clinical care. Today, researchers have access to large genomic datasets to study associations between variants and complex traits. However, availability of such genomic datasets also results in new privacy concerns about personal information of the participants in genomic studies. Differential privacy (DP) is one of the rigorous privacy concepts, which received widespread interest for sharing summary statistics from genomic datasets while protecting the privacy of participants against inference attacks. However, DP has a known drawback as it does not consider the correlation between dataset tuples. Therefore, privacy guarantees of DP-based mechanisms may degrade if the dataset includes dependent tuples, which is a common situation for genomic datasets due to the inherent correlations between genomes of family members. Results In this article, using two real-life genomic datasets, we show that exploiting the correlation between the dataset participants results in significant information leak from differentially private results of complex queries. We formulate this as an attribute inference attack and show the privacy loss in minor allele frequency (MAF) and chi-square queries. Our results show that using the results of differentially private MAF queries and utilizing the dependency between tuples, an adversary can reveal up to 50% more sensitive information about the genome of a target (compared to original privacy guarantees of standard DP-based mechanisms), while differentially privacy chi-square queries can reveal up to 40% more sensitive information. Furthermore, we show that the adversary can use the inferred genomic data obtained from the attribute inference attack to infer the membership of a target in another genomic dataset (e.g. associated with a sensitive trait). Using a log-likelihood-ratio test, our results also show that the inference power of the adversary can be significantly high in such an attack even using inferred (and hence partially incorrect) genomes. Availability and implementation https://github.com/nourmadhoun/Inference-Attacks-Differential-Privacy

Download Full-text

Emerging Technologies in a Modern Competitive Scenario

Digital Transformation and Challenges to Data Security and Privacy - Advances in Information Security, Privacy, and Ethics ◽

10.4018/978-1-7998-4201-9.ch001 ◽

2021 ◽

pp. 1-16

Author(s):

George Leal Jamil ◽

Alexis Rocha da Silva

Keyword(s):

Machine Learning ◽

Neural Networks ◽

Statistical Analysis ◽

Differential Privacy ◽

Social Issues ◽

Training Data ◽

Sensitive Data ◽

Machine Learning Model ◽

Highly Sensitive ◽

Inference Attacks

Users' personal, highly sensitive data such as photos and voice recordings are kept indefinitely by the companies that collect it. Users can neither delete nor restrict the purposes for which it is used. Learning how to machine learning that protects privacy, we can make a huge difference in solving many social issues like curing disease, etc. Deep neural networks are susceptible to various inference attacks as they remember information about their training data. In this chapter, the authors introduce differential privacy, which ensures that different kinds of statistical analysis don't compromise privacy and federated learning, training a machine learning model on a data to which we do not have access to.

Download Full-text

An Overview on Protecting User Private-Attribute Information on Social Networks

Handbook of Research on Cyber Crime and Information Privacy - Advances in Information Security, Privacy, and Ethics ◽

10.4018/978-1-7998-5728-0.ch006 ◽

2021 ◽

pp. 102-117

Author(s):

Walaa Alnasser ◽

Ghazaleh Beigi ◽

Huan Liu

Keyword(s):

Social Networks ◽

Social Media ◽

Online Social Networks ◽

Differential Privacy ◽

Future Research ◽

Online Publishing ◽

Social Media Data ◽

User Data ◽

Inference Attacks ◽

Media Data

Online social networks enable users to participate in different activities, such as connecting with each other and sharing different contents online. These activities lead to the generation of vast amounts of user data online. Publishing user-generated data causes the problem of user privacy as this data includes information about users' private and sensitive attributes. This privacy issue mandates social media data publishers to protect users' privacy by anonymizing user-generated social media data. Existing private-attribute inference attacks can be classified into two classes: friend-based private-attribute attacks and behavior-based private-attribute attacks. Consequently, various privacy protection models are proposed to protect users against private-attribute inference attacks such as k-anonymity and differential privacy. This chapter will overview and compare recent state-of-the-art researches in terms of private-attribute inference attacks and corresponding anonymization techniques. In addition, open problems and future research directions will be discussed.

Download Full-text

A Differential Privacy Framework for Collaborative Filtering

Mathematical Problems in Engineering ◽

10.1155/2019/1460234 ◽

2019 ◽

Vol 2019 ◽

pp. 1-11 ◽

Cited By ~ 2

Author(s):

Jing Yang ◽

Xiaoye Li ◽

Zhenlong Sun ◽

Jianpei Zhang

Keyword(s):

Collaborative Filtering ◽

Recommender Systems ◽

Perturbation Methods ◽

Differential Privacy ◽

Feasible Solution ◽

Experimental Results ◽

Input Method ◽

Recommendation Algorithms ◽

Inference Attacks ◽

Privacy Issues

Focusing on the privacy issues in recommender systems, we propose a framework containing two perturbation methods for differentially private collaborative filtering to prevent the threat of inference attacks against users. To conceal individual ratings and provide valuable predictions, we consider some representative algorithms to calculate the predicted scores and provide specific solutions for adding Laplace noise. The DPI (Differentially Private Input) method perturbs the original ratings, which can be followed by any recommendation algorithms. By contrast, the DPM (Differentially Private Manner) method is based on the original ratings, which perturbs the measurements during implementation of the algorithms and releases the predicted scores. The experimental results showed that both methods can provide valuable prediction results while guaranteeing DP, which suggests it is a feasible solution and can be competent to make private recommendations.

Download Full-text

Analysis of Application Examples of Differential Privacy in Deep Learning

Computational Intelligence and Neuroscience ◽

10.1155/2021/4244040 ◽

2021 ◽

Vol 2021 ◽

pp. 1-15

Author(s):

Zhidong Shen ◽

Ting Zhong

Keyword(s):

Artificial Intelligence ◽

Neural Networks ◽

Deep Learning ◽

Deep Neural Networks ◽

Differential Privacy ◽

Generative Adversarial Networks ◽

Learning Models ◽

Privacy Leakage ◽

Adversarial Networks ◽

Inference Attacks

Artificial Intelligence has been widely applied today, and the subsequent privacy leakage problems have also been paid attention to. Attacks such as model inference attacks on deep neural networks can easily extract user information from neural networks. Therefore, it is necessary to protect privacy in deep learning. Differential privacy, as a popular topic in privacy-preserving in recent years, which provides rigorous privacy guarantee, can also be used to preserve privacy in deep learning. Although many articles have proposed different methods to combine differential privacy and deep learning, there are no comprehensive papers to analyze and compare the differences and connections between these technologies. For this purpose, this paper is proposed to compare different differential private methods in deep learning. We comparatively analyze and classify several deep learning models under differential privacy. Meanwhile, we also pay attention to the application of differential privacy in Generative Adversarial Networks (GANs), comparing and analyzing these models. Finally, we summarize the application of differential privacy in deep neural networks.

Download Full-text

Federated Model Distillation with Noise-Free Differential Privacy

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2021/216 ◽

2021 ◽

Author(s):

Lichao Sun ◽

Lingjuan Lyu

Keyword(s):

Data Privacy ◽

Differential Privacy ◽

Random Noise ◽

Model Performance ◽

Training Data ◽

Heterogeneous Model ◽

The Public ◽

Inference Attacks ◽

Public Datasets ◽

Privacy Budget

Conventional federated learning directly averages model weights, which is only possible for collaboration between models with homogeneous architectures. Sharing prediction instead of weight removes this obstacle and eliminates the risk of white-box inference attacks in conventional federated learning. However, the predictions from local models are sensitive and would leak training data privacy to the public. To address this issue, one naive approach is adding the differentially private random noise to the predictions, which however brings a substantial trade-off between privacy budget and model performance. In this paper, we propose a novel framework called FEDMD-NFDP, which applies a Noise-FreeDifferential Privacy (NFDP) mechanism into a federated model distillation framework. Our extensive experimental results on various datasets validate that FEDMD-NFDP can deliver not only comparable utility and communication efficiency but also provide a noise-free differential privacy guarantee. We also demonstrate the feasibility of our FEDMD-NFDP by considering both IID and Non-IID settings, heterogeneous model architectures, and unlabelled public datasets from a different distribution.

Download Full-text