scholarly journals Differentially-private Federated Neural Architecture Search

2020 ◽  
Author(s):  
Ishika Singh ◽  
Haoyi Zhou ◽  
Kunlin Yang ◽  
Meng Ding ◽  
Bill Lin ◽  
...  
Keyword(s):  
Neural Networks ◽  
Differential Privacy ◽  
Random Noise ◽  
Privacy Concerns ◽  
Max Pooling ◽  
Neural Architecture ◽  
Remarkable Progress

Neural architecture search, which aims to automatically search for architectures (e.g., convolution, max pooling) of neural networks that maximize validation performance, has achieved remarkable progress recently. In many application scenarios, several parties would like to collaboratively search for a shared neural architecture by leveraging data from all parties. However, due to privacy concerns, no party wants its data to be seen by other parties. To address this problem, we propose federated neural architecture search (FNAS), where different parties collectively search for a differentiable architecture by exchanging gradients of architecture variables without exposing their data to other parties. To further preserve privacy, we study differentially-private FNAS (DP-FNAS), which adds random noise to the gradients of architecture variables. We provide theoretical guarantees of DP-FNAS in achieving differential privacy. Experiments show that DP-FNAS can search highly-performant neural architectures while protecting the privacy of individual parties. The code is available at https://github.com/UCSD-AI4H/DP-FNAS

scholarly journals Differentially-private Federated Neural Architecture Search

2020 ◽  
Author(s):  
Ishika Singh ◽  
Haoyi Zhou ◽  
Kunlin Yang ◽  
Meng Ding ◽  
Bill Lin ◽  
...  
Keyword(s):  
Neural Networks ◽  
Differential Privacy ◽  
Random Noise ◽  
Privacy Concerns ◽  
Max Pooling ◽  
Neural Architecture ◽  
Remarkable Progress

Neural architecture search, which aims to automatically search for architectures (e.g., convolution, max pooling) of neural networks that maximize validation performance, has achieved remarkable progress recently. In many application scenarios, several parties would like to collaboratively search for a shared neural architecture by leveraging data from all parties. However, due to privacy concerns, no party wants its data to be seen by other parties. To address this problem, we propose federated neural architecture search (FNAS), where different parties collectively search for a differentiable architecture by exchanging gradients of architecture variables without exposing their data to other parties. To further preserve privacy, we study differentially-private FNAS (DP-FNAS), which adds random noise to the gradients of architecture variables. We provide theoretical guarantees of DP-FNAS in achieving differential privacy. Experiments show that DP-FNAS can search highly-performant neural architectures while protecting the privacy of individual parties. The code is available at https://github.com/UCSD-AI4H/DP-FNAS

scholarly journals Inductive learning and local differential privacy for privacy-preserving offloading in mobile edge intelligent systems

2021 ◽  
Author(s):  
Jude TCHAYE-KONDI ◽  
Yanlong Zhai ◽  
Liehuang Zhu
Keyword(s):  
Intelligent Systems ◽  
Differential Privacy ◽  
Inductive Learning ◽  
Random Noise ◽  
Privacy Preserving ◽  
Sensitive Data ◽  
Privacy Concerns ◽  
Feature Extractor ◽  
Data Source ◽  
Series Of Experiments

<div>We address privacy and latency issues in the edge/cloud computing environment while training a centralized AI model. In our particular case, the edge devices are the only data source for the model to train on the central server. Current privacy-preserving and reducing network latency solutions rely on a pre-trained feature extractor deployed on the devices to help extract only important features from the sensitive dataset. However, finding a pre-trained model or pubic dataset to build a feature extractor for certain tasks may turn out to be very challenging. With the large amount of data generated by edge devices, the edge environment does not really lack data, but its improper access may lead to privacy concerns. In this paper, we present DeepGuess , a new privacy-preserving, and latency aware deeplearning framework. DeepGuess uses a new learning mechanism enabled by the AutoEncoder(AE) architecture called Inductive Learning, which makes it possible to train a central neural network using the data produced by end-devices while preserving their privacy. With inductive learning, sensitive data remains on devices and is not explicitly involved in any backpropagation process. The AE’s Encoder is deployed on devices to extracts and transfers important features to the server. To enhance privacy, we propose a new local deferentially private algorithm that allows the Edge devices to apply random noise to features extracted from their sensitive data before transferred to an untrusted server. The experimental evaluation of DeepGuess demonstrates its effectiveness and ability to converge on a series of experiments.</div>

Inductive learning and local differential privacy for privacy-preserving offloading in mobile edge intelligent systems

2021 ◽  
Author(s):  
Jude TCHAYE-KONDI ◽  
Yanlong Zhai ◽  
Liehuang Zhu
Keyword(s):  
Intelligent Systems ◽  
Differential Privacy ◽  
Inductive Learning ◽  
Random Noise ◽  
Privacy Preserving ◽  
Sensitive Data ◽  
Privacy Concerns ◽  
Feature Extractor ◽  
Data Source ◽  
Series Of Experiments

<div>We address privacy and latency issues in the edge/cloud computing environment while training a centralized AI model. In our particular case, the edge devices are the only data source for the model to train on the central server. Current privacy-preserving and reducing network latency solutions rely on a pre-trained feature extractor deployed on the devices to help extract only important features from the sensitive dataset. However, finding a pre-trained model or pubic dataset to build a feature extractor for certain tasks may turn out to be very challenging. With the large amount of data generated by edge devices, the edge environment does not really lack data, but its improper access may lead to privacy concerns. In this paper, we present DeepGuess , a new privacy-preserving, and latency aware deeplearning framework. DeepGuess uses a new learning mechanism enabled by the AutoEncoder(AE) architecture called Inductive Learning, which makes it possible to train a central neural network using the data produced by end-devices while preserving their privacy. With inductive learning, sensitive data remains on devices and is not explicitly involved in any backpropagation process. The AE’s Encoder is deployed on devices to extracts and transfers important features to the server. To enhance privacy, we propose a new local deferentially private algorithm that allows the Edge devices to apply random noise to features extracted from their sensitive data before transferred to an untrusted server. The experimental evaluation of DeepGuess demonstrates its effectiveness and ability to converge on a series of experiments.</div>

scholarly journals Inductive learning and local differential privacy for privacy-preserving offloading in mobile edge intelligent systems

2021 ◽  
Author(s):  
Jude TCHAYE-KONDI ◽  
Yanlong Zhai ◽  
Liehuang Zhu
Keyword(s):  
Intelligent Systems ◽  
Differential Privacy ◽  
Inductive Learning ◽  
Random Noise ◽  
Privacy Preserving ◽  
Sensitive Data ◽  
Privacy Concerns ◽  
Feature Extractor ◽  
Data Source ◽  
Series Of Experiments

<div>We address privacy and latency issues in the edge/cloud computing environment while training a centralized AI model. In our particular case, the edge devices are the only data source for the model to train on the central server. Current privacy-preserving and reducing network latency solutions rely on a pre-trained feature extractor deployed on the devices to help extract only important features from the sensitive dataset. However, finding a pre-trained model or pubic dataset to build a feature extractor for certain tasks may turn out to be very challenging. With the large amount of data generated by edge devices, the edge environment does not really lack data, but its improper access may lead to privacy concerns. In this paper, we present DeepGuess , a new privacy-preserving, and latency aware deeplearning framework. DeepGuess uses a new learning mechanism enabled by the AutoEncoder(AE) architecture called Inductive Learning, which makes it possible to train a central neural network using the data produced by end-devices while preserving their privacy. With inductive learning, sensitive data remains on devices and is not explicitly involved in any backpropagation process. The AE’s Encoder is deployed on devices to extracts and transfers important features to the server. To enhance privacy, we propose a new local deferentially private algorithm that allows the Edge devices to apply random noise to features extracted from their sensitive data before transferred to an untrusted server. The experimental evaluation of DeepGuess demonstrates its effectiveness and ability to converge on a series of experiments.</div>

Location- and Person-Independent Activity Recognition with WiFi, Deep Neural Networks, and Reinforcement Learning

2021 ◽  
Vol 2 (1) ◽  
pp. 1-25
Author(s):  
Yongsen Ma ◽  
Sheheryar Arshad ◽  
Swetha Muniraju ◽  
Eric Torkildson ◽  
Enrico Rantala ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Reinforcement Learning ◽  
Activity Recognition ◽  
Deep Neural Networks ◽  
State Machine ◽  
Recognition Algorithm ◽  
The State ◽  
Neural Architecture ◽  
Learning Agent

In recent years, Channel State Information (CSI) measured by WiFi is widely used for human activity recognition. In this article, we propose a deep learning design for location- and person-independent activity recognition with WiFi. The proposed design consists of three Deep Neural Networks (DNNs): a 2D Convolutional Neural Network (CNN) as the recognition algorithm, a 1D CNN as the state machine, and a reinforcement learning agent for neural architecture search. The recognition algorithm learns location- and person-independent features from different perspectives of CSI data. The state machine learns temporal dependency information from history classification results. The reinforcement learning agent optimizes the neural architecture of the recognition algorithm using a Recurrent Neural Network (RNN) with Long Short-Term Memory (LSTM). The proposed design is evaluated in a lab environment with different WiFi device locations, antenna orientations, sitting/standing/walking locations/orientations, and multiple persons. The proposed design has 97% average accuracy when testing devices and persons are not seen during training. The proposed design is also evaluated by two public datasets with accuracy of 80% and 83%. The proposed design needs very little human efforts for ground truth labeling, feature engineering, signal processing, and tuning of learning parameters and hyperparameters.

Using Neural Architecture Search to Optimize Neural Networks for Embedded Devices

2019 ◽  
pp. 684-693
Author(s):  
Thomas Cassimon ◽  
Simon Vanneste ◽  
Stig Bosmans ◽  
Siegfried Mercelis ◽  
Peter Hellinckx
Keyword(s):  
Neural Networks ◽  
Embedded Devices ◽  
Neural Architecture

scholarly journals The use of Convolutional Neural Networks for signal-background classification in Particle Physics experiments

2020 ◽  
Vol 245 ◽  
pp. 06003
Author(s):  
Venkitesh Ayyar ◽  
Wahid Bhimji ◽  
Lisa Gerhardt ◽  
Sally Robertson ◽  
Zahra Ronaghi
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Particle Physics ◽  
Simulated Data ◽  
Image Data ◽  
Use Case ◽  
Neural Architecture ◽  
Ice Cube ◽  
2D And 3D ◽  
Physics Experiments

The success of Convolutional Neural Networks (CNNs) in image classification has prompted efforts to study their use for classifying image data obtained in Particle Physics experiments. Here, we discuss our efforts to apply CNNs to 2D and 3D image data from particle physics experiments to classify signal from background. In this work we present an extensive convolutional neural architecture search, achieving high accuracy for signal/background discrimination for a HEP classification use-case based on simulated data from the Ice Cube neutrino observatory and an ATLAS-like detector. We demonstrate among other things that we can achieve the same accuracy as complex ResNet architectures with CNNs with less parameters, and present comparisons of computational requirements, training and inference times.

A Multi-Objective Evolutionary Approach Based on Graph-in-Graph for Neural Architecture Search of Convolutional Neural Networks

2021 ◽  
pp. 2150035
Author(s):  
Yu Xue ◽  
Pengcheng Jiang ◽  
Ferrante Neri ◽  
Jiayu Liang
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Network Structure ◽  
Ad Hoc ◽  
Network Design Problem ◽  
Multi Objective ◽  
Neural Architecture ◽  
Mutation Operators ◽  
Small Structure ◽  
High Flexibility

With the development of deep learning, the design of an appropriate network structure becomes fundamental. In recent years, the successful practice of Neural Architecture Search (NAS) has indicated that an automated design of the network structure can efficiently replace the design performed by human experts. Most NAS algorithms make the assumption that the overall structure of the network is linear and focus solely on accuracy to assess the performance of candidate networks. This paper introduces a novel NAS algorithm based on a multi-objective modeling of the network design problem to design accurate Convolutional Neural Networks (CNNs) with a small structure. The proposed algorithm makes use of a graph-based representation of the solutions which enables a high flexibility in the automatic design. Furthermore, the proposed algorithm includes novel ad-hoc crossover and mutation operators. We also propose a mechanism to accelerate the evaluation of the candidate solutions. Experimental results demonstrate that the proposed NAS approach can design accurate neural networks with limited size.

Sign in / Sign up

Export Citation Format

Share Document