Learning Model with Error -- Exposing the Hidden Model of BAYHENN

Privacy-preserving deep neural network (DNN) inference remains an intriguing problem even after the rapid developments of different communities. One challenge is that cryptographic techniques such as homomorphic encryption (HE) do not natively support non-linear computations (e.g., sigmoid). A recent work, BAYHENN (Xie et al., IJCAI'19), considers HE over the Bayesian neural network (BNN). The novelty lies in "meta-prediction" over a few noisy DNNs. The claim was that the clients can get intermediate outputs (to apply non-linear function) but are still prevented from learning the exact model parameters, which was justified via the widely-used learning-with-error (LWE) assumption (with Gaussian noises as the error). This paper refutes the security claim of BAYHENN via both theoretical and empirical analyses. We formally define a security game with different oracle queries capturing two realistic threat models. Our attack assuming a semi-honest adversary reveals all the parameters of single-layer BAYHENN, which generalizes to recovering the whole model that is "as good as" the BNN approximation of the original DNN, either under the malicious adversary model or with an increased number of oracle queries. This shows the need for rigorous security analysis ("the noise introduced by BNN can obfuscate the model" fails -- it is beyond what LWE guarantees) and calls for the collaboration between cryptographers and machine-learning experts to devise practical yet provably-secure solutions.

Download Full-text

Application of Machine Learning in Battery: State of Charge Estimation Using Feed Forward Neural Network for Sodium-Ion Battery

Electrochem ◽

10.3390/electrochem3010003 ◽

2022 ◽

Vol 3 (1) ◽

pp. 42-57

Author(s):

Devendrasinh Darbar ◽

Indranil Bhattacharya

Keyword(s):

Neural Network ◽

Equivalent Circuit Model ◽

Reference Electrode ◽

State Of Charge ◽

Training Data ◽

Model Parameters ◽

Feed Forward Neural Network ◽

Feed Forward ◽

Non Linear ◽

Battery State Of Charge

Estimating the accurate State of Charge (SOC) of a battery is important to avoid the over/undercharging and protect the battery pack from low cycle life. Current methods of SOC estimation use complex equations in the Extended Kalman Filter (EKF) and the equivalent circuit model. In this paper, we used a Feed Forward Neural Network (FNN) to estimate the SOC value accurately where battery parameters such as current, voltage, and charge are mapped directly to the SOC value at the output. A FNN could self-learn the weights with each training data point and update the model parameters such as weights and bias using a combination of two gradient descents (Adam). This model comprises the Dropout technique, which can have many neural network architectures by dropping the neuron/mode at each epoch/training cycle using the same weights and biases. Our FNN model was trained with data comprising different current rates and tested for different cycling data, for example, 5th, 10th, 20th, and 50th cycles and at a different cutoff voltage (4.5 V). The battery used for estimating the SOC value was a Na-ion based battery, which is highly non-linear, and it was fabricated in a house using Na0.67Fe0.5Mn0.5O2 (NFM) as a cathode and Na metal as a reference electrode. The FNN successfully estimated the SOC value for the highly non-linear nature of the Na-ion battery at different current rates (0.05 C, 0.1 C, 0.5 C, 1 C, 2 C), for different cycling data, and at higher cut-off voltage of –4.5 V Na+, reaching the R2 value of ~0.97–~0.99, ~0.99, and ~0.98, respectively.

Download Full-text

Bayesian inference of non-linear multiscale model parameters accelerated by a Deep Neural Network

Computer Methods in Applied Mechanics and Engineering ◽

10.1016/j.cma.2019.112693 ◽

2020 ◽

Vol 360 ◽

pp. 112693 ◽

Cited By ~ 3

Author(s):

Ling Wu ◽

Kepa Zulueta ◽

Zoltan Major ◽

Aitor Arriaga ◽

Ludovic Noels

Keyword(s):

Neural Network ◽

Bayesian Inference ◽

Deep Neural Network ◽

Multiscale Model ◽

Model Parameters ◽

Non Linear

Download Full-text

Neural Computing Enhanced Parameter Estimation for Multi-Input and Multi-Output Total Non-Linear Dynamic Models

Entropy ◽

10.3390/e22050510 ◽

2020 ◽

Vol 22 (5) ◽

pp. 510 ◽

Cited By ~ 2

Author(s):

Longlong Liu ◽

Di Ma ◽

Ahmad Taher Azar ◽

Quanmin Zhu

Keyword(s):

Neural Network ◽

Parameter Estimation ◽

Linear Model ◽

Dynamic Models ◽

Bench Test ◽

Model Parameters ◽

Linear Dynamic ◽

Gradient Descent Algorithm ◽

Non Linear ◽

Linear Dynamic Models

In this paper, a gradient descent algorithm is proposed for the parameter estimation of multi-input and multi-output (MIMO) total non-linear dynamic models. Firstly, the MIMO total non-linear model is mapped to a non-completely connected feedforward neural network, that is, the parameters of the total non-linear model are mapped to the connection weights of the neural network. Then, based on the minimization of network error, a weight-updating algorithm, that is, an estimation algorithm of model parameters, is proposed with the convergence conditions of a non-completely connected feedforward network. In further determining the variables of the model set, a method of model structure detection is proposed for selecting a group of important items from the whole variable candidate set. In order to verify the usefulness of the parameter identification process, we provide a virtual bench test example for the numerical analysis and user-friendly instructions for potential applications.

Download Full-text

Applying Hierarchical Bayesian Neural Network in Failure Time Prediction

Mathematical Problems in Engineering ◽

10.1155/2012/953848 ◽

2012 ◽

Vol 2012 ◽

pp. 1-11 ◽

Cited By ~ 1

Author(s):

Ling-Jing Kao ◽

Hsin-Fen Chen

Keyword(s):

Neural Network ◽

Failure Time ◽

Training Sample ◽

Parameter Estimates ◽

Model Parameters ◽

Hierarchical Bayesian ◽

Finite Sample ◽

Bayesian Neural Network ◽

Time Prediction ◽

Proposed Model

With the rapid technology development and improvement, the product failure time prediction becomes an even harder task because only few failures in the product life tests are recorded. The classical statistical model relies on the asymptotic theory and cannot guarantee that the estimator has the finite sample property. To solve this problem, we apply the hierarchical Bayesian neural network (HBNN) approach to predict the failure time and utilize the Gibbs sampler of Markov chain Monte Carlo (MCMC) to estimate model parameters. In this proposed method, the hierarchical structure is specified to study the heterogeneity among products. Engineers can use the heterogeneity estimates to identify the causes of the quality differences and further enhance the product quality. In order to demonstrate the effectiveness of the proposed hierarchical Bayesian neural network model, the prediction performance of the proposed model is evaluated using multiple performance measurement criteria. Sensitivity analysis of the proposed model is also conducted using different number of hidden nodes and training sample sizes. The result shows that HBNN can provide not only the predictive distribution but also the heterogeneous parameter estimates for each path.

Download Full-text

Privacy-Preserving Federated Neural Network Learning for Disease-Associated Cell Classification

10.1101/2022.01.10.475610 ◽

2022 ◽

Author(s):

Sinem Sav ◽

Jean-Philippe Bossuat ◽

Juan R. Troncoso-Pastoriza ◽

Manfred Claassen ◽

Jean-Pierre Hubaux

Keyword(s):

Neural Network ◽

Machine Learning ◽

Neural Networks ◽

Network Architecture ◽

Homomorphic Encryption ◽

Privacy Preserving ◽

Model Parameters ◽

Patient Privacy ◽

Learning Models ◽

Machine Learning Models

Training accurate and robust machine learning models requires a large amount of data that is usually scattered across data-silos. Sharing or centralizing the data of different healthcare institutions is, however, unfeasible or prohibitively difficult due to privacy regulations. In this work, we address this problem by using a novel privacy-preserving federated learning-based approach, PriCell, for complex machine learning models such as convolutional neural networks. PriCell relies on multiparty homomorphic encryption and enables the collaborative training of encrypted neural networks with multiple healthcare institutions. We preserve the confidentiality of each institutions' input data, of any intermediate values, and of the trained model parameters. We efficiently replicate the training of a published state-of-the-art convolutional neural network architecture in a decentralized and privacy-preserving manner. Our solution achieves an accuracy comparable to the one obtained with the centralized solution, with an improvement of at least one-order-of-magnitude in execution time with respect to prior secure solutions. Our work guarantees patient privacy and ensures data utility for efficient multi-center studies involving complex healthcare data.

Download Full-text

Neural Network Modelling Applied for Model-Based Fault Detection

Volume 3 ◽

10.1115/esda2004-58197 ◽

2004 ◽

Author(s):

Zhanqun Shi ◽

Yibo Fan ◽

Fengshou Gu ◽

Abdul-Hannan Ali ◽

Andrew Ball

Keyword(s):

Neural Network ◽

Mathematical Model ◽

Fault Detection ◽

Linear Part ◽

Model Parameters ◽

Network Modelling ◽

Model Based ◽

Neural Network Modelling ◽

Non Linear ◽

Non Linear System

This paper aims to combine neural network modelling with model-based fault detection. An accurate and robust model is critical in model-based fault detection. However, the development of such a model is the most difficult task especially when a non-linear system is involved. The problem comes not only from the lack of concerned information about model parameters, but also from the inevitable linearization. In order to solve this problem, neural networks are introduced in this paper. Instead of using conventional neural network modelling, the neural network is only used to approximate the non-linear part of the system, leaving the linear part to be represented by a mathematical model. This new scheme of integration between neural network and mathematical model (NNMM) allows the compensation of the error from conventional modelling methods. Simultaneously, it keeps the residual signatures physically interpretable.

Download Full-text

Variational Bayesian Neural Network for Ensemble Flood Forecasting

Water ◽

10.3390/w12102740 ◽

2020 ◽

Vol 12 (10) ◽

pp. 2740

Author(s):

Xiaoyan Zhan ◽

Hui Qin ◽

Yongqi Liu ◽

Liqiang Yao ◽

Wei Xie ◽

...

Keyword(s):

Neural Network ◽

Gaussian Process Regression ◽

Flood Forecasting ◽

Economic Losses ◽

Model Parameters ◽

Probabilistic Forecasting ◽

Bayesian Neural Network ◽

Forecast Uncertainty ◽

Variational Bayesian ◽

Forecasting Models

Disastrous floods are destructive and likely to cause widespread economic losses. An understanding of flood forecasting and its potential forecast uncertainty is essential for water resource managers. Reliable forecasting may provide future streamflow information to assist in an assessment of the benefits of reservoirs and the risk of flood disasters. However, deterministic forecasting models are not able to provide forecast uncertainty information. To quantify the forecast uncertainty, a variational Bayesian neural network (VBNN) model for ensemble flood forecasting is proposed in this study. In VBNN, the posterior distribution is approximated by the variational distribution, which can avoid the heavy computational costs in the traditional Bayesian neural network. To transform the model parameters’ uncertainty into the model output uncertainty, a Monte Carlo sample is applied to give ensemble forecast results. The proposed method is verified by a flood forecasting case study on the upper Yangtze River. A point forecasting model neural network and two probabilistic forecasting models, including hidden Markov Model and Gaussian process regression, are also applied to compare with the proposed model. The experimental results show that the VBNN performs better than other comparable models in terms of both accuracy and reliability. Finally, the result of uncertainty estimation shows that the VBNN can effectively handle heteroscedastic flood streamflow data.

Download Full-text

Optimizing Single-Layer Raster Cellular Neural Network Simulator Using Simulated Annealing Technique and RK4(2), RK4(3) and RK 6(4)

International Journal of Scientific Research ◽

10.15373/22778179/june2013/35 ◽

2012 ◽

Vol 2 (6) ◽

pp. 108-112

Author(s):

O. H. Abdelwahed O. H. Abdelwahed ◽

◽

M. El-Sayed Wahed

Keyword(s):

Neural Network ◽

Simulated Annealing ◽

Single Layer ◽

Cellular Neural Network ◽

Network Simulator

Download Full-text

Adaptive control of time-varying non-linear plants by speed-gradient algorithms

Information and Control Systems ◽

10.31799/1684-8853-2019-3-37-44 ◽

2019 ◽

pp. 37-44 ◽

Cited By ~ 1

Author(s):

O. P. Tomchina ◽

D. N. Polyakhov ◽

O. I. Tokareva ◽

A. L. Fradkov

Keyword(s):

Adaptive Control ◽

Adaptive Systems ◽

Reference Model ◽

Maximum Deviation ◽

Model Parameters ◽

Time Varying ◽

System Solution ◽

Non Linear ◽

Initial Uncertainty ◽

Speed Gradient

Introduction: The motion of many real world systems is described by essentially non-linear and non-stationary models. A number of approaches to the control of such plants are based on constructing an internal model of non-stationarity. However, the non-stationarity model parameters can vary widely, leading to more errors. It is only assumed in this paper that the change rate of the object parameters is limited, while the initial uncertainty can be quite large.Purpose: Analysis of adaptive control algorithms for non-linear and time-varying systems with an explicit reference model, synthesized by the speed gradient method.Results: An estimate was obtained for the maximum deviation of a closed-loop system solution from the reference model solution. It is shown that with sufficiently slow changes in the parameters and a small initial uncertainty, the limit error in the system can be made arbitrarily small. Systems designed by the direct approach and systems based on the identification approach are both considered. The procedures for the synthesis of an adaptive regulator and analysis of the synthesized system are illustrated by an example.Practical relevance: The obtained results allow us to build and analyze a broad class of adaptive systems with reference models under non-stationary conditions.

Download Full-text