Vector approximate message passing algorithm for compressed sensing with structured matrix perturbation

2020 ◽  
Vol 166 ◽  
pp. 107248 ◽  
Author(s):  
Jiang Zhu ◽  
Qi Zhang ◽  
Xiangming Meng ◽  
Zhiwei Xu
Keyword(s):  
Compressed Sensing ◽  
Message Passing ◽  
Matrix Perturbation ◽  
Approximate Message Passing ◽  
Structured Matrix ◽  
Message Passing Algorithm

Matrix inference and estimation in multi-layer models*

2021 ◽  
Vol 2021 (12) ◽  
pp. 124004
Author(s):  
Parthe Pandit ◽  
Mojtaba Sahraee-Ardakan ◽  
Sundeep Rangan ◽  
Philip Schniter ◽  
Alyson K Fletcher
Keyword(s):  
Neural Network ◽  
Message Passing ◽  
Hidden Variables ◽  
Signal Recovery ◽  
Inference Problem ◽  
Neural Network Learning ◽  
Approximate Message Passing ◽  
Message Passing Algorithm ◽  
Training Samples ◽  
Mixed Regression

Abstract We consider the problem of estimating the input and hidden variables of a stochastic multi-layer neural network (NN) from an observation of the output. The hidden variables in each layer are represented as matrices with statistical interactions along both rows as well as columns. This problem applies to matrix imputation, signal recovery via deep generative prior models, multi-task and mixed regression, and learning certain classes of two-layer NNs. We extend a recently-developed algorithm—multi-layer vector approximate message passing, for this matrix-valued inference problem. It is shown that the performance of the proposed multi-layer matrix vector approximate message passing algorithm can be exactly predicted in a certain random large-system limit, where the dimensions N × d of the unknown quantities grow as N → ∞ with d fixed. In the two-layer neural-network learning problem, this scaling corresponds to the case where the number of input features as well as training samples grow to infinity but the number of hidden nodes stays fixed. The analysis enables a precise prediction of the parameter and test error of the learning.

scholarly journals Performance Analysis of Approximate Message Passing for Distributed Compressed Sensing

2018 ◽  
Vol 12 (5) ◽  
pp. 857-870 ◽  
Author(s):  
Gabor Hannak ◽  
Alessandro Perelli ◽  
Norbert Goertz ◽  
Gerald Matz ◽  
Mike E. Davies
Keyword(s):  
Performance Analysis ◽  
Compressed Sensing ◽  
Message Passing ◽  
Distributed Compressed Sensing ◽  
Approximate Message Passing
Sign in / Sign up

Export Citation Format

Share Document