Sequential Feature Screening for Generalized Linear Models with Sparse Ultra-High Dimensional Data

2020 ◽  
Vol 33 (2) ◽  
pp. 510-526
Author(s):  
Junying Zhang ◽  
Hang Wang ◽  
Riquan Zhang ◽  
Jiajia Zhang
Keyword(s):  
Generalized Linear Models ◽  
Linear Models ◽  
High Dimensional Data ◽  
High Dimensional ◽  
Feature Screening

Variable selection in high-dimensional double generalized linear models

2012 ◽  
Vol 55 (2) ◽  
pp. 327-347 ◽  
Author(s):  
Dengke Xu ◽  
Zhongzhan Zhang ◽  
Liucang Wu
Keyword(s):  
Variable Selection ◽  
Generalized Linear Models ◽  
Linear Models ◽  
High Dimensional

scholarly journals Optimal errors and phase transitions in high-dimensional generalized linear models

2019 ◽  
Vol 116 (12) ◽  
pp. 5451-5460 ◽  
Author(s):  
Jean Barbier ◽  
Florent Krzakala ◽  
Nicolas Macris ◽  
Léo Miolane ◽  
Lenka Zdeborová
Keyword(s):  
Phase Transitions ◽  
Generalized Linear Models ◽  
Message Passing ◽  
Statistical Physics ◽  
Linear Models ◽  
Optimal Estimation ◽  
Error Correcting Codes ◽  
Data Matrix ◽  
High Dimensional ◽  
Special Cases

Generalized linear models (GLMs) are used in high-dimensional machine learning, statistics, communications, and signal processing. In this paper we analyze GLMs when the data matrix is random, as relevant in problems such as compressed sensing, error-correcting codes, or benchmark models in neural networks. We evaluate the mutual information (or “free entropy”) from which we deduce the Bayes-optimal estimation and generalization errors. Our analysis applies to the high-dimensional limit where both the number of samples and the dimension are large and their ratio is fixed. Nonrigorous predictions for the optimal errors existed for special cases of GLMs, e.g., for the perceptron, in the field of statistical physics based on the so-called replica method. Our present paper rigorously establishes those decades-old conjectures and brings forward their algorithmic interpretation in terms of performance of the generalized approximate message-passing algorithm. Furthermore, we tightly characterize, for many learning problems, regions of parameters for which this algorithm achieves the optimal performance and locate the associated sharp phase transitions separating learnable and nonlearnable regions. We believe that this random version of GLMs can serve as a challenging benchmark for multipurpose algorithms.

Sign in / Sign up

Export Citation Format

Share Document