On the Approximation Rate of Hierarchical Mixtures-of-Experts for Generalized Linear Models

1999 ◽  
Vol 11 (5) ◽  
pp. 1183-1198 ◽  
Author(s):  
Wenxin Jiang ◽  
Martin A. Tanner
Keyword(s):  
Generalized Linear Models ◽  
Upper Bound ◽  
Linear Models ◽  
Sobolev Class ◽  
Link Function ◽  
Approximation Rate ◽  
Smooth Functions ◽  
Mixtures Of Experts ◽  
The Family

We investigate a class of hierarchical mixtures-of-experts (HME) models where generalized linear models with nonlinear mean functions of the form ψ(α + xTβ) are mixed. Here ψ(·) is the inverse link function. It is shown that mixtures of such mean functions can approximate a class of smooth functions of the form ψ(h(x)), where h(·) ε W∞2;k (a Sobolev class over [0, 1]s, as the number of experts m in the network increases. An upper bound of the approximation rate is given as O(m−2/s) in Lp norm. This rate can be achieved within the family of HME structures with no more than s-layers, where s is the dimension of the predictor x.

Posterior contraction in sparse generalized linear models

Biometrika ◽  
2020 ◽  
Author(s):  
Seonghyun Jeong ◽  
Subhashis Ghosal
Keyword(s):  
Generalized Linear Models ◽  
Bayesian Methods ◽  
Generalized Linear Model ◽  
Linear Models ◽  
Continuous Distribution ◽  
Fractional Power ◽  
Regression Coefficients ◽  
Link Function ◽  
Convergence Properties ◽  
Set Up

Summary We study posterior contraction rates in sparse high-dimensional generalized linear models using priors incorporating sparsity. A mixture of a point mass at zero and a continuous distribution is used as the prior distribution on regression coefficients. In addition to the usual posterior, the fractional posterior, which is obtained by applying Bayes theorem with a fractional power of the likelihood, is also considered. The latter allows uniformity in posterior contraction over a larger subset of the parameter space. In our set-up, the link function of the generalized linear model need not be canonical. We show that Bayesian methods achieve convergence properties analogous to lasso-type procedures. Our results can be used to derive posterior contraction rates in many generalized linear models including logistic, Poisson regression and others.

scholarly journals An Exploration of Link Functions Used in Ordinal Regression

2020 ◽  
Vol 18 (1) ◽  
pp. 2-15
Author(s):  
Thomas J. Smith ◽  
David A. Walker ◽  
Cornelius M. McKenna
Keyword(s):  
Survey Data ◽  
Generalized Linear Models ◽  
Linear Models ◽  
Simulated Data ◽  
Ordinal Regression ◽  
Research Center ◽  
Link Function ◽  
25Th Anniversary ◽  
Link Functions ◽  
The Web

The purpose of this study is to examine issues involved with choice of a link function in generalized linear models with ordinal outcomes, including distributional appropriateness, link specificity, and palindromic invariance are discussed and an exemplar analysis provided using the Pew Research Center 25th anniversary of the Web Omnibus Survey data. Simulated data are used to compare the relative palindromic invariance of four distinct indices of determination/discrimination, including a newly proposed index by Smith et al. (2017).

Generalized Linear Models

2021 ◽  
pp. 195-208
Author(s):  
Andy Hector
Keyword(s):  
Regression Analysis ◽  
Maximum Likelihood ◽  
Generalized Linear Models ◽  
Linear Models ◽  
Square Root ◽  
Link Function ◽  
Data Transformations ◽  
Normal Distributions ◽  
Equal Variance ◽  
The Mean

This chapter revisits a regression analysis to explore the normal least squares assumption of approximately equal variance. It also considers some of the data transformations that can be used to achieve this. A linear regression of transformed data is compared with a generalized linear-model equivalent that avoids transformation by using a link function and non-normal distributions. Generalized linear models based on maximum likelihood use a link function to model the mean (in this case a square-root link) and a variance function to model the variability (in this case the gamma distribution, where the variance increases as the square of the mean). The Box–Cox family of transformations is explained in detail.

scholarly journals A Framework to Interpret Nonstandard Log-Linear Models

2016 ◽  
Vol 36 (2) ◽  
Author(s):  
Patrick Mair
Keyword(s):  
Generalized Linear Models ◽  
Categorical Data ◽  
Linear Models ◽  
Contingency Tables ◽  
Design Matrix ◽  
Nested Models ◽  
Testing Strategies ◽  
The Family ◽  
Parsimonious Models ◽  
Log Linear

The formulation of log-linear models within the framework of Generalized Linear Models offers new possibilities in modeling categorical data. The resulting models are not restricted to the analysis of contingency tables in terms of ordinary hierarchical interactions. Such models are considered as the family of nonstandard log-linear models. The problem that can arise is an ambiguous interpretation of parameters. In the current paperthis problem is solved by looking at the effects coded in the design matrix and determining the numerical contribution of single effects. Based on these results, stepwise approaches are proposed in order to achieve parsimonious models. In addition, some testing strategies are presented to test such (eventually non-nested) models against each other. As a result, a whole interpretation framework is elaborated to examine nonstandard log-linear models in depth.

Sign in / Sign up

Export Citation Format

Share Document