The Fisher Information Matrix for Log Linear Models Arguing Conditionally on Observed Explanatory Variables

Biometrika ◽  
1981 ◽  
Vol 68 (2) ◽  
pp. 563 ◽  
Author(s):  
Juni Palmgren
Keyword(s):  
Fisher Information ◽  
Fisher Information Matrix ◽  
Linear Models ◽  
Information Matrix ◽  
Explanatory Variables ◽  
Log Linear

Synthesis of optimal experiment designs under the conditions of Generalized Lambda-distribution of errors for models with errors in variables

2020 ◽  
pp. 130-141
Author(s):  
Владимир Семенович Тимофеев ◽  
Екатерина Алексеевна Хайленко
Keyword(s):  
Experimental Design ◽  
Fisher Information ◽  
Fisher Information Matrix ◽  
Information Matrix ◽  
Experimental Designs ◽  
Optimal Experimental Design ◽  
Efficiency Function ◽  
Explanatory Variables ◽  
Generalized Lambda Distribution ◽  
Optimal Experiment

Рассмотрена задача планирования эксперимента в условиях появления ошибок в объясняющих переменных. Сформулировано и доказано утверждение о способе вычисления элементов информационной матрицы Фишера с использованием обобщенного лямбда-распределения, доказано следствие о способе вычисления функции эффективности плана эксперимента. Сравнение результатов вычисления функции эффективности с использованием выведенного в следствии соотношения и с помощью известного соотношения для нормального распределения ошибок показало, что результаты совпадают. Построены оптимальные планы эксперимента для различных распределений случайных компонент. The problem of experimental design under conditions of errors in the explanatory variables is considered. The proposition of the method for calculating the Fisher information matrix elements using the Generalized Lambda-distribution is formulated and proved, the consequence of the method for calculating the efficiency function of the experimental design is proved. This method of calculating the Fisher information matrix takes into account the heterogeneity of the errors in random distribution throughout the planning area. In this paper, studies of the synthesis of optimal experimental designs using proven proposition and consequence under various conditions of computational experiments are presented. The results of calculating the efficiency function using the obtained relation and using the known relation for the normal distribution of errors are compared, it is found that the results coincide. Optimal experimental designs are constructed for various distributions of random components. The results of the synthesis of optimal experimental design showed that when function of efficiency is constant throughout the planning area then the optimal experimental design is equilibrium plan. When there are differences in the values of the efficiency function in the planning area, the optimal plan ceases to be equilibrium

scholarly journals Matrix Algebraic Properties of the Fisher Information Matrix of Stationary Processes

Entropy ◽  
2014 ◽  
Vol 16 (4) ◽  
pp. 2023-2055 ◽  
Author(s):  
André Klein
Keyword(s):  
Fisher Information ◽  
Fisher Information Matrix ◽  
Information Matrix ◽  
Stationary Processes ◽  
Algebraic Properties

Exact Fisher Information Matrix With State-Dependent Probability of Detection

2017 ◽  
Vol 53 (3) ◽  
pp. 1555-1561
Author(s):  
Junkun Yan ◽  
Hongwei Liu ◽  
Wenqiang Pu ◽  
Zheng Bao
Keyword(s):  
Fisher Information ◽  
Fisher Information Matrix ◽  
Information Matrix ◽  
Probability Of Detection ◽  
State Dependent ◽  
Dependent Probability

scholarly journals Optimal design for population pk/pd models

2012 ◽  
Vol 51 (1) ◽  
pp. 115-130
Author(s):  
Sergei Leonov ◽  
Alexander Aliev
Keyword(s):  
Optimal Design ◽  
Fisher Information ◽  
Fisher Information Matrix ◽  
Information Matrix ◽  
Optimal Sampling ◽  
Design Algorithm ◽  
Sampling Schemes ◽  
Population Pk ◽  
Different Types

ABSTRACT We provide some details of the implementation of optimal design algorithm in the PkStaMp library which is intended for constructing optimal sampling schemes for pharmacokinetic (PK) and pharmacodynamic (PD) studies. We discuss different types of approximation of individual Fisher information matrix and describe a user-defined option of the library.

scholarly journals Tensor Sylvester matrices and the Fisher information matrix of VARMAX processes

2010 ◽  
Vol 432 (8) ◽  
pp. 1975-1989 ◽  
Author(s):  
André Klein ◽  
Peter Spreij
Keyword(s):  
Fisher Information ◽  
Fisher Information Matrix ◽  
Information Matrix

Singularities Affect Dynamics of Learning in Neuromanifolds

2006 ◽  
Vol 18 (5) ◽  
pp. 1007-1065 ◽  
Author(s):  
Shun-ichi Amari ◽  
Hyeyoung Park ◽  
Tomoko Ozeki
Keyword(s):  
Model Selection ◽  
Parameter Space ◽  
Fisher Information ◽  
Fisher Information Matrix ◽  
Information Matrix ◽  
Gaussian Mixture ◽  
Multilayer Perceptrons ◽  
Hierarchical Systems ◽  
Space Forms ◽  
Training Error

The parameter spaces of hierarchical systems such as multilayer perceptrons include singularities due to the symmetry and degeneration of hidden units. A parameter space forms a geometrical manifold, called the neuromanifold in the case of neural networks. Such a model is identified with a statistical model, and a Riemannian metric is given by the Fisher information matrix. However, the matrix degenerates at singularities. Such a singular structure is ubiquitous not only in multilayer perceptrons but also in the gaussian mixture probability densities, ARMA time-series model, and many other cases. The standard statistical paradigm of the Cramér-Rao theorem does not hold, and the singularity gives rise to strange behaviors in parameter estimation, hypothesis testing, Bayesian inference, model selection, and in particular, the dynamics of learning from examples. Prevailing theories so far have not paid much attention to the problem caused by singularity, relying only on ordinary statistical theories developed for regular (nonsingular) models. Only recently have researchers remarked on the effects of singularity, and theories are now being developed. This article gives an overview of the phenomena caused by the singularities of statistical manifolds related to multilayer perceptrons and gaussian mixtures. We demonstrate our recent results on these problems. Simple toy models are also used to show explicit solutions. We explain that the maximum likelihood estimator is no longer subject to the gaussian distribution even asymptotically, because the Fisher information matrix degenerates, that the model selection criteria such as AIC, BIC, and MDL fail to hold in these models, that a smooth Bayesian prior becomes singular in such models, and that the trajectories of dynamics of learning are strongly affected by the singularity, causing plateaus or slow manifolds in the parameter space. The natural gradient method is shown to perform well because it takes the singular geometrical structure into account. The generalization error and the training error are studied in some examples.

Sign in / Sign up

Export Citation Format

Share Document