scholarly journals A parallel algorithm for ridge-penalized estimation of the multivariate exponential family from data of mixed types

2021 ◽  
Vol 31 (4) ◽  
Author(s):  
Diederik S. Laman Trip ◽  
Wessel N. van Wieringen
Keyword(s):  
Markov Random Fields ◽  
Numerical Evaluation ◽  
Exponential Family ◽  
Computationally Efficient ◽  
Penalized Estimation ◽  
Pseudo Likelihood ◽  
Mixed Types ◽  
Special Case ◽  
Multivariate Exponential ◽  
Integrative Omics

AbstractComputationally efficient evaluation of penalized estimators of multivariate exponential family distributions is sought. These distributions encompass among others Markov random fields with variates of mixed type (e.g., binary and continuous) as special case of interest. The model parameter is estimated by maximization of the pseudo-likelihood augmented with a convex penalty. The estimator is shown to be consistent. With a world of multi-core computers in mind, a computationally efficient parallel Newton–Raphson algorithm is presented for numerical evaluation of the estimator alongside conditions for its convergence. Parallelization comprises the division of the parameter vector into subvectors that are estimated simultaneously and subsequently aggregated to form an estimate of the original parameter. This approach may also enable efficient numerical evaluation of other high-dimensional estimators. The performance of the proposed estimator and algorithm are evaluated and compared in a simulation study. Finally, the presented methodology is applied to data of an integrative omics study.

scholarly journals Evaluating the Observed Log-Likelihood Function in Two-Level Structural Equation Modeling with Missing Data: From Formulas to R Code

Psych ◽  
2021 ◽  
Vol 3 (2) ◽  
pp. 197-232
Author(s):  
Yves Rosseel
Keyword(s):  
Structural Equation ◽  
Structural Equation Models ◽  
Likelihood Function ◽  
Likelihood Estimation ◽  
Missing At Random ◽  
Equation Modeling ◽  
Computationally Efficient ◽  
Log Likelihood ◽  
Efficient Expression ◽  
Special Case

This paper discusses maximum likelihood estimation for two-level structural equation models when data are missing at random at both levels. Building on existing literature, a computationally efficient expression is derived to evaluate the observed log-likelihood. Unlike previous work, the expression is valid for the special case where the model implied variance–covariance matrix at the between level is singular. Next, the log-likelihood function is translated to R code. A sequence of R scripts is presented, starting from a naive implementation and ending at the final implementation as found in the lavaan package. Along the way, various computational tips and tricks are given.

scholarly journals Multi-fidelity modelling via recursive co-kriging and Gaussian–Markov random fields

2015 ◽  
Vol 471 (2179) ◽  
pp. 20150018 ◽  
Author(s):  
P. Perdikaris ◽  
D. Venturi ◽  
J. O. Royset ◽  
G. E. Karniadakis
Keyword(s):  
Random Fields ◽  
Markov Random Fields ◽  
Stochastic Modelling ◽  
Response Surfaces ◽  
Circular Cylinders ◽  
Computationally Efficient ◽  
Initial State ◽  
Gaussian Markov Random Fields ◽  
Markov Random ◽  
Multi Level

We propose a new framework for design under uncertainty based on stochastic computer simulations and multi-level recursive co-kriging. The proposed methodology simultaneously takes into account multi-fidelity in models, such as direct numerical simulations versus empirical formulae, as well as multi-fidelity in the probability space (e.g. sparse grids versus tensor product multi-element probabilistic collocation). We are able to construct response surfaces of complex dynamical systems by blending multiple information sources via auto-regressive stochastic modelling. A computationally efficient machine learning framework is developed based on multi-level recursive co-kriging with sparse precision matrices of Gaussian–Markov random fields. The effectiveness of the new algorithms is demonstrated in numerical examples involving a prototype problem in risk-averse design, regression of random functions, as well as uncertainty quantification in fluid mechanics involving the evolution of a Burgers equation from a random initial state, and random laminar wakes behind circular cylinders.

A central limit theorem for conditionally centred random fields with an application to Markov fields

1998 ◽  
Vol 35 (03) ◽  
pp. 608-621
Author(s):  
Francis Comets ◽  
Martin Janžura
Keyword(s):  
Central Limit Theorem ◽  
Limit Theorem ◽  
Central Limit ◽  
Random Fields ◽  
Markov Random Fields ◽  
Moment Condition ◽  
Markov Random ◽  
Pseudo Likelihood ◽  
Strict Positivity ◽  
Markov Fields

We prove a central limit theorem for conditionally centred random fields, under a moment condition and strict positivity of the empirical variance per observation. We use a random normalization, which fits non-stationary situations. The theorem applies directly to Markov random fields, including the cases of phase transition and lack of stationarity. One consequence is the asymptotic normality of the maximum pseudo-likelihood estimator for Markov fields in complete generality.

scholarly journals Wavepath eikonal traveltime inversion: Theory

Geophysics ◽  
1993 ◽  
Vol 58 (9) ◽  
pp. 1314-1323 ◽  
Author(s):  
Gerard T. Schuster ◽  
Aksel Quintus‐Bosz
Keyword(s):  
Ray Tracing ◽  
Weighting Factor ◽  
Back Projection ◽  
Computationally Efficient ◽  
Traveltime Inversion ◽  
Traveltime Tomography ◽  
Inversion Theory ◽  
Order Of Magnitude ◽  
Band Limited ◽  
Special Case

We present a general formula for the back projection of traveltime residuals in traveltime tomography. For special choices of an arbitrary weighting factor this formula reduces to the asymptotic back‐projection term in ray‐tracing tomography (RT), the Woodward‐Rocca method, wavepath eikonal traveltime inversion (WET), and wave‐equation traveltime inversion (WT). This unification provides for an understanding of the differences and similarities among these traveltime tomography methods. The special case of the WET formula leads to a computationally efficient inversion scheme in the space‐time domain that is, in principle, almost as effective as WT inversion yet is an order of magnitude faster. It also leads to an analytic formula for the fast computation of wavepaths. Unlike ray‐tracing tomography, WET partially accounts for band‐limited source and shadow effects in the data. Several numerical tests of the WET method are used to illustrate its properties.

An accelerated majorization-minimization algorithm with convergence guarantee for non-Lipschitz wavelet synthesis model *

2021 ◽  
Vol 38 (1) ◽  
pp. 015001
Author(s):  
Yanan Zhao ◽  
Chunlin Wu ◽  
Qiaoli Dong ◽  
Yufei Zhao
Keyword(s):  
Minimization Problem ◽  
Stationary Point ◽  
Numerical Experiments ◽  
Complete Convergence ◽  
Convergence Result ◽  
Image Process ◽  
Minimization Algorithm ◽  
Computationally Efficient ◽  
Special Case ◽  
Reconstruction Model

Abstract We consider a wavelet based image reconstruction model with the ℓ p (0 < p < 1) quasi-norm regularization, which is a non-convex and non-Lipschitz minimization problem. For solving this model, Figueiredo et al (2007 IEEE Trans. Image Process. 16 2980–2991) utilized the classical majorization-minimization framework and proposed the so-called Isoft algorithm. This algorithm is computationally efficient, but whether it converges or not has not been concluded yet. In this paper, we propose a new algorithm to accelerate the Isoft algorithm, which is based on Nesterov’s extrapolation technique. Furthermore, a complete convergence analysis for the new algorithm is established. We prove that the whole sequence generated by this algorithm converges to a stationary point of the objective function. This convergence result contains the convergence of Isoft algorithm as a special case. Numerical experiments demonstrate good performance of our new algorithm.

scholarly journals COMPUTATIONALLY EFFICIENT RECURSIONS FOR TOP-ORDER INVARIANT POLYNOMIALS WITH APPLICATIONS

2009 ◽  
Vol 25 (1) ◽  
pp. 211-242 ◽  
Author(s):  
Grant Hillier ◽  
Raymond Kan ◽  
Xiaolu Wang
Keyword(s):  
International Economic ◽  
Quadratic Forms ◽  
Numerical Evaluation ◽  
Distribution Theory ◽  
Recursive Algorithms ◽  
Computationally Efficient ◽  
Invariant Polynomials ◽  
Zonal Polynomials ◽  
Theoretical Results ◽  
Order Invariant

The top-order zonal polynomials Ck(A), and top-order invariant polynomials Ck1,…,kr (A1, …, Ar) in which each of the partitions of ki, i = 1, …, r, has only one part, occur frequently in multivariate distribution theory, and econometrics — see, for example, Phillips (1980, Econometrica 48, 861–878; 1984, Journal of Econometrics 26, 387–398; 1985, International Economic Review 26, 21–36; 1986, Econometrica 54, 881–896), Hillier (1985, Econometric Theory 1, 53–72; 2001, Econometric Theory 17, 1–28), Hillier and Satchell (1986, Econometric Theory 2, 66–74), and Smith (1989, Journal of Multivariate Analysis 31, 244–257; 1993, Australian Journal of Statistics 35, 271–282). However, even with the recursive algorithms of Ruben (1962, Annals of Mathematical Statistics 33, 542–570) and Chikuse (1987, Econometric Theory 3, 195–207), numerical evaluation of these invariant polynomials is extremely time consuming. As a result, the value of invariant polynomials has been largely confined to analytic work on distribution theory. In this paper we present new, very much more efficient, algorithms for computing both the top-order zonal and invariant polynomials. These results should make the theoretical results involving these functions much more valuable for direct practical study. We demonstrate the value of our results by providing fast and accurate algorithms for computing the moments of a ratio of quadratic forms in normal random variables.

Bivariate life distributions from Pólya's urn model for contagion

1993 ◽  
Vol 30 (03) ◽  
pp. 497-508 ◽  
Author(s):  
Albert W. Marshall ◽  
Ingram Olkin
Keyword(s):  
Poisson Processes ◽  
Urn Model ◽  
Exponential Distributions ◽  
Bivariate Exponential ◽  
Shock Models ◽  
The Family ◽  
Life Distributions ◽  
Pólya’S Urn ◽  
Special Case ◽  
Multivariate Exponential

Shock models based on Poisson processes have been used to derive univariate and multivariate exponential distributions. But in many applications, Poisson processes are not realistic models of physical shock processes because they have independent increments; expanded models that allow for possibly dependent increments are of interest. In this paper, univariate and bivariate Pólya urn schemes are used to derive models of shock sources. The life distributions obtained from these models form a large parametric family that includes the exponential distribution. Even in the univariate case these life distributions have not been widely used, though they form a large and flexible family. In the bivariate case, the family includes the bivariate exponential distributions of Marshall and Olkin as a special case.

Sign in / Sign up

Export Citation Format

Share Document