scholarly journals Decomposition of Gaussian processes, and factorization of positive definite kernels

2019 ◽  
Vol 39 (4) ◽  
pp. 497-541
Author(s):  
Palle Jorgensen ◽  
Feng Tian
Keyword(s):  
Gaussian Process ◽  
Gaussian Processes ◽  
Point Processes ◽  
Positive Definite ◽  
The Other ◽  
Matrix Factorizations ◽  
Infinite Dimensions ◽  
Positive Definite Kernels ◽  
Sampling Schemes ◽  
Graph Laplacians

We establish a duality for two factorization questions, one for general positive definite (p.d.) kernels \(K\), and the other for Gaussian processes, say \(V\). The latter notion, for Gaussian processes is stated via Ito-integration. Our approach to factorization for p.d. kernels is intuitively motivated by matrix factorizations, but in infinite dimensions, subtle measure theoretic issues must be addressed. Consider a given p.d. kernel \(K\), presented as a covariance kernel for a Gaussian process \(V\). We then give an explicit duality for these two seemingly different notions of factorization, for p.d. kernel \(K\), vs for Gaussian process \(V\). Our result is in the form of an explicit correspondence. It states that the analytic data which determine the variety of factorizations for \(K\) is the exact same as that which yield factorizations for \(V\). Examples and applications are included: point-processes, sampling schemes, constructive discretization, graph-Laplacians, and boundary-value problems.

scholarly journals Generalized Gaussian processes and relations with random matrices and positive definite functions on permutation groups

2015 ◽  
Vol 18 (03) ◽  
pp. 1550020
Author(s):  
Marek Bożejko ◽  
Wojciech Bożejko
Keyword(s):  
Probability Measure ◽  
Gaussian Process ◽  
Gaussian Processes ◽  
Positive Definite Function ◽  
Explicit Construction ◽  
Positive Definite ◽  
Positive Definite Functions ◽  
Permutation Groups ◽  
Definite Function ◽  
Combinatorial Formulas

The main purpose of this paper is an explicit construction of generalized Gaussian process with function tb(V) = bH(V), where H(V) = n - h(V), h(V) is the number of singletons in a pair-partition V ∈ 𝒫2(2n). This gives another proof of Theorem of A. Buchholtz15 that tb is positive definite function on the set of all pair-partitions. Here there are some new combinatorial formulas presented. Connections with free additive convolutions probability measure on ℝ are also done. There are new positive definite functions on permutations presented. What is more, it is proven that the function H is norm (on the group S(∞) = ⋃S(n)).

scholarly journals Strictly Positive-Definite Spike Train Kernels for Point-Process Divergences

2012 ◽  
Vol 24 (8) ◽  
pp. 2223-2250 ◽  
Author(s):  
Il Memming Park ◽  
Sohan Seth ◽  
Murali Rao ◽  
José C. Príncipe
Keyword(s):  
Hypothesis Testing ◽  
Spike Train ◽  
Point Processes ◽  
Principal Component ◽  
Spike Trains ◽  
Positive Definite ◽  
Kernel Principal Component Analysis ◽  
Structural Representation ◽  
Positive Definite Kernels ◽  
Statistical Measures

Exploratory tools that are sensitive to arbitrary statistical variations in spike train observations open up the possibility of novel neuroscientific discoveries. Developing such tools, however, is difficult due to the lack of Euclidean structure of the spike train space, and an experimenter usually prefers simpler tools that capture only limited statistical features of the spike train, such as mean spike count or mean firing rate. We explore strictly positive-definite kernels on the space of spike trains to offer both a structural representation of this space and a platform for developing statistical measures that explore features beyond count or rate. We apply these kernels to construct measures of divergence between two point processes and use them for hypothesis testing, that is, to observe if two sets of spike trains originate from the same underlying probability law. Although there exist positive-definite spike train kernels in the literature, we establish that these kernels are not strictly definite and thus do not induce measures of divergence. We discuss the properties of both of these existing nonstrict kernels and the novel strict kernels in terms of their computational complexity, choice of free parameters, and performance on both synthetic and real data through kernel principal component analysis and hypothesis testing.

scholarly journals Harmonic analysis of network systems via kernels and their boundary realizations

2021 ◽  
Vol 0 (0) ◽  
pp. 0
Author(s):  
Palle Jorgensen ◽  
James Tian
Keyword(s):  
Gaussian Processes ◽  
Reproducing Kernel ◽  
Geometric Analysis ◽  
Iterated Function Systems ◽  
Positive Definite ◽  
Positive Definite Kernels ◽  
Network Graph ◽  
Network Systems ◽  
Positive Definite Kernel ◽  
Measure Spaces

<p style='text-indent:20px;'>With view to applications to harmonic and stochastic analysis of infinite network/graph models, we introduce new tools for realizations and transforms of positive definite kernels (p.d.) <inline-formula><tex-math id="M1">\begin{document}$ K $\end{document}</tex-math></inline-formula> and their associated reproducing kernel Hilbert spaces. With this we establish two kinds of factorizations: (i) Probabilistic: Starting with a positive definite kernel <inline-formula><tex-math id="M2">\begin{document}$ K $\end{document}</tex-math></inline-formula> we analyze associated Gaussian processes <inline-formula><tex-math id="M3">\begin{document}$ V $\end{document}</tex-math></inline-formula>. Properties of the Gaussian processes will be derived from certain factorizations of <inline-formula><tex-math id="M4">\begin{document}$ K $\end{document}</tex-math></inline-formula>, arising as a covariance kernel of <inline-formula><tex-math id="M5">\begin{document}$ V $\end{document}</tex-math></inline-formula>. (ii) Geometric analysis: We discuss families of measure spaces arising as boundaries for <inline-formula><tex-math id="M6">\begin{document}$ K $\end{document}</tex-math></inline-formula>. Our results entail an analysis of a partial order on families of p.d. kernels, a duality for operators and frames, optimization, Karhunen–Loève expansions, and factorizations. Applications include a new boundary analysis for the Drury-Arveson kernel, and for certain fractals arising as iterated function systems; and an identification of optimal feature spaces in machine learning models.</p>

scholarly journals Active Learning for Enumerating Local Minima Based on Gaussian Process Derivatives

2020 ◽  
Vol 32 (10) ◽  
pp. 2032-2068 ◽  
Author(s):  
Yu Inatsu ◽  
Daisuke Sugita ◽  
Kazuaki Toyoura ◽  
Ichiro Takeuchi
Keyword(s):  
Active Learning ◽  
Gaussian Process ◽  
Confidence Intervals ◽  
Gaussian Processes ◽  
Numerical Experiments ◽  
Local Minimum ◽  
Black Box ◽  
Positive Definite ◽  
Local Minima ◽  
Local Solutions

We study active learning (AL) based on gaussian processes (GPs) for efficiently enumerating all of the local minimum solutions of a black-box function. This problem is challenging because local solutions are characterized by their zero gradient and positive-definite Hessian properties, but those derivatives cannot be directly observed. We propose a new AL method in which the input points are sequentially selected such that the confidence intervals of the GP derivatives are effectively updated for enumerating local minimum solutions. We theoretically analyze the proposed method and demonstrate its usefulness through numerical experiments.

Minimal distributions in a stochastic partial ordering and bounds for Gaussian processes

1983 ◽  
Vol 20 (03) ◽  
pp. 529-536
Author(s):  
W. J. R. Eplett
Keyword(s):  
Gaussian Process ◽  
Gaussian Processes ◽  
Partial Ordering ◽  
Boundary Crossing ◽  
Conditional Probabilities ◽  
Life Distribution ◽  
Life Distributions ◽  
Bounded Below ◽  
Crossing Probabilities

A natural requirement to impose upon the life distribution of a component is that after inspection at some randomly chosen time to check whether it is still functioning, its life distribution from the time of checking should be bounded below by some specified distribution which may be defined by external considerations. Furthermore, the life distribution should ideally be minimal in the partial ordering obtained from the conditional probabilities. We prove that these specifications provide an apparently new characterization of the DFRA class of life distributions with a corresponding result for IFRA distributions. These results may be transferred, using Slepian's lemma, to obtain bounds for the boundary crossing probabilities of a stationary Gaussian process.

scholarly journals Bias correction for direct spectral estimation from irregularly sampled data including sampling schemes with correlation

2021 ◽  
Vol 2021 (1) ◽  
Author(s):  
Nils Damaschke ◽  
Volker Kühn ◽  
Holger Nobach
Keyword(s):  
Spectral Estimation ◽  
Continuous Distribution ◽  
Discrete Distribution ◽  
Irregular Sampling ◽  
The Other ◽  
Sampled Data ◽  
Sampling Schemes ◽  
Other Hand ◽  
Sampling Intervals ◽  
Irregularly Sampled Data

AbstractThe prediction and correction of systematic errors in direct spectral estimation from irregularly sampled data taken from a stochastic process is investigated. Different sampling schemes are investigated, which lead to such an irregular sampling of the observed process. Both kinds of sampling schemes are considered, stochastic sampling with non-equidistant sampling intervals from a continuous distribution and, on the other hand, nominally equidistant sampling with missing individual samples yielding a discrete distribution of sampling intervals. For both distributions of sampling intervals, continuous and discrete, different sampling rules are investigated. On the one hand, purely random and independent sampling times are considered. This is given only in those cases, where the occurrence of one sample at a certain time has no influence on other samples in the sequence. This excludes any preferred delay intervals or external selection processes, which introduce correlations between the sampling instances. On the other hand, sampling schemes with interdependency and thus correlation between the individual sampling instances are investigated. This is given whenever the occurrence of one sample in any way influences further sampling instances, e.g., any recovery times after one instance, any preferences of sampling intervals including, e.g., sampling jitter or any external source with correlation influencing the validity of samples. A bias-free estimation of the spectral content of the observed random process from such irregularly sampled data is the goal of this investigation.

scholarly journals High-performance sampling of generic determinantal point processes

2020 ◽  
Vol 378 (2166) ◽  
pp. 20190059 ◽  
Author(s):  
Jack Poulson
Keyword(s):  
Point Process ◽  
Spectral Decomposition ◽  
High Performance ◽  
Point Processes ◽  
Cholesky Factorization ◽  
Determinantal Point Processes ◽  
Determinantal Point Process ◽  
Decomposition Approach ◽  
Map Inference ◽  
Sampling Schemes

Determinantal point processes (DPPs) were introduced by Macchi (Macchi 1975 Adv. Appl. Probab. 7 , 83–122) as a model for repulsive (fermionic) particle distributions. But their recent popularization is largely due to their usefulness for encouraging diversity in the final stage of a recommender system (Kulesza & Taskar 2012 Found. Trends Mach. Learn. 5 , 123–286). The standard sampling scheme for finite DPPs is a spectral decomposition followed by an equivalent of a randomly diagonally pivoted Cholesky factorization of an orthogonal projection, which is only applicable to Hermitian kernels and has an expensive set-up cost. Researchers Launay et al. 2018 ( http://arxiv.org/abs/1802.08429 ); Chen & Zhang 2018 NeurIPS ( https://papers.nips.cc/paper/7805-fast-greedy-map-inference-for-determinantal-point-process-to-improve-recommendation-diversity.pdf ) have begun to connect DPP sampling to LDL H factorizations as a means of avoiding the initial spectral decomposition, but existing approaches have only outperformed the spectral decomposition approach in special circumstances, where the number of kept modes is a small percentage of the ground set size. This article proves that trivial modifications of LU and LDL H factorizations yield efficient direct sampling schemes for non-Hermitian and Hermitian DPP kernels, respectively. Furthermore, it is experimentally shown that even dynamically scheduled, shared-memory parallelizations of high-performance dense and sparse-direct factorizations can be trivially modified to yield DPP sampling schemes with essentially identical performance. The software developed as part of this research, Catamari ( hodgestar.com/catamari ) is released under the Mozilla Public License v.2.0. It contains header-only, C++14 plus OpenMP 4.0 implementations of dense and sparse-direct, Hermitian and non-Hermitian DPP samplers. This article is part of a discussion meeting issue ‘Numerical algorithms for high-performance computational science’.

Sign in / Sign up

Export Citation Format

Share Document