Outlier Detection

This is an advance summary of a forthcoming article in the Oxford Research Encyclopedia of Economics and Finance. Please check back later for the full article. Detection of outliers is an important explorative step in empirical analysis. Once detected, the investigator will have to decide how to model the outliers depending on the context. Indeed, the outliers may represent noisy observations that are best left out of the analysis or they may be very informative observations that would have a particularly important role in the analysis. For regression analysis in time series a number of outlier algorithms are available, including impulse indicator saturation and methods from robust statistics. The algorithms are complex and their statistical properties are not fully understood. Extensive simulation studies have been made, but the formal theory is lacking. Some progress has been made toward an asymptotic theory of the algorithms. A number of asymptotic results are already available building on empirical process theory.

Complexity of possibly gapped histogram and analysis of histogram

We demonstrate that gaps and distributional patterns embedded within real-valued measurements are inseparable biological and mechanistic information contents of the system. Such patterns are discovered through data-driven possibly gapped histogram, which further leads to the geometry-based analysis of histogram (ANOHT). Constructing a possibly gapped histogram is a complex problem of statistical mechanics due to the ensemble of candidate histograms being captured by a two-layer Ising model. This construction is also a distinctive problem of Information Theory from the perspective of data compression via uniformity. By defining a Hamiltonian (or energy) as a sum of total coding lengths of boundaries and total decoding errors within bins, this issue of computing the minimum energy macroscopic states is surprisingly resolved by applying the hierarchical clustering algorithm. Thus, a possibly gapped histogram corresponds to a macro-state. And then the first phase of ANOHT is developed for simultaneous comparison of multiple treatments, while the second phase of ANOHT is developed based on classical empirical process theory for a tree-geometry that can check the authenticity of branches of the treatment tree. The well-known Iris data are used to illustrate our technical developments. Also, a large baseball pitching dataset and a heavily right-censored divorce data are analysed to showcase the existential gaps and utilities of ANOHT.

UNIFORM CONVERGENCE RATES OF KERNEL-BASED NONPARAMETRIC ESTIMATORS FOR CONTINUOUS TIME DIFFUSION PROCESSES: A DAMPING FUNCTION APPROACH

In this paper, we derive uniform convergence rates of nonparametric estimators for continuous time diffusion processes. In particular, we consider kernel-based estimators of the Nadaraya–Watson type, introducing a new technical device called adamping function. This device allows us to derive sharp uniform rates over an infinite interval with minimal requirements on the processes: The existence of the moment of any order is not required and the boundedness of relevant functions can be significantly relaxed. Restrictions on kernel functions are also minimal: We allow for kernels with discontinuity, unbounded support, and slowly decaying tails. Our proofs proceed by using the covering-number technique from empirical process theory and exploiting the mixing and martingale properties of the processes. We also present new results on the path-continuity property of Brownian motions and diffusion processes over an infinite time horizon. These path-continuity results, which should also be of some independent interest, are used to control discretization biases of the nonparametric estimators. The obtained convergence results are useful for non/semiparametric estimation and testing problems of diffusion processes.

Learning with Convex Loss and Indefinite Kernels

We consider a kind of kernel-based regression with general convex loss functions in a regularization scheme. The kernels used in the scheme are not necessarily symmetric and thus are not positive semidefinite; l1−norm of the coefficients in the kernel ensembles is taken as the regularizer. Our setting in this letter is quite different from the classical regularized regression algorithms such as regularized networks and support vector machines regression. Under an established error decomposition that consists of approximation error, hypothesis error, and sample error, we present a detailed mathematical analysis for this scheme and, in particular, its learning rate. A reweighted empirical process theory is applied to the analysis of produced learning algorithms, which plays a key role in deriving the explicit learning rate under some assumptions.

General Bootstrap for Dual ϕ-Divergence Estimates

A general notion of bootstrappedϕ-divergence estimates constructed by exchangeably weighting sample is introduced. Asymptotic properties of these generalized bootstrappedϕ-divergence estimates are obtained, by means of the empirical process theory, which are applied to construct the bootstrap confidence set with asymptotically correct coverage probability. Some of practical problems are discussed, including, in particular, the choice of escort parameter, and several examples of divergences are investigated. Simulation results are provided to illustrate the finite sample performance of the proposed estimators.

Comment on ‘A multiple process solution …’ (B. Macwhinney)

The central question posed by the so called ‘logical problem of language acquisition’ is how it comes to be that children are able to GENERALIZE from a finite set of linguistic data to acquire (learn, develop, grow) a computational system (grammar) that applies to novel examples not encountered before. The difficulty of this generalization problem was first posed cogently by Gold and while Macwhinney discusses the Gold framework and the linguistic literature on this matter, it is worth noting that the Gold framework is not the only one. There are at least two important new sources of insight from computational learning theory in the decades following Gold that need to be kept in mind. First, there is the development of empirical process theory that forms the basis of any analysis of statistical learning (see summary in Vapnik, 1998). Applying this approach to language (see Niyogi, 1998 for a treatment), one concludes that the family of learnable grammars must have a finite Vapnik Chervonenkis (VC) dimension. The VC dimension is a combinatorial measure of the complexity of a class of functions. Grammars may be viewed as functions mapping sentences to their grammaticality value. In this more sophisticated sense of the VC dimension, the class of grammars must be constrained. Second, there is the development of the theory of computational complexity suggesting that while a learning algorithm might exist, it may not be efficient, i.e. run in polynomial time. These two developments come together in the influential Probably Approximately Correct (PAC) model (Valiant, 1984).