scholarly journals Parametric inference in the large data limit using maximally informative models

2013 ◽  
Author(s):  
Justin B. Kinney ◽  
Gurinder S. Atwal
Keyword(s):  
Mutual Information ◽  
Parameter Space ◽  
A Priori ◽  
Large Data ◽  
High Dimensional ◽  
A Priori Knowledge ◽  
Dependence Measure ◽  
Noise Function ◽  
Data Processing Inequality ◽  
Noisy Measurement

Motivated by data-rich experiments in transcriptional regulation and sensory neuroscience, we consider the following general problem in statistical inference. When exposed to a high-dimensional signal S, a system of interest computes a representation R of that signal which is then observed through a noisy measurement M. From a large number of signals and measurements, we wish to infer the "filter" that maps S to R. However, the standard method for solving such problems, likelihood-based inference, requires perfect a priori knowledge of the "noise function" mapping R to M. In practice such noise functions are usually known only approximately, if at all, and using an incorrect noise function will typically bias the inferred filter. Here we show that, in the large data limit, this need for a pre-characterized noise function can be circumvented by searching for filters that instead maximize the mutual information I[M;R] between observed measurements and predicted representations. Moreover, if the correct filter lies within the space of filters being explored, maximizing mutual information becomes equivalent to simultaneously maximizing every dependence measure that satisfies the Data Processing Inequality. It is important to note that maximizing mutual information will typically leave a small number of directions in parameter space unconstrained. We term these directions "diffeomorphic modes" and present an equation that allows these modes to be derived systematically. The presence of diffeomorphic modes reflects a fundamental and nontrivial substructure within parameter space, one that is obscured by standard likelihood-based inference.

scholarly journals Parametric Inference in the Large Data Limit Using Maximally Informative Models

2014 ◽  
Vol 26 (4) ◽  
pp. 637-653 ◽  
Author(s):  
Justin B. Kinney ◽  
Gurinder S. Atwal
Keyword(s):  
Mutual Information ◽  
Parameter Space ◽  
A Priori ◽  
Large Data ◽  
High Dimensional ◽  
A Priori Knowledge ◽  
Dependence Measure ◽  
Noise Function ◽  
Data Processing Inequality ◽  
Noisy Measurement

Motivated by data-rich experiments in transcriptional regulation and sensory neuroscience, we consider the following general problem in statistical inference: when exposed to a high-dimensional signal S, a system of interest computes a representation R of that signal, which is then observed through a noisy measurement M. From a large number of signals and measurements, we wish to infer the “filter” that maps S to R. However, the standard method for solving such problems, likelihood-based inference, requires perfect a priori knowledge of the “noise function” mapping R to M. In practice such noise functions are usually known only approximately, if at all, and using an incorrect noise function will typically bias the inferred filter. Here we show that in the large data limit, this need for a precharacterized noise function can be circumvented by searching for filters that instead maximize the mutual information I[M; R] between observed measurements and predicted representations. Moreover, if the correct filter lies within the space of filters being explored, maximizing mutual information becomes equivalent to simultaneously maximizing every dependence measure that satisfies the data processing inequality. It is important to note that maximizing mutual information will typically leave a small number of directions in parameter space unconstrained. We term these directions diffeomorphic modes and present an equation that allows these modes to be derived systematically. The presence of diffeomorphic modes reflects a fundamental and nontrivial substructure within parameter space, one that is obscured by standard likelihood-based inference.

scholarly journals A Partial Information Decomposition Based on Causal Tensors

2020 ◽  
Author(s):  
David Sigtermans
Keyword(s):  
Mutual Information ◽  
Partial Information ◽  
A Priori ◽  
Exact Expression ◽  
Transfer Entropy ◽  
A Posteriori ◽  
Source Data ◽  
Data Processing Inequality ◽  
Indirect Association ◽  
Definition Of

We propose a partial information decomposition based on the newly introduced framework of causal tensors, i.e., multilinear stochastic maps that transform source data into destination data. The innovation that causal tensors introduce is that the framework allows for an exact expression of an indirect association in terms of the constituting, direct associations. This is not possible when expressing associations only in measures like mutual information or transfer entropy. Instead of a priori expressing associations in terms of mutual information or transfer entropy, the a posteriori expression of associations in these terms results in an intuitive definition of a nonnegative and left monotonic redundancy. The proposed redundancy satisfies the three axioms introduced by William and Beer. The symmetry and self-redundancy axioms follow directly from our definition. The data processing inequality ensures that the monotonicity axiom is satisfied. Because causal tensors can be used to describe both mutual information as transfer entropy, the partial information decomposition applies to both measures. Results show that the decomposition closely resembles the decomposition of other another approach that expresses associations in terms of mutual information a posteriori.

scholarly journals A Partial Information Decomposition Based on Causal Tensors

2020 ◽  
Author(s):  
David Sigtermans
Keyword(s):  
Mutual Information ◽  
Partial Information ◽  
A Priori ◽  
Exact Expression ◽  
Transfer Entropy ◽  
A Posteriori ◽  
Source Data ◽  
Data Processing Inequality ◽  
Indirect Association ◽  
Definition Of

We propose a partial information decomposition based on the newly introduced framework of causal tensors, i.e., multilinear stochastic maps that transform source data into destination data. The innovation that causal tensors introduce is that the framework allows for an exact expression of an indirect association in terms of the constituting, direct associations. This is not possible when expressing associations only in measures like mutual information or transfer entropy. Instead of a priori expressing associations in terms of mutual information or transfer entropy, the a posteriori expression of associations in these terms results in an intuitive definition of a nonnegative and left monotonic redundancy, which also meets the identity property. Our proposed redundancy satisfies the three axioms introduced by Williams and Beer. Symmetry and self-redundancy axioms follow directly from our definition. The data processing inequality ensures that the monotonicity axiom is satisfied. Because causal tensors can describe both mutual information as transfer entropy, the partial information decomposition applies to both measures. Results show that the decomposition closely resembles the decomposition of other another approach that expresses associations in terms of mutual information a posteriori. A negative synergistic term could indicate that there is an unobserved common cause.

scholarly journals A Partial Information Decomposition Based on Causal Tensors

2020 ◽  
Author(s):  
David Sigtermans
Keyword(s):  
Mutual Information ◽  
Partial Information ◽  
A Priori ◽  
Exact Expression ◽  
Transfer Entropy ◽  
A Posteriori ◽  
Source Data ◽  
Data Processing Inequality ◽  
Indirect Association ◽  
Definition Of

We propose a partial information decomposition based on the newly introduced framework of causal tensors, i.e., multilinear stochastic maps that transform source data into destination data. The innovation that causal tensors introduce is that the framework allows for an exact expression of an indirect association in terms of the constituting, direct associations. This is not possible when expressing associations only in measures like mutual information or transfer entropy. Instead of a priori expressing associations in terms of mutual information or transfer entropy, the a posteriori expression of associations in these terms results in an intuitive definition of a nonnegative and left monotonic redundancy, which also meets the identity property. Our proposed redundancy satisfies the three axioms introduced by Williams and Beer. Symmetry and self-redundancy axioms follow directly from our definition. The data processing inequality ensures that the monotonicity axiom is satisfied. Because causal tensors can describe both mutual information as transfer entropy, the partial information decomposition applies to both measures. Results show that the decomposition closely resembles the decomposition of other another approach that expresses associations in terms of mutual information a posteriori. A negative synergistic term could indicate that there is an unobserved common cause.

Seeing, Knowing, and Doing

2020 ◽  
Author(s):  
Robert Audi
Keyword(s):  
Practical Reasoning ◽  
Practical Knowledge ◽  
A Priori ◽  
Human Action ◽  
A Priori Knowledge ◽  
Rational Action ◽  
Reasons For Action ◽  
Discriminative Response ◽  
Rich Information ◽  
Priori Knowledge

This book provides an overall theory of perception and an account of knowledge and justification concerning the physical, the abstract, and the normative. It has the rigor appropriate for professionals but explains its main points using concrete examples. It accounts for two important aspects of perception on which philosophers have said too little: its relevance to a priori knowledge—traditionally conceived as independent of perception—and its role in human action. Overall, the book provides a full-scale account of perception, presents a theory of the a priori, and explains how perception guides action. It also clarifies the relation between action and practical reasoning; the notion of rational action; and the relation between propositional and practical knowledge. Part One develops a theory of perception as experiential, representational, and causally connected with its objects: as a discriminative response to those objects, embodying phenomenally distinctive elements; and as yielding rich information that underlies human knowledge. Part Two presents a theory of self-evidence and the a priori. The theory is perceptualist in explicating the apprehension of a priori truths by articulating its parallels to perception. The theory unifies empirical and a priori knowledge by clarifying their reliable connections with their objects—connections many have thought impossible for a priori knowledge as about the abstract. Part Three explores how perception guides action; the relation between knowing how and knowing that; the nature of reasons for action; the role of inference in determining action; and the overall conditions for rational action.

How Reality is Reasonable

2018 ◽  
Author(s):  
Donald C. Williams
Keyword(s):  
A Priori ◽  
A Priori Knowledge ◽  
Dimensional Manifold ◽  
The World ◽  
Being There ◽  
Ways Of Being ◽  
Fundamental Entity ◽  
Spatiotemporal Relations ◽  
Priori Knowledge

This chapter begins with a systematic presentation of the doctrine of actualism. According to actualism, all that exists is actual, determinate, and of one way of being. There are no possible objects, nor is there any indeterminacy in the world. In addition, there are no ways of being. It is proposed that actual entities stand in three fundamental relations: mereological, spatiotemporal, and resemblance relations. These relations govern the fundamental entities. Each fundamental entity stands in parthood relations, spatiotemporal relations, and resemblance relations to other entities. The resulting picture is one that represents the world as a four-dimensional manifold of actual ‘qualitied contents’—upon which all else supervenes. It is then explained how actualism accounts for classes, quantity, number, causation, laws, a priori knowledge, necessity, and induction.

How Do We Know that We’re Not Brains in Vats?

2018 ◽  
Author(s):  
Keith DeRose
Keyword(s):  
Epistemic Justification ◽  
A Priori ◽  
A Priori Knowledge ◽  
Conservative Approach ◽  
Contingent Fact ◽  
Brains In Vats ◽  
Priori Knowledge

In this chapter the contextualist Moorean account of how we know by ordinary standards that we are not brains in vats (BIVs) utilized in Chapter 1 is developed and defended, and the picture of knowledge and justification that emerges is explained. The account (a) is based on a double-safety picture of knowledge; (b) has it that our knowledge that we’re not BIVs is in an important way a priori; and (c) is knowledge that is easily obtained, without any need for fancy philosophical arguments to the effect that we’re not BIVs; and the account is one that (d) utilizes a conservative approach to epistemic justification. Special attention is devoted to defending the claim that we have a priori knowledge of the deeply contingent fact that we’re not BIVs, and to distinguishing this a prioritist account of this knowledge from the kind of “dogmatist” account prominently championed by James Pryor.

scholarly journals Iterative Variable Selection for High-Dimensional Data: Prediction of Pathological Response in Triple-Negative Breast Cancer

Mathematics ◽  
2021 ◽  
Vol 9 (3) ◽  
pp. 222
Author(s):  
Juan C. Laria ◽  
M. Carmen Aguilera-Morillo ◽  
Enrique Álvarez ◽  
Rosa E. Lillo ◽  
Sara López-Taruella ◽  
...  
Keyword(s):  
Breast Cancer ◽  
Variable Selection ◽  
Triple Negative Breast Cancer ◽  
Triple Negative ◽  
A Priori ◽  
Simulated Data ◽  
Point Of View ◽  
High Dimensional ◽  
Whole Genome ◽  
Genome Context

Over the last decade, regularized regression methods have offered alternatives for performing multi-marker analysis and feature selection in a whole genome context. The process of defining a list of genes that will characterize an expression profile remains unclear. It currently relies upon advanced statistics and can use an agnostic point of view or include some a priori knowledge, but overfitting remains a problem. This paper introduces a methodology to deal with the variable selection and model estimation problems in the high-dimensional set-up, which can be particularly useful in the whole genome context. Results are validated using simulated data and a real dataset from a triple-negative breast cancer study.

Sign in / Sign up

Export Citation Format

Share Document