Using Maximum Entropy to Estimate Missing Information in Tree-like Causal Networks

Author(s):  
Gerald R Garside ◽  
Dawn E Holmes ◽  
Paul C Rhodes
Author(s):  
MICHAEL J. MARKHAM

In an expert system having a consistent set of linear constraints it is known that the Method of Tribus may be used to determine a probability distribution which exhibits maximised entropy. The method is extended here to include independence constraints (Accommodation). The paper proceeds to discusses this extension, and its limitations, then goes on to advance a technique for determining a small set of independencies which can be added to the linear constraints required in a particular representation of an expert system called a causal network, so that the Maximum Entropy and Causal Networks methodologies give matching distributions (Emulation). This technique may also be applied in cases where no initial independencies are given and the linear constraints are incomplete, in order to provide an optimal ME fill-in for the missing information.


1998 ◽  
Vol 54 (3) ◽  
pp. 221-230 ◽  
Author(s):  
G. H. Rao ◽  
I. D. Brown

The distribution of valence among the bonds in the bond graph of an inorganic compound is used to calculate an `entropy'. We show that the distribution of valence that maximizes this entropy (ME) is similar, but not identical, to that obtained using the equal-valence rule (EVR) proposed by Brown [Acta Cryst. (1977), B33, 1305–1310]. Since the ME solutions are maximally non-committal with regard to missing information, they give better predictions of the observed valence distributions than the EVR solutions when lattice constraints or electronic anisotropies are present, but worse predictions when these effects are absent. Since valences calculated using ME are necessarily positive, they give significantly better predictions in cases where EVR predicts a negative bond valence. In the absence of electronic distortions the observed bond graph is either the graph with the highest maximum entropy or it has an entropy within 1% of this value. Since the entropy depends on the oxidation states of the atoms, compounds with the same stoichiometry and cation coordination numbers but different atomic valences may adopt different bond graphs and hence different structures.


Author(s):  
MICHAEL J. MARKHAM ◽  
PAUL C. RHODES

The desire to use Causal Networks as Expert Systems even when the causal information is incomplete and/or when non-causal information is available has led researchers to look into the possibility of utilising Maximum Entropy. If this approach is taken, the known information is supplemented by maximising entropy to provide a unique initial probability distribution which would otherwise have been a consequence of the known information and the independence relationships implied by the network. Traditional maximising techniques can be used if the constraints are linear but the independence relationships give rise to non-linear constraints. This paper extends traditional maximising techniques to incorporate those types of non-linear constraints that arise from the independence relationships and presents an algorithm for implementing the extended method. Maximising entropy does not involve the concept of "causal" information. Consequently, the extended method will accept any mutually consistent set of conditional probabilities and expressions of independence. The paper provides a small example of how this property can be used to provide complete causal information, for use in a causal network, when the known information is incomplete and not in a causal form.


Author(s):  
J. B. PARIS

This paper considers the problem and appropriateness of filling-in missing conditional probabilities in causal networks by the use of maximum entropy. Results generalizing earlier work of Rhodes, Garside & Holmes are proved straightforwardly by the direct application of principles satisfied by the maximum entropy inference process under the assumed uniqueness of the maximum entropy solution. It is however demonstrated that the implicit assumption of uniqueness in the Rhodes, Garside & Holmes papers may fail even in the case of inverted trees. An alternative approach to filling in missing values using the limiting centre of mass inference process is then described which does not suffer this shortcoming, is trivially computationally feasible and arguably enjoys more justification in the context when the probabilities are objective (for example derived from frequencies) than by taking maximum entropy values.


Author(s):  
Amos Golan

In this chapter I develop the complete info-metrics framework for inferring problems and theories under all types of uncertainty and missing information. That framework allows for uncertainty in the observed values and about the functional form, as captured by the constraints. Using the derivations of Chapter 8, it also extends the info-metrics framework to include priors. The basic properties of the complete framework are developed as well. Generally speaking, that framework can be viewed as a “meta-theory”—a theory of how to construct theories and consistent models given the available information. This accrues all the benefits of the maximum entropy formalism but additionally accommodates a larger class of problems. The derivations are complemented with a complete visual representation of the info-metrics framework. Theoretical and empirical applications are provided.


1984 ◽  
Vol 75 ◽  
pp. 461-469 ◽  
Author(s):  
Robert W. Hart

ABSTRACTThis paper models maximum entropy configurations of idealized gravitational ring systems. Such configurations are of interest because systems generally evolve toward an ultimate state of maximum randomness. For simplicity, attention is confined to ultimate states for which interparticle interactions are no longer of first order importance. The planets, in their orbits about the sun, are one example of such a ring system. The extent to which the present approximation yields insight into ring systems such as Saturn's is explored briefly.


Author(s):  
Neng-Yu Zhang ◽  
Bruce F. McEwen ◽  
Joachim Frank

Reconstructions of asymmetric objects computed by electron tomography are distorted due to the absence of information, usually in an angular range from 60 to 90°, which produces a “missing wedge” in Fourier space. These distortions often interfere with the interpretation of results and thus limit biological ultrastructural information which can be obtained. We have attempted to use the Method of Projections Onto Convex Sets (POCS) for restoring the missing information. In POCS, use is made of the fact that known constraints such as positivity, spatial boundedness or an upper energy bound define convex sets in function space. Enforcement of such constraints takes place by iterating a sequence of function-space projections, starting from the original reconstruction, onto the convex sets, until a function in the intersection of all sets is found. First applications of this technique in the field of electron microscopy have been promising.To test POCS on experimental data, we have artificially reduced the range of an existing projection set of a selectively stained Golgi apparatus from ±60° to ±50°, and computed the reconstruction from the reduced set (51 projections). The specimen was prepared from a bull frog spinal ganglion as described by Lindsey and Ellisman and imaged in the high-voltage electron microscope.


2012 ◽  
Author(s):  
Reid Hastie ◽  
Benjamin M. Rottman
Keyword(s):  

Sign in / Sign up

Export Citation Format

Share Document