scholarly journals ON FILLING-IN MISSING CONDITIONAL PROBABILITIES IN CAUSAL NETWORKS

2005 ◽  
Vol 13 (03) ◽  
pp. 263-280 ◽  
Author(s):  
J. B. PARIS
Keyword(s):  
Maximum Entropy ◽  
Missing Values ◽  
Entropy Solution ◽  
Implicit Assumption ◽  
Conditional Probabilities ◽  
Centre Of Mass ◽  
Inference Process ◽  
Alternative Approach ◽  
Causal Networks ◽  
Maximum Entropy Inference

This paper considers the problem and appropriateness of filling-in missing conditional probabilities in causal networks by the use of maximum entropy. Results generalizing earlier work of Rhodes, Garside & Holmes are proved straightforwardly by the direct application of principles satisfied by the maximum entropy inference process under the assumed uniqueness of the maximum entropy solution. It is however demonstrated that the implicit assumption of uniqueness in the Rhodes, Garside & Holmes papers may fail even in the case of inverted trees. An alternative approach to filling in missing values using the limiting centre of mass inference process is then described which does not suffer this shortcoming, is trivially computationally feasible and arguably enjoys more justification in the context when the probabilities are objective (for example derived from frequencies) than by taking maximum entropy values.

MAXIMISING ENTROPY TO DEDUCE AN INITIAL PROBABILITY DISTRIBUTION FOR A CAUSAL NETWORK

1999 ◽  
Vol 07 (01) ◽  
pp. 63-80 ◽  
Author(s):  
MICHAEL J. MARKHAM ◽  
PAUL C. RHODES
Keyword(s):  
Probability Distribution ◽  
Expert Systems ◽  
Maximum Entropy ◽  
Linear Constraints ◽  
Causal Network ◽  
Conditional Probabilities ◽  
Causal Information ◽  
Non Linear ◽  
Causal Networks ◽  
Initial Probability Distribution

The desire to use Causal Networks as Expert Systems even when the causal information is incomplete and/or when non-causal information is available has led researchers to look into the possibility of utilising Maximum Entropy. If this approach is taken, the known information is supplemented by maximising entropy to provide a unique initial probability distribution which would otherwise have been a consequence of the known information and the independence relationships implied by the network. Traditional maximising techniques can be used if the constraints are linear but the independence relationships give rise to non-linear constraints. This paper extends traditional maximising techniques to incorporate those types of non-linear constraints that arise from the independence relationships and presents an algorithm for implementing the extended method. Maximising entropy does not involve the concept of "causal" information. Consequently, the extended method will accept any mutually consistent set of conditional probabilities and expressions of independence. The paper provides a small example of how this property can be used to provide complete causal information, for use in a causal network, when the known information is incomplete and not in a causal form.

Alternative approach to maximum-entropy inference

1984 ◽  
Vol 30 (5) ◽  
pp. 2638-2644 ◽  
Author(s):  
Y. Tikochinsky ◽  
N. Z. Tishby ◽  
R. D. Levine
Keyword(s):  
Maximum Entropy ◽  
Alternative Approach ◽  
Maximum Entropy Inference

INDEPENDENCE IN COMPLETE AND INCOMPLETE CAUSAL NETWORKS UNDER MAXIMUM ENTROPY

2008 ◽  
Vol 16 (05) ◽  
pp. 699-713 ◽  
Author(s):  
MICHAEL J. MARKHAM
Keyword(s):  
Expert System ◽  
Probability Distribution ◽  
Maximum Entropy ◽  
Linear Constraints ◽  
Causal Network ◽  
Missing Information ◽  
Causal Networks ◽  
Small Set

In an expert system having a consistent set of linear constraints it is known that the Method of Tribus may be used to determine a probability distribution which exhibits maximised entropy. The method is extended here to include independence constraints (Accommodation). The paper proceeds to discusses this extension, and its limitations, then goes on to advance a technique for determining a small set of independencies which can be added to the linear constraints required in a particular representation of an expert system called a causal network, so that the Maximum Entropy and Causal Networks methodologies give matching distributions (Emulation). This technique may also be applied in cases where no initial independencies are given and the linear constraints are incomplete, in order to provide an optimal ME fill-in for the missing information.

scholarly journals An Entropy-Based Machine Learning Algorithm for Combining Macroeconomic Forecasts

Entropy ◽  
2019 ◽  
Vol 21 (10) ◽  
pp. 1015 ◽  
Author(s):  
Carles Bretó ◽  
Priscila Espinosa ◽  
Penélope Hernández ◽  
Jose M. Pavía
Keyword(s):  
Machine Learning ◽  
Gross Domestic Product ◽  
Maximum Entropy ◽  
Simulation Study ◽  
Learning Algorithm ◽  
Predictive Ability ◽  
Learning Approach ◽  
Machine Learning Algorithm ◽  
Machine Learning Approach ◽  
Maximum Entropy Inference

This paper applies a Machine Learning approach with the aim of providing a single aggregated prediction from a set of individual predictions. Departing from the well-known maximum-entropy inference methodology, a new factor capturing the distance between the true and the estimated aggregated predictions presents a new problem. Algorithms such as ridge, lasso or elastic net help in finding a new methodology to tackle this issue. We carry out a simulation study to evaluate the performance of such a procedure and apply it in order to forecast and measure predictive ability using a dataset of predictions on Spanish gross domestic product.

scholarly journals Information Entropy Approach for a Disorderless One-Dimensional Lattice

2020 ◽  
Vol 2 (1) ◽  
pp. 107-113
Author(s):  
Luis Arturo Juárez-Villegas ◽  
Moisés Martínez-Mares
Keyword(s):  
Maximum Entropy ◽  
Information Entropy ◽  
Information Criterion ◽  
System Size ◽  
Reflection Symmetry ◽  
Fixed Size ◽  
Dimensional Lattice ◽  
One Dimensional ◽  
Fixed Energy ◽  
Alternative Approach

Dimensionless conductance through a disorderless lattice is studied using an alternative approach. Usually, the conductance of an ordered lattice is studied at a fixed size, either finite or infinite if the crystalline limit is reached. Here, we propose one to consider the set of systems of all sizes from zero to infinite. As a consequence, we find that the conductance presents fluctuations, with respect to system size, at a fixed energy. At the band edge, these fluctuations are described by a statistical distribution satisfied by an ensemble of chaotic cavities with reflection symmetry, which also satisfies a maximum-entropy, or minimum-information, criterion.

Sign in / Sign up

Export Citation Format

Share Document