Pseudo-Independent Models and Decision Theoretic Knowledge Discovery

2011 ◽  
pp. 1632-1638
Author(s):  
Yang Xiang
Keyword(s):  
Probability Distribution ◽  
Knowledge Discovery ◽  
Graphical Models ◽  
Intelligent Systems ◽  
Probabilistic Reasoning ◽  
Graphical Model ◽  
Full Range ◽  
Simple Problem ◽  
Data Discovery ◽  
Simplifying Assumptions

Graphical models such as Bayesian networks (BNs) (Pearl, 1988; Jensen & Nielsen, 2007) and decomposable Markov networks (DMNs) (Xiang, Wong., & Cercone, 1997) have been widely applied to probabilistic reasoning in intelligent systems. Knowledge representation using such models for a simple problem domain is illustrated in Figure 1: Virus can damage computer files and so can a power glitch. Power glitch also causes a VCR to reset. Links and lack of them convey dependency and independency relations among these variables and the strength of each link is quantified by a probability distribution. The networks are useful for inferring whether the computer has virus after checking files and VCR. This chapter considers how to discover them from data. Discovery of graphical models (Neapolitan, 2004) by testing all alternatives is intractable. Hence, heuristic search are commonly applied (Cooper & Herskovits, 1992; Spirtes, Glymour, & Scheines, 1993; Lam & Bacchus, 1994; Heckerman, Geiger, & Chickering, 1995; Friedman, Geiger, & Goldszmidt, 1997; Xiang, Wong, & Cercone, 1997). All heuristics make simplifying assumptions about the unknown data-generating models. These assumptions preclude certain models to gain efficiency. Often assumptions and models they exclude are not explicitly stated. Users of such heuristics may suffer from such exclusion without even knowing. This chapter examines assumptions underlying common heuristics and their consequences to graphical model discovery. A decision theoretic strategy for choosing heuristics is introduced that can take into account a full range of consequences (including efficiency in discovery, efficiency in inference using the discovered model, and cost of inference with an incorrectly discovered model) and resolve the above issue.

Pseudo Independent Models

2011 ◽  
pp. 935-940
Author(s):  
Yang Xiang
Keyword(s):  
Probability Distribution ◽  
Computer System ◽  
Conditional Probability ◽  
Graphical Models ◽  
Intelligent Systems ◽  
Probabilistic Reasoning ◽  
Joint Probability ◽  
Joint Probability Distribution ◽  
Conditional Probability Distribution ◽  
Binary Variables

Graphical models such as Bayesian networks (BNs) (Pearl, 1988) and decomposable Markov networks (DMNs) (Xiang, Wong & Cercone, 1997) have been applied widely to probabilistic reasoning in intelligent systems. Figure1 illustrates a BN and a DMN on a trivial uncertain domain: A virus can damage computer files, and so can a power glitch. A power glitch also causes a VCR to reset. The BN in (a) has four nodes, corresponding to four binary variables taking values from {true, false}. The graph structure encodes a set of dependence and independence assumptions (e.g., that f is directly dependent on v, and p but is independent of r, once the value of p is known). Each node is associated with a conditional probability distribution conditioned on its parent nodes (e.g., P(f | v, p)). The joint probability distribution is the product P(v, p, f, r) = P(f | v, p) P(r | p) P(v) P(p). The DMN in (b) has two groups of nodes that are maximally pair-wise connected, called cliques. Each clique is associated with a probability distribution (e.g., clique {v, p, f} is assigned P(v, p, f)). The joint probability distribution is P(v, p, f, r) = P(v, p, f) P(r, p) / P(p), where P(p) can be derived from one of the clique distributions. The networks, for instance, can be used to reason about whether there are viruses in the computer system, after observations on f and r are made.

Learning Bayesian Networks

2011 ◽  
pp. 315-321
Author(s):  
Marco F. Ramoni ◽  
Paola Sebastiani
Keyword(s):  
Probability Distribution ◽  
Bayesian Networks ◽  
Bayesian Network ◽  
Graphical Models ◽  
Graphical Model ◽  
Joint Probability ◽  
Joint Probability Distribution ◽  
Probabilistic Graphical Model ◽  
Conditional Probability Distribution ◽  
Statistics And Probability

Born at the intersection of artificial intelligence, statistics, and probability, Bayesian networks (Pearl, 1988) are a representation formalism at the cutting edge of knowledge discovery and data mining (Heckerman, 1997). Bayesian networks belong to a more general class of models called probabilistic graphical models (Whittaker, 1990; Lauritzen, 1996) that arise from the combination of graph theory and probability theory, and their success rests on their ability to handle complex probabilistic models by decomposing them into smaller, amenable components. A probabilistic graphical model is defined by a graph, where nodes represent stochastic variables and arcs represent dependencies among such variables. These arcs are annotated by probability distribution shaping the interaction between the linked variables. A probabilistic graphical model is called a Bayesian network, when the graph connecting its variables is a directed acyclic graph (DAG). This graph represents conditional independence assumptions that are used to factorize the joint probability distribution of the network variables, thus making the process of learning from a large database amenable to computations. A Bayesian network induced from data can be used to investigate distant relationships between variables, as well as making prediction and explanation, by computing the conditional probability distribution of one variable, given the values of some others.

Learning probabilistic networks

1999 ◽  
Vol 13 (4) ◽  
pp. 321-351 ◽  
Author(s):  
PAUL J. KRAUSE
Keyword(s):  
Probability Theory ◽  
Probability Distribution ◽  
Prior Knowledge ◽  
Graphical Models ◽  
Incomplete Data ◽  
Graphical Model ◽  
Probabilistic Graphical Models ◽  
Short Discussion ◽  
Probabilistic Networks ◽  
Probabilistic Network

A probabilistic network is a graphical model that encodes probabilistic relationships between variables of interest. Such a model records qualitative influences between variables in addition to the numerical parameters of the probability distribution. As such it provides an ideal form for combining prior knowledge, which might be limited solely to experience of the influences between some of the variables of interest, and data. In this paper, we first show how data can be used to revise initial estimates of the parameters of a model. We then progress to showing how the structure of the model can be revised as data is obtained. Techniques for learning with incomplete data are also covered. In order to make the paper as self contained as possible, we start with an introduction to probability theory and probabilistic graphical models. The paper concludes with a short discussion on how these techniques can be applied to the problem of learning causal relationships between variables in a domain of interest.

Learning Bayesian Networks

2011 ◽  
pp. 674-677
Author(s):  
Marco F. Ramoni ◽  
Paola Sebastiani
Keyword(s):  
Probability Distribution ◽  
Bayesian Networks ◽  
Bayesian Network ◽  
Graphical Models ◽  
Graphical Model ◽  
Joint Probability ◽  
Joint Probability Distribution ◽  
Probabilistic Graphical Model ◽  
Conditional Probability Distribution ◽  
Statistics And Probability

Born at the intersection of artificial intelligence, statistics, and probability, Bayesian networks (Pearl, 1988) are a representation formalism at the cutting edge of knowledge discovery and data mining (Heckerman, 1997). Bayesian networks belong to a more general class of models called probabilistic graphical models (Whittaker, 1990; Lauritzen, 1996) that arise from the combination of graph theory and probability theory, and their success rests on their ability to handle complex probabilistic models by decomposing them into smaller, amenable components. A probabilistic graphical model is defined by a graph, where nodes represent stochastic variables and arcs represent dependencies among such variables. These arcs are annotated by probability distribution shaping the interaction between the linked variables. A probabilistic graphical model is called a Bayesian network, when the graph connecting its variables is a directed acyclic graph (DAG). This graph represents conditional independence assumptions that are used to factorize the joint probability distribution of the network variables, thus making the process of learning from a large database amenable to computations. A Bayesian network induced from data can be used to investigate distant relationships between variables, as well as making prediction and explanation, by computing the conditional probability distribution of one variable, given the values of some others.

Learning Bayesian Networks

2011 ◽  
pp. 1124-1128
Author(s):  
Marco F. Ramoni ◽  
Paola Sebastiani
Keyword(s):  
Probability Distribution ◽  
Bayesian Networks ◽  
Bayesian Network ◽  
Graphical Models ◽  
Graphical Model ◽  
Joint Probability ◽  
Joint Probability Distribution ◽  
Probabilistic Graphical Model ◽  
Conditional Probability Distribution ◽  
Statistics And Probability

Born at the intersection of artificial intelligence, statistics, and probability, Bayesian networks (Pearl, 1988) are a representation formalism at the cutting edge of knowledge discovery and data mining (Heckerman, 1997). Bayesian networks belong to a more general class of models called probabilistic graphical models (Whittaker, 1990; Lauritzen, 1996) that arise from the combination of graph theory and probability theory, and their success rests on their ability to handle complex probabilistic models by decomposing them into smaller, amenable components. A probabilistic graphical model is defined by a graph, where nodes represent stochastic variables and arcs represent dependencies among such variables. These arcs are annotated by probability distribution shaping the interaction between the linked variables. A probabilistic graphical model is called a Bayesian network, when the graph connecting its variables is a directed acyclic graph (DAG). This graph represents conditional independence assumptions that are used to factorize the joint probability distribution of the network variables, thus making the process of learning from a large database amenable to computations. A Bayesian network induced from data can be used to investigate distant relationships between variables, as well as making prediction and explanation, by computing the conditional probability distribution of one variable, given the values of some others.

scholarly journals Estimating Differential Latent Variable Graphical Models with Applications to Brain Connectivity

Biometrika ◽  
2020 ◽  
Author(s):  
S Na ◽  
M Kolar ◽  
O Koyejo
Keyword(s):  
Graphical Models ◽  
Latent Variable ◽  
Graphical Model ◽  
Brain Connectivity ◽  
Statistical Error ◽  
Real Data ◽  
Low Rank ◽  
Conditional Dependence ◽  
Gradient Descent Algorithm ◽  
The Difference

Abstract Differential graphical models are designed to represent the difference between the conditional dependence structures of two groups, thus are of particular interest for scientific investigation. Motivated by modern applications, this manuscript considers an extended setting where each group is generated by a latent variable Gaussian graphical model. Due to the existence of latent factors, the differential network is decomposed into sparse and low-rank components, both of which are symmetric indefinite matrices. We estimate these two components simultaneously using a two-stage procedure: (i) an initialization stage, which computes a simple, consistent estimator, and (ii) a convergence stage, implemented using a projected alternating gradient descent algorithm applied to a nonconvex objective, initialized using the output of the first stage. We prove that given the initialization, the estimator converges linearly with a nontrivial, minimax optimal statistical error. Experiments on synthetic and real data illustrate that the proposed nonconvex procedure outperforms existing methods.

scholarly journals Disentangling microbial associations from hidden environmental and technical factors via latent graphical models

2019 ◽  
Author(s):  
Zachary D. Kurtz ◽  
Richard Bonneau ◽  
Christian L. Müller
Keyword(s):  
Graphical Models ◽  
Graphical Model ◽  
R Package ◽  
Comprehensive Collection ◽  
Performance Guarantees ◽  
Ecological Association ◽  
Microbial Associations ◽  
Sparse Inverse Covariance Estimation ◽  
Statistical Relationships ◽  
And Function

AbstractDetecting community-wide statistical relationships from targeted amplicon-based and metagenomic profiling of microbes in their natural environment is an important step toward understanding the organization and function of these communities. We present a robust and computationally tractable latent graphical model inference scheme that allows simultaneous identification of parsimonious statistical relationships among microbial species and unobserved factors that influence the prevalence and variability of the abundance measurements. Our method comes with theoretical performance guarantees and is available within the SParse InversE Covariance estimation for Ecological ASsociation Inference (SPIEC-EASI) framework (‘SpiecEasi’ R-package). Using simulations, as well as a comprehensive collection of amplicon-based gut microbiome datasets, we illustrate the method’s ability to jointly identify compositional biases, latent factors that correlate with observed technical covariates, and robust statistical microbial associations that replicate across different gut microbial data sets.

scholarly journals Functional connectomics from neural dynamics: probabilistic graphical models for neuronal network of Caenorhabditis elegans

2018 ◽  
Vol 373 (1758) ◽  
pp. 20170377 ◽  
Author(s):  
Hexuan Liu ◽  
Jimin Kim ◽  
Eli Shlizerman
Keyword(s):  
Time Series ◽  
Nervous System ◽  
Caenorhabditis Elegans ◽  
Graphical Models ◽  
Neuronal Network ◽  
Graphical Model ◽  
Time Series Data ◽  
Series Data ◽  
Neuronal Responses ◽  
C Elegans

We propose an approach to represent neuronal network dynamics as a probabilistic graphical model (PGM). To construct the PGM, we collect time series of neuronal responses produced by the neuronal network and use singular value decomposition to obtain a low-dimensional projection of the time-series data. We then extract dominant patterns from the projections to get pairwise dependency information and create a graphical model for the full network. The outcome model is a functional connectome that captures how stimuli propagate through the network and thus represents causal dependencies between neurons and stimuli. We apply our methodology to a model of the Caenorhabditis elegans somatic nervous system to validate and show an example of our approach. The structure and dynamics of the C. elegans nervous system are well studied and a model that generates neuronal responses is available. The resulting PGM enables us to obtain and verify underlying neuronal pathways for known behavioural scenarios and detect possible pathways for novel scenarios. This article is part of a discussion meeting issue ‘Connectome to behaviour: modelling C. elegans at cellular resolution’.

Basic skills in a complex task: A graphical model relating memory and lexical retrieval to simultaneous interpreting

2003 ◽  
Vol 6 (3) ◽  
pp. 201-211 ◽  
Author(s):  
INGRID K. CHRISTOFFELS ◽  
ANNETTE M. B. DE GROOT ◽  
LOURENS J. WALDORP
Keyword(s):  
Graphical Models ◽  
Picture Naming ◽  
Graphical Model ◽  
Digit Span ◽  
Span Task ◽  
Lexical Retrieval ◽  
Translation Efficiency ◽  
Reading Span ◽  
Simultaneous Interpreting ◽  
Word Translation

Simultaneous interpreting (SI) is a complex skill, where language comprehension and production take place at the same time in two different languages. In this study we identified some of the basic cognitive skills involved in SI, focusing on the roles of memory and lexical retrieval. We administered a reading span task in two languages and a verbal digit span task in the native language to assess memory capacity, and a picture naming and a word translation task to tap the retrieval time of lexical items in two languages, and we related performance on these four tasks to interpreting skill in untrained bilinguals. The results showed that word translation and picture naming latencies correlate with interpreting performance. Also digit span and reading span were associated with SI performance, only less strongly so. A graphical models analysis indicated that specifically word translation efficiency and working memory form independent subskills of SI performance in untrained bilinguals.

Cortical Circuitry Implementing Graphical Models

2009 ◽  
Vol 21 (11) ◽  
pp. 3010-3056 ◽  
Author(s):  
Shai Litvak ◽  
Shimon Ullman
Keyword(s):  
Graphical Models ◽  
Large Scale ◽  
Graphical Model ◽  
Current Model ◽  
Building Blocks ◽  
Population Based ◽  
Spiking Neurons ◽  
Inhibitory Neurons ◽  
Basket Cells ◽  
Local Circuitry

In this letter, we develop and simulate a large-scale network of spiking neurons that approximates the inference computations performed by graphical models. Unlike previous related schemes, which used sum and product operations in either the log or linear domains, the current model uses an inference scheme based on the sum and maximization operations in the log domain. Simulations show that using these operations, a large-scale circuit, which combines populations of spiking neurons as basic building blocks, is capable of finding close approximations to the full mathematical computations performed by graphical models within a few hundred milliseconds. The circuit is general in the sense that it can be wired for any graph structure, it supports multistate variables, and it uses standard leaky integrate-and-fire neuronal units. Following previous work, which proposed relations between graphical models and the large-scale cortical anatomy, we focus on the cortical microcircuitry and propose how anatomical and physiological aspects of the local circuitry may map onto elements of the graphical model implementation. We discuss in particular the roles of three major types of inhibitory neurons (small fast-spiking basket cells, large layer 2/3 basket cells, and double-bouquet neurons), subpopulations of strongly interconnected neurons with their unique connectivity patterns in different cortical layers, and the possible role of minicolumns in the realization of the population-based maximum operation.

Sign in / Sign up

Export Citation Format

Share Document