scholarly journals Informeasure: an R/Bioconductor package to quantify nonlinear dependence between variables in biological networks from an information theory perspective

2021 ◽  
Author(s):  
CHU PAN
Keyword(s):  
Information Theory ◽  
Mutual Information ◽  
Biological Networks ◽  
Regulatory Networks ◽  
Partial Information ◽  
R Package ◽  
Nonlinear Dependence ◽  
Bioconductor Package ◽  
Information Measures ◽  
Biological Regulatory Networks

Using information measures to infer biological regulatory networks can observe nonlinear relationship between variables, but it is computationally challenging and there is currently no convenient tool available. We here describe an information theory R package named Informeasure that devotes to quantifying nonlinear dependence between variables in biological regulatory networks from an information theory perspective. This package compiles most of the information measures currently available: mutual information, conditional mutual information, interaction information, partial information decomposition and part mutual information. The first estimator is used to infer bivariate networks while the last four estimators are dedicated to analysis of trivariate networks. The base installation of this turn-key package allows users to approach these information measures out of the box. Informeasure is implemented in R program and is available as an R/Bioconductor package at https://bioconductor.org/packages/Informeasure.

Information Measures

2015 ◽  
pp. 129-142
Keyword(s):  
Information Theory ◽  
Mutual Information ◽  
Order Logic ◽  
Unified Framework ◽  
Higher Order Logic ◽  
Absolutely Continuous ◽  
Information Measures ◽  
Finite Spaces ◽  
Nikodym Derivative ◽  
Lebesgue Integration

This chapter presents a higher-order-logic formalization of the main concepts of information theory (Cover & Thomas, 1991), such as the Shannon entropy and mutual information, using the formalization of the foundational theories of measure, Lebesgue integration, and probability. The main results of the chapter include the formalizations of the Radon-Nikodym derivative and the Kullback-Leibler (KL) divergence (Coble, 2010). The latter provides a unified framework based on which most of the commonly used measures of information can be defined. The chapter then provides the general definitions that are valid for both discrete and continuous cases and then proves the corresponding reduced expressions where the measures considered are absolutely continuous over finite spaces.

DReSS: a method to quantitatively describe the influence of structural perturbations on state spaces of genetic regulatory networks

2020 ◽  
Author(s):  
Ziqiao Yin ◽  
Binghui Guo ◽  
Shuangge Ma ◽  
Yifan Sun ◽  
Zhilong Mi ◽  
...  
Keyword(s):  
Biological Networks ◽  
Regulatory Networks ◽  
Genetic Regulatory Networks ◽  
Random Networks ◽  
Graph Distance ◽  
State Spaces ◽  
Biological Regulatory Networks ◽  
Structural Perturbations ◽  
The Relationship ◽  
Index Family

Abstract Structures of genetic regulatory networks are not fixed. These structural perturbations can cause changes to the reachability of systems’ state spaces. As system structures are related to genotypes and state spaces are related to phenotypes, it is important to study the relationship between structures and state spaces. However, there is still no method can quantitively describe the reachability differences of two state spaces caused by structural perturbations. Therefore, Difference in Reachability between State Spaces (DReSS) is proposed. DReSS index family can quantitively describe differences of reachability, attractor sets between two state spaces and can help find the key structure in a system, which may influence system’s state space significantly. First, basic properties of DReSS including non-negativity, symmetry and subadditivity are proved. Then, typical examples are shown to explain the meaning of DReSS and the differences between DReSS and traditional graph distance. Finally, differences of DReSS distribution between real biological regulatory networks and random networks are compared. Results show most structural perturbations in biological networks tend to affect reachability inside and between attractor basins rather than to affect attractor set itself when compared with random networks, which illustrates that most genotype differences tend to influence the proportion of different phenotypes and only a few ones can create new phenotypes. DReSS can provide researchers with a new insight to study the relation between genotypes and phenotypes.

scholarly journals A Path-Based Partial Information Decomposition

Entropy ◽  
2020 ◽  
Vol 22 (9) ◽  
pp. 952
Author(s):  
David Sigtermans
Keyword(s):  
Information Theory ◽  
Mutual Information ◽  
Graphical Model ◽  
Partial Information ◽  
Communication Channels ◽  
Negative Information ◽  
Information Measure ◽  
Probability Mass ◽  
Information Components ◽  
Mass Functions

Based on the conceptual basis of information theory, we propose a novel mutual information measure—‘path-based mutual information’. This information measure results from the representation of a set of random variables as a probabilistic graphical model. The edges in this graph are modeled as discrete memoryless communication channels, that is, the underlying data is ergodic, stationary, and the Markov condition is assumed to be applicable. The associated multilinear stochastic maps, tensors, transform source probability mass functions into destination probability mass functions. This allows for an exact expression of the resulting tensor of a cascade of discrete memoryless communication channels in terms of the tensors of the constituting communication channels in the paths. The resulting path-based information measure gives rise to intuitive, non-negative, and additive path-based information components—redundant, unique, and synergistic information—as proposed by Williams and Beer. The path-based redundancy satisfies the axioms postulated by Williams and Beer, the identity axiom postulated by Harder, and the left monotonicity axiom postulated Bertschinger. The ordering relations between redundancies of different joint collections of sources, as captured in the redundancy lattices of Williams and Beer, follow from the data processing inequality. Although negative information components can arise, we speculate that these either result from unobserved variables, or from adding additional sources that are statistically independent from all other sources to a system containing only non-negative information components. This path-based approach illustrates that information theory provides the concepts and measures for a partial information decomposition.

scholarly journals On the α-q-Mutual Information and the α-q-Capacities

Entropy ◽  
2021 ◽  
Vol 23 (6) ◽  
pp. 702
Author(s):  
Velimir Ilić ◽  
Ivan Djordjević
Keyword(s):  
Information Theory ◽  
Mutual Information ◽  
Information Transfer ◽  
Communication Channels ◽  
Detection Error ◽  
Maximum Likelihood Detection ◽  
Information Measures ◽  
Special Cases ◽  
Previous Definition ◽  
Two Parameter

The measures of information transfer which correspond to non-additive entropies have intensively been studied in previous decades. The majority of the work includes the ones belonging to the Sharma–Mittal entropy class, such as the Rényi, the Tsallis, the Landsberg–Vedral and the Gaussian entropies. All of the considerations follow the same approach, mimicking some of the various and mutually equivalent definitions of Shannon information measures, and the information transfer is quantified by an appropriately defined measure of mutual information, while the maximal information transfer is considered as a generalized channel capacity. However, all of the previous approaches fail to satisfy at least one of the ineluctable properties which a measure of (maximal) information transfer should satisfy, leading to counterintuitive conclusions and predicting nonphysical behavior even in the case of very simple communication channels. This paper fills the gap by proposing two parameter measures named the α-q-mutual information and the α-q-capacity. In addition to standard Shannon approaches, special cases of these measures include the α-mutual information and the α-capacity, which are well established in the information theory literature as measures of additive Rényi information transfer, while the cases of the Tsallis, the Landsberg–Vedral and the Gaussian entropies can also be accessed by special choices of the parameters α and q. It is shown that, unlike the previous definition, the α-q-mutual information and the α-q-capacity satisfy the set of properties, which are stated as axioms, by which they reduce to zero in the case of totally destructive channels and to the (maximal) input Sharma–Mittal entropy in the case of perfect transmission, which is consistent with the maximum likelihood detection error. In addition, they are non-negative and less than or equal to the input and the output Sharma–Mittal entropies, in general. Thus, unlike the previous approaches, the proposed (maximal) information transfer measures do not manifest nonphysical behaviors such as sub-capacitance or super-capacitance, which could qualify them as appropriate measures of the Sharma–Mittal information transfer.

scholarly journals Empirical Estimation of Information Measures: A Literature Guide

Entropy ◽  
2019 ◽  
Vol 21 (8) ◽  
pp. 720 ◽  
Author(s):  
Sergio Verdú
Keyword(s):  
Machine Learning ◽  
Information Theory ◽  
Mutual Information ◽  
Relative Entropy ◽  
Data Science ◽  
Differential Entropy ◽  
Empirical Estimation ◽  
Central Importance ◽  
Information Measures ◽  
Related Information

We give a brief survey of the literature on the empirical estimation of entropy, differential entropy, relative entropy, mutual information and related information measures. While those quantities are of central importance in information theory, universal algorithms for their estimation are increasingly important in data science, machine learning, biology, neuroscience, economics, language, and other experimental sciences.

Quantifying Stimulus Discriminability: A Comparison of Information Theory and Ideal Observer Analysis

2005 ◽  
Vol 17 (4) ◽  
pp. 741-778 ◽  
Author(s):  
Eric E. Thomson ◽  
William B. Kristan
Keyword(s):  
Information Theory ◽  
Mutual Information ◽  
Conditional Entropy ◽  
Ideal Observer ◽  
Upper And Lower Bounds ◽  
Specific Information ◽  
Information Theoretic ◽  
Information Measures ◽  
Improved Performance ◽  
Ideal Observer Analysis

Performance in sensory discrimination tasks is commonly quantified using either information theory or ideal observer analysis. These two quantitative frameworks are often assumed to be equivalent. For example, higher mutual information is said to correspond to improved performance of an ideal observer in a stimulus estimation task. To the contrary, drawing on and extending previous results, we show that five information-theoretic quantities (entropy, response-conditional entropy, specific information, equivocation, and mutual information) violate this assumption. More positively, we show how these information measures can be used to calculate upper and lower bounds on ideal observer performance, and vice versa. The results show that the mathematical resources of ideal observer analysis are preferable to information theory for evaluating performance in a stimulus discrimination task. We also discuss the applicability of information theory to questions that ideal observer analysis cannot address.

scholarly journals Biological regulatory networks are less nonlinear than expected by chance

2021 ◽  
Author(s):  
Santosh Manicka ◽  
Kathleen Johnson ◽  
David Murrugarra ◽  
Michael Levin
Keyword(s):  
Biological Networks ◽  
Regulatory Networks ◽  
Boolean Network ◽  
Network Models ◽  
Decomposition Methods ◽  
Boolean Logic ◽  
Nonlinear Interactions ◽  
Biological Regulatory Networks ◽  
Probabilistic Generalization ◽  
Selection For

Nonlinearity is a characteristic of complex biological regulatory networks that has implications ranging from therapy to control. To better understand its nature, we analyzed a suite of published Boolean network models, containing a variety of complex nonlinear interactions, with an approach involving a probabilistic generalization of Boolean logic that George Boole himself had proposed. Leveraging the continuous-nature of this formulation using Taylor-decomposition methods revealed the distinct layers of nonlinearity of the models. A comparison of the resulting series of model approximations with the corresponding sets of randomized ensembles furthermore revealed that the biological networks are relatively more linearly approximable. We hypothesize that this is a result of optimization by natural selection for the purpose of controllability.

scholarly journals bioLQM: a java library for the manipulation and conversion of Logical Qualitative Models of biological networks

2018 ◽  
Author(s):  
Aurélien Naldi
Keyword(s):  
Biological Networks ◽  
Regulatory Networks ◽  
Model Simulation ◽  
Biological Regulatory Networks ◽  
Interactive Tools ◽  
Graphical Interfaces ◽  
Qualitative Models ◽  
Import And Export ◽  
Definition Of ◽  
Java Library

AbstractHere we introduce bioLQM, a new Java software toolkit for the conversion, modification, and analysis of Logical Qualitative Models of biological regulatory networks, aiming to foster the development of novel complementary tools by providing core modelling operations. Based on the definition of multi-valued logical models, it implements import and export facilities, notably for the recent SBML-qual exchange format, as well as for formats used by several popular tools, facilitating the design of workflows combining these tools. Model modifications enable the definition of various perturbations, as well as model reduction, easing the analysis of large models. Another modification enables the study of multi-valued models with tools limited to the Boolean case. Finally, bioLQM provides a framework for the development of novel analysis tools. The current version implements the usual updating modes for model simulation (notably synchronous, asynchronous, and random asynchronous), as well as some static analysis features for the identification of attractors. The bioLQM software can be integrated into analysis workflows through command line and scripting interfaces. As a Java library, it further provides core data structures to the GINsim and EpiLog interactive tools, which supply graphical interfaces and additional analysis methods for cellular and multi-cellular qualitative models.

Algorithmic and Stochastic Representations of Gene Regulatory Networks and Protein-Protein Interactions

2019 ◽  
Vol 19 (6) ◽  
pp. 413-425 ◽  
Author(s):  
Athanasios Alexiou ◽  
Stylianos Chatzichronis ◽  
Asma Perveen ◽  
Abdul Hafeez ◽  
Ghulam Md. Ashraf
Keyword(s):  
Protein Interactions ◽  
Biological Networks ◽  
Regulatory Networks ◽  
Review Paper ◽  
Boolean Networks ◽  
Diagnostic Tools ◽  
Complex Nature ◽  
Cellular Interactions ◽  
Protein Protein Interactions ◽  
Deterministic Models

Background:Latest studies reveal the importance of Protein-Protein interactions on physiologic functions and biological structures. Several stochastic and algorithmic methods have been published until now, for the modeling of the complex nature of the biological systems.Objective:Biological Networks computational modeling is still a challenging task. The formulation of the complex cellular interactions is a research field of great interest. In this review paper, several computational methods for the modeling of GRN and PPI are presented analytically.Methods:Several well-known GRN and PPI models are presented and discussed in this review study such as: Graphs representation, Boolean Networks, Generalized Logical Networks, Bayesian Networks, Relevance Networks, Graphical Gaussian models, Weight Matrices, Reverse Engineering Approach, Evolutionary Algorithms, Forward Modeling Approach, Deterministic models, Static models, Hybrid models, Stochastic models, Petri Nets, BioAmbients calculus and Differential Equations.Results:GRN and PPI methods have been already applied in various clinical processes with potential positive results, establishing promising diagnostic tools.Conclusion:In literature many stochastic algorithms are focused in the simulation, analysis and visualization of the various biological networks and their dynamics interactions, which are referred and described in depth in this review paper.

Sign in / Sign up

Export Citation Format

Share Document