scholarly journals Decomposing information into copying versus transformation

2019 ◽  
Author(s):  
Artemy Kolchinsky ◽  
Bernat Corominas-Murtra
Keyword(s):  
Amino Acid ◽  
Information Transfer ◽  
World Systems ◽  
Information Theoretic ◽  
Versus Transformation ◽  
Information Theoretic Measures ◽  
Functional Consequences ◽  
Standard Information ◽  
Biological Reproduction ◽  
Minimal Work

In many real-world systems, information can be transmitted in two qualitatively different ways: by copying or by transformation. Copying occurs when messages are transmitted without modification, e.g., when an offspring receives an unaltered copy of a gene from its parent. Transformation occurs when messages are modified systematically during transmission, e.g., when non-random mutations occur during biological reproduction. Standard information-theoretic measures do not distinguish these two modes of information transfer, although they may reflect different mechanisms and have different functional consequences. Starting from a few simple axioms, we derive a decomposition of mutual information into the information transmitted by copying and by transformation. Our decomposition applies whenever the source and destination of the channel have the same set of outcomes, so that a notion of message identity exists, although generalizations to other kinds of channels and similarity notions are explored. Furthermore, copy information can be interpreted as the minimal work needed by a physical copying process, relevant to better understand the physics of replication. We use the proposed decomposition to explore a model of amino acid substitution rates. Our results apply to any system in which the fidelity of copying, rather than simple predictability, is of critical relevance.

scholarly journals Decomposing information into copying versus transformation

2020 ◽  
Vol 17 (162) ◽  
pp. 20190623 ◽  
Author(s):  
Artemy Kolchinsky ◽  
Bernat Corominas-Murtra
Keyword(s):  
Amino Acid ◽  
Mutual Information ◽  
Information Transfer ◽  
World Systems ◽  
Information Theoretic ◽  
Versus Transformation ◽  
Information Theoretic Measures ◽  
Functional Consequences ◽  
Standard Information ◽  
Minimal Work

In many real-world systems, information can be transmitted in two qualitatively different ways: by copying or by transformation . Copying occurs when messages are transmitted without modification, e.g. when an offspring receives an unaltered copy of a gene from its parent. Transformation occurs when messages are modified systematically during transmission, e.g. when mutational biases occur during genetic replication. Standard information-theoretic measures do not distinguish these two modes of information transfer, although they may reflect different mechanisms and have different functional consequences. Starting from a few simple axioms, we derive a decomposition of mutual information into the information transmitted by copying versus the information transmitted by transformation. We begin with a decomposition that applies when the source and destination of the channel have the same set of messages and a notion of message identity exists. We then generalize our decomposition to other kinds of channels, which can involve different source and destination sets and broader notions of similarity. In addition, we show that copy information can be interpreted as the minimal work needed by a physical copying process, which is relevant for understanding the physics of replication. We use the proposed decomposition to explore a model of amino acid substitution rates. Our results apply to any system in which the fidelity of copying, rather than simple predictability, is of critical relevance.

scholarly journals Information-theoretic measures for nonlinear causality detection: application to social media sentiment and cryptocurrency prices

2020 ◽  
Vol 7 (9) ◽  
pp. 200863
Author(s):  
Z. Keskin ◽  
T. Aste
Keyword(s):  
Time Series ◽  
Information Transfer ◽  
Synthetic Data ◽  
Transfer Entropy ◽  
Significant Information ◽  
Information Theoretic ◽  
Order Of Magnitude ◽  
Information Theoretic Measures ◽  
Media Sentiment ◽  
Non Parametric

Information transfer between time series is calculated using the asymmetric information-theoretic measure known as transfer entropy. Geweke’s autoregressive formulation of Granger causality is used to compute linear transfer entropy, and Schreiber’s general, non-parametric, information-theoretic formulation is used to quantify nonlinear transfer entropy. We first validate these measures against synthetic data. Then we apply these measures to detect statistical causality between social sentiment changes and cryptocurrency returns. We validate results by performing permutation tests by shuffling the time series, and calculate the Z -score. We also investigate different approaches for partitioning in non-parametric density estimation which can improve the significance. Using these techniques on sentiment and price data over a 48-month period to August 2018, for four major cryptocurrencies, namely bitcoin (BTC), ripple (XRP), litecoin (LTC) and ethereum (ETH), we detect significant information transfer, on hourly timescales, with greater net information transfer from sentiment to price for XRP and LTC, and instead from price to sentiment for BTC and ETH. We report the scale of nonlinear statistical causality to be an order of magnitude larger than the linear case.

scholarly journals Unified framework for information integration based on information geometry

2016 ◽  
Vol 113 (51) ◽  
pp. 14817-14822 ◽  
Author(s):  
Masafumi Oizumi ◽  
Naotsugu Tsuchiya ◽  
Shun-ichi Amari
Keyword(s):  
Probability Distribution ◽  
Information Integration ◽  
Information Transfer ◽  
Holistic Approach ◽  
Information Geometry ◽  
Unified Framework ◽  
Information Theoretic ◽  
Research Fields ◽  
Information Theoretic Measures ◽  
Integrated Information

Assessment of causal influences is a ubiquitous and important subject across diverse research fields. Drawn from consciousness studies, integrated information is a measure that defines integration as the degree of causal influences among elements. Whereas pairwise causal influences between elements can be quantified with existing methods, quantifying multiple influences among many elements poses two major mathematical difficulties. First, overestimation occurs due to interdependence among influences if each influence is separately quantified in a part-based manner and then simply summed over. Second, it is difficult to isolate causal influences while avoiding noncausal confounding influences. To resolve these difficulties, we propose a theoretical framework based on information geometry for the quantification of multiple causal influences with a holistic approach. We derive a measure of integrated information, which is geometrically interpreted as the divergence between the actual probability distribution of a system and an approximated probability distribution where causal influences among elements are statistically disconnected. This framework provides intuitive geometric interpretations harmonizing various information theoretic measures in a unified manner, including mutual information, transfer entropy, stochastic interaction, and integrated information, each of which is characterized by how causal influences are disconnected. In addition to the mathematical assessment of consciousness, our framework should help to analyze causal relationships in complex systems in a complete and hierarchical manner.

scholarly journals Assessment of Cardiorespiratory Interactions during Apneic Events in Sleep via Fuzzy Kernel Measures of Information Dynamics

Entropy ◽  
2021 ◽  
Vol 23 (6) ◽  
pp. 698
Author(s):  
Ivan Lazic ◽  
Riccardo Pernice ◽  
Tatjana Loncar-Turukalo ◽  
Gorana Mijatovic ◽  
Luca Faes
Keyword(s):  
Information Transfer ◽  
Information Storage ◽  
Heart Period ◽  
Sinus Arrhythmia ◽  
Predictive Information ◽  
Information Theoretic ◽  
Metabolic Systems ◽  
Respiratory Event ◽  
Information Theoretic Measures ◽  
The Impact

Apnea and other breathing-related disorders have been linked to the development of hypertension or impairments of the cardiovascular, cognitive or metabolic systems. The combined assessment of multiple physiological signals acquired during sleep is of fundamental importance for providing additional insights about breathing disorder events and the associated impairments. In this work, we apply information-theoretic measures to describe the joint dynamics of cardiorespiratory physiological processes in a large group of patients reporting repeated episodes of hypopneas, apneas (central, obstructive, mixed) and respiratory effort related arousals (RERAs). We analyze the heart period as the target process and the airflow amplitude as the driver, computing the predictive information, the information storage, the information transfer, the internal information and the cross information, using a fuzzy kernel entropy estimator. The analyses were performed comparing the information measures among segments during, immediately before and after the respiratory event and with control segments. Results highlight a general tendency to decrease of predictive information and information storage of heart period, as well as of cross information and information transfer from respiration to heart period, during the breathing disordered events. The information-theoretic measures also vary according to the breathing disorder, and significant changes of information transfer can be detected during RERAs, suggesting that the latter could represent a risk factor for developing cardiovascular diseases. These findings reflect the impact of different sleep breathing disorders on respiratory sinus arrhythmia, suggesting overall higher complexity of the cardiac dynamics and weaker cardiorespiratory interactions which may have physiological and clinical relevance.

scholarly journals Information transfer and recovery for the sense of touch

2020 ◽  
Author(s):  
Chao Huang ◽  
Bernhard Englitz ◽  
Andrey Reznik ◽  
Fleur Zeldenrust ◽  
Tansu Celikel
Keyword(s):  
Information Transfer ◽  
Sensory Information ◽  
Single Neurons ◽  
Information Theoretic ◽  
Rate Limiting Step ◽  
Information Theoretic Measures ◽  
Sensory Cortices ◽  
A Cell ◽  
Trial Basis ◽  
Sense Of Touch

Transformation of postsynaptic potentials (PSPs) into action potentials (APs) is the rate-limiting step of communication in neural networks. The efficiency of this intracellular information transfer also powerfully shapes stimulus representations in sensory cortices. Using whole-cell recordings and information-theoretic measures, we show herein that somatic PSPs accurately represent stimulus location on a trial-by-trial basis in single neurons even 4 synapses away from the sensory periphery in the whisker system. This information is largely lost during AP generation but can be rapidly (<20 ms) recovered using complementary information in local populations in a cell-type-specific manner. These results show that as sensory information is transferred from one neural locus to another, the circuits reconstruct the stimulus with high fidelity so that sensory representations of single neurons faithfully represent the stimulus in the periphery, but only in their PSPs, resulting in lossless information processing for the sense of touch in the primary somatosensory cortex.

scholarly journals Transfer Entropy Analysis of Interactions between Bats Using Position and Echolocation Data

Entropy ◽  
2020 ◽  
Vol 22 (10) ◽  
pp. 1176
Author(s):  
Irena Shaffer ◽  
Nicole Abaid
Keyword(s):  
Information Transfer ◽  
Animal Species ◽  
Transfer Entropy ◽  
Signal Interference ◽  
Information Theoretic ◽  
Echolocation Calls ◽  
Primary Means ◽  
Information Theoretic Measures ◽  
Large Groups

Many animal species, including many species of bats, exhibit collective behavior where groups of individuals coordinate their motion. Bats are unique among these animals in that they use the active sensing mechanism of echolocation as their primary means of navigation. Due to their use of echolocation in large groups, bats run the risk of signal interference from sonar jamming. However, several species of bats have developed strategies to prevent interference, which may lead to different behavior when flying with conspecifics than when flying alone. This study seeks to explore the role of this acoustic sensing on the behavior of bat pairs flying together. Field data from a maternity colony of gray bats (Myotis grisescens) were collected using an array of cameras and microphones. These data were analyzed using the information theoretic measure of transfer entropy in order to quantify the interaction between pairs of bats and to determine the effect echolocation calls have on this interaction. This study expands on previous work that only computed information theoretic measures on the 3D position of bats without echolocation calls or that looked at the echolocation calls without using information theoretic analyses. Results show that there is evidence of information transfer between bats flying in pairs when time series for the speed of the bats and their turning behavior are used in the analysis. Unidirectional information transfer was found in some subsets of the data which could be evidence of a leader–follower interaction.

Large-sample confidence intervals of information-theoretic measures in linguistics

2020 ◽  
Vol 6 (1) ◽  
pp. 19-54
Author(s):  
Ryan Ka Yau Lai ◽  
Youngah Do
Keyword(s):  
Maximum Likelihood ◽  
Corpus Linguistics ◽  
Delta Method ◽  
Confidence Bounds ◽  
Likelihood Estimator ◽  
Information Theoretic ◽  
Leibler Divergence ◽  
Information Theoretic Measures ◽  
Data Points ◽  
Measure Of Uncertainty

This article explores a method of creating confidence bounds for information-theoretic measures in linguistics, such as entropy, Kullback-Leibler Divergence (KLD), and mutual information. We show that a useful measure of uncertainty can be derived from simple statistical principles, namely the asymptotic distribution of the maximum likelihood estimator (MLE) and the delta method. Three case studies from phonology and corpus linguistics are used to demonstrate how to apply it and examine its robustness against common violations of its assumptions in linguistics, such as insufficient sample size and non-independence of data points.

scholarly journals Optimized permutation testing for information theoretic measures of multi-gene interactions

2021 ◽  
Vol 22 (1) ◽  
Author(s):  
James M. Kunert-Graf ◽  
Nikita A. Sakhanenko ◽  
David J. Galas
Keyword(s):  
Large Scale ◽  
Permutation Test ◽  
Association Studies ◽  
Genome Wide Association Studies ◽  
Permutation Testing ◽  
Exact Test ◽  
Information Theoretic ◽  
Information Theoretic Measures ◽  
Full Analysis ◽  
Computational Bottleneck

Abstract Background Permutation testing is often considered the “gold standard” for multi-test significance analysis, as it is an exact test requiring few assumptions about the distribution being computed. However, it can be computationally very expensive, particularly in its naive form in which the full analysis pipeline is re-run after permuting the phenotype labels. This can become intractable in multi-locus genome-wide association studies (GWAS), in which the number of potential interactions to be tested is combinatorially large. Results In this paper, we develop an approach for permutation testing in multi-locus GWAS, specifically focusing on SNP–SNP-phenotype interactions using multivariable measures that can be computed from frequency count tables, such as those based in Information Theory. We find that the computational bottleneck in this process is the construction of the count tables themselves, and that this step can be eliminated at each iteration of the permutation testing by transforming the count tables directly. This leads to a speed-up by a factor of over 103 for a typical permutation test compared to the naive approach. Additionally, this approach is insensitive to the number of samples making it suitable for datasets with large number of samples. Conclusions The proliferation of large-scale datasets with genotype data for hundreds of thousands of individuals enables new and more powerful approaches for the detection of multi-locus genotype-phenotype interactions. Our approach significantly improves the computational tractability of permutation testing for these studies. Moreover, our approach is insensitive to the large number of samples in these modern datasets. The code for performing these computations and replicating the figures in this paper is freely available at https://github.com/kunert/permute-counts.

Insights into codeswitching from online communication: Effects of language preference and conditions arising from vocabulary richness

2021 ◽  
pp. 1-7
Author(s):  
Laurie Beth Feldman ◽  
Vidhushini Srinivasan ◽  
Rachel B. Fernandes ◽  
Samira Shaikh
Keyword(s):  
Online Communication ◽  
Language Preference ◽  
Lexical Diversity ◽  
Information Theoretic ◽  
Vocabulary Richness ◽  
Twitter Data ◽  
Language Mixing ◽  
Communication Effects ◽  
Information Theoretic Measures ◽  
Spanish Bilinguals

Abstract Twitter data from a crisis that impacted many English–Spanish bilinguals show that the direction of codeswitches is associated with the statistically documented tendency of single speakers to prefer one language over another in their tweets, as gleaned from their tweeting history. Further, lexical diversity, a measure of vocabulary richness derived from information-theoretic measures of uncertainty in communication, is greater in proximity to a codeswitch than in productions remote from a switch. The prospects of a role for lexical diversity in characterizing the conditions for a language switch suggest that communicative precision may induce conditions that attenuate constraints against language mixing.

Sign in / Sign up

Export Citation Format

Share Document