scholarly journals What’s Worth Talking About? Information Theory Reveals How Children Balance Informativeness and Ease of Production

2017 ◽  
Vol 28 (7) ◽  
pp. 954-966 ◽  
Author(s):  
Colin Bannard ◽  
Marla Rosner ◽  
Danielle Matthews
Keyword(s):  
Information Theory ◽  
Information Content ◽  
Low Frequency ◽  
Initial Experiment ◽  
Information Theoretic ◽  
Information Theoretic Measures

Of all the things a person could say in a given situation, what determines what is worth saying? Greenfield’s principle of informativeness states that right from the onset of language, humans selectively comment on whatever they find unexpected. In this article, we quantify this tendency using information-theoretic measures and report on a study in which we tested the counterintuitive prediction that children will produce words that have a low frequency given the context, because these will be most informative. Using corpora of child-directed speech, we identified adjectives that varied in how informative (i.e., unexpected) they were given the noun they modified. In an initial experiment ( N = 31) and in a replication ( N = 13), 3-year-olds heard an experimenter use these adjectives to describe pictures. The children’s task was then to describe the pictures to another person. As the information content of the experimenter’s adjective increased, so did children’s tendency to comment on the feature that adjective had encoded. Furthermore, our analyses suggest that children balance informativeness with a competing drive to ease production.

scholarly journals Quantifying Information Content in Multispectral Remote-Sensing Images Based on Image Transforms and Geostatistical Modelling

2020 ◽  
Vol 12 (5) ◽  
pp. 880
Author(s):  
Ying Zhang ◽  
Jingxiong Zhang ◽  
Wenjing Yang
Keyword(s):  
Remote Sensing ◽  
Information Theory ◽  
Information Content ◽  
Spatial Dependence ◽  
Spectral Dependence ◽  
Systematic Research ◽  
Multispectral Images ◽  
Remote Sensing Images ◽  
Information Theoretic ◽  
Information Processes

Quantifying information content in remote-sensing images is fundamental for information-theoretic characterization of remote sensing information processes, with the images being usually information sources. Information-theoretic methods, being complementary to conventional statistical methods, enable images and their derivatives to be described and analyzed in terms of information as defined in information theory rather than data per se. However, accurately quantifying images’ information content is nontrivial, as information redundancy due to spectral and spatial dependence needs to be properly handled. There has been little systematic research on this, hampering wide applications of information theory. This paper seeks to fill this important research niche by proposing a strategy for quantifying information content in multispectral images based on information theory, geostatistics, and image transformations, by which interband spectral dependence, intraband spatial dependence, and additive noise inherent to multispectral images are effectively dealt with. Specifically, to handle spectral dependence, independent component analysis (ICA) is performed to transform a multispectral image into one with statistically independent image bands (not spectral bands of the original image). The ICA-transformed image is further normal-transformed to facilitate computation of information content based on entropy formulas for Gaussian distributions. Normal transform facilitates straightforward incorporation of spatial dependence in entropy computation for the aforementioned double-transformed image bands with inter-pixel spatial correlation modeled via variograms. Experiments were undertaken using Landsat ETM+ and TM image subsets featuring different dominant land cover types (i.e., built-up, agricultural, and hilly). The experimental results confirm that the proposed methods provide more objective estimates of information content than otherwise when spectral dependence, spatial dependence, or non-normality is not accommodated properly. The differences in information content between image subsets obtained with ETM+ and TM were found to be about 3.6 bits/pixel, indicating the former’s greater information content. The proposed methods can be adapted for information-theoretic analyses of remote sensing information processes.

scholarly journals The Expectancy Dynamics of Anti-Tonal Twelve-Tone Rows: A Commentary and Reanalysis of von Hippel & Huron (2020)

2020 ◽  
Vol 15 (1-2) ◽  
pp. 128
Author(s):  
Niels Chr. Hansen
Keyword(s):  
Computational Model ◽  
Information Content ◽  
Music Cognition ◽  
Arnold Schoenberg ◽  
Anton Webern ◽  
Alban Berg ◽  
Twelve Tone ◽  
Information Theoretic ◽  
Information Theoretic Measures ◽  
High Uncertainty

This commentary provides two methodological expansions of von Hippel and Huron's (2020) empirical report on (anti-)tonality in twelve-tone rows by Arnold Schoenberg, Anton Webern, and Alban Berg. First, motivated by the theoretical importance of equality between all pitch classes in twelve-tone music, a full replication of their findings of "anti-tonality" in rows by Schoenberg and Webern is offered using a revised tonal fit measure which is not biased towards row-initial and row-final sub-segments. Second, motivated by a long-standing debate in music cognition research between distributional and sequential/dynamic tonality concepts, information-theoretic measures of entropy and information content are estimated by a computational model and pitted against distributional tonal fit measures. Whereas Schoenberg's rows are characterized by low distributional tonal fit, but do not strongly capitalize on tonal expectancy dynamics, Berg's rows exhibit tonal traits in terms of low unexpectedness, and Webern's rows achieve anti-tonal traits by combining high uncertainty and low unexpectedness through prominent use of the semitone interval. This analysis offers a complementary–and arguably more nuanced–picture of dodecaphonic compositional practice.

A Note on the Residual Entropy Function

1993 ◽  
Vol 7 (3) ◽  
pp. 413-420 ◽  
Author(s):  
Pietro Muliere ◽  
Giovanni Parmigiani ◽  
Nicholas G. Polson
Keyword(s):  
Information Theory ◽  
Probability Distribution ◽  
Random Variable ◽  
Reliability Theory ◽  
Entropy Function ◽  
Informational Content ◽  
Residual Entropy ◽  
Regularity Conditions ◽  
Information Theoretic ◽  
Information Theoretic Measures

Interest in the informational content of truncation motivates the study of the residual entropy function, that is, the entropy of a right truncated random variable as a function of the truncation point. In this note we show that, under mild regularity conditions, the residual entropy function characterizes the probability distribution. We also derive relationships among residual entropy, monotonicity of the failure rate, and stochastic dominance. Information theoretic measures of distances between distributions are also revisited from a similar perspective. In particular, we study the residual divergence between two positive random variables and investigate some of its monotonicity properties. The results are relevant to information theory, reliability theory, search problems, and experimental design.

Limitations of Archaeological Inference: An Information-Theoretic Approach with Applications in Methodology

1973 ◽  
Vol 38 (2) ◽  
pp. 131-149 ◽  
Author(s):  
John S. Justeson
Keyword(s):  
Information Theory ◽  
Theoretic Approach ◽  
Information Theoretic ◽  
Design Elements ◽  
Utility Of Information ◽  
Information Theoretic Measures ◽  
Information Theoretic Approach

AbstractA framework is established for the application of information-theoretic concepts to the study of archaeological inference, ultimately to provide an estimate of the degree to which archaeologists, or anthropologists in general, can provide legitimate answers to the questions they investigate. Particular information-theoretic measures are applied to the design elements on the ceramics of a southwestern pueblo to show the methodological utility of information theory in helping to reach closer to that limit.

Large-sample confidence intervals of information-theoretic measures in linguistics

2020 ◽  
Vol 6 (1) ◽  
pp. 19-54
Author(s):  
Ryan Ka Yau Lai ◽  
Youngah Do
Keyword(s):  
Maximum Likelihood ◽  
Corpus Linguistics ◽  
Delta Method ◽  
Confidence Bounds ◽  
Likelihood Estimator ◽  
Information Theoretic ◽  
Leibler Divergence ◽  
Information Theoretic Measures ◽  
Data Points ◽  
Measure Of Uncertainty

This article explores a method of creating confidence bounds for information-theoretic measures in linguistics, such as entropy, Kullback-Leibler Divergence (KLD), and mutual information. We show that a useful measure of uncertainty can be derived from simple statistical principles, namely the asymptotic distribution of the maximum likelihood estimator (MLE) and the delta method. Three case studies from phonology and corpus linguistics are used to demonstrate how to apply it and examine its robustness against common violations of its assumptions in linguistics, such as insufficient sample size and non-independence of data points.

scholarly journals Optimized permutation testing for information theoretic measures of multi-gene interactions

2021 ◽  
Vol 22 (1) ◽  
Author(s):  
James M. Kunert-Graf ◽  
Nikita A. Sakhanenko ◽  
David J. Galas
Keyword(s):  
Large Scale ◽  
Permutation Test ◽  
Association Studies ◽  
Genome Wide Association Studies ◽  
Permutation Testing ◽  
Exact Test ◽  
Information Theoretic ◽  
Information Theoretic Measures ◽  
Full Analysis ◽  
Computational Bottleneck

Abstract Background Permutation testing is often considered the “gold standard” for multi-test significance analysis, as it is an exact test requiring few assumptions about the distribution being computed. However, it can be computationally very expensive, particularly in its naive form in which the full analysis pipeline is re-run after permuting the phenotype labels. This can become intractable in multi-locus genome-wide association studies (GWAS), in which the number of potential interactions to be tested is combinatorially large. Results In this paper, we develop an approach for permutation testing in multi-locus GWAS, specifically focusing on SNP–SNP-phenotype interactions using multivariable measures that can be computed from frequency count tables, such as those based in Information Theory. We find that the computational bottleneck in this process is the construction of the count tables themselves, and that this step can be eliminated at each iteration of the permutation testing by transforming the count tables directly. This leads to a speed-up by a factor of over 103 for a typical permutation test compared to the naive approach. Additionally, this approach is insensitive to the number of samples making it suitable for datasets with large number of samples. Conclusions The proliferation of large-scale datasets with genotype data for hundreds of thousands of individuals enables new and more powerful approaches for the detection of multi-locus genotype-phenotype interactions. Our approach significantly improves the computational tractability of permutation testing for these studies. Moreover, our approach is insensitive to the large number of samples in these modern datasets. The code for performing these computations and replicating the figures in this paper is freely available at https://github.com/kunert/permute-counts.

Insights into codeswitching from online communication: Effects of language preference and conditions arising from vocabulary richness

2021 ◽  
pp. 1-7
Author(s):  
Laurie Beth Feldman ◽  
Vidhushini Srinivasan ◽  
Rachel B. Fernandes ◽  
Samira Shaikh
Keyword(s):  
Online Communication ◽  
Language Preference ◽  
Lexical Diversity ◽  
Information Theoretic ◽  
Vocabulary Richness ◽  
Twitter Data ◽  
Language Mixing ◽  
Communication Effects ◽  
Information Theoretic Measures ◽  
Spanish Bilinguals

Abstract Twitter data from a crisis that impacted many English–Spanish bilinguals show that the direction of codeswitches is associated with the statistically documented tendency of single speakers to prefer one language over another in their tweets, as gleaned from their tweeting history. Further, lexical diversity, a measure of vocabulary richness derived from information-theoretic measures of uncertainty in communication, is greater in proximity to a codeswitch than in productions remote from a switch. The prospects of a role for lexical diversity in characterizing the conditions for a language switch suggest that communicative precision may induce conditions that attenuate constraints against language mixing.

scholarly journals Area Entropy and Quantized Mass of Black Holes from Information Theory

Entropy ◽  
2021 ◽  
Vol 23 (7) ◽  
pp. 858
Author(s):  
Dongshan He ◽  
Qingyu Cai
Keyword(s):  
Information Theory ◽  
Black Hole ◽  
Black Hole Entropy ◽  
Quantum Corrections ◽  
Compton Wavelength ◽  
Information Theoretic ◽  
Curved Space ◽  
Thermodynamic Entropy ◽  
Hole Area ◽  
The Relationship

In this paper, we present a derivation of the black hole area entropy with the relationship between entropy and information. The curved space of a black hole allows objects to be imaged in the same way as camera lenses. The maximal information that a black hole can gain is limited by both the Compton wavelength of the object and the diameter of the black hole. When an object falls into a black hole, its information disappears due to the no-hair theorem, and the entropy of the black hole increases correspondingly. The area entropy of a black hole can thus be obtained, which indicates that the Bekenstein–Hawking entropy is information entropy rather than thermodynamic entropy. The quantum corrections of black hole entropy are also obtained according to the limit of Compton wavelength of the captured particles, which makes the mass of a black hole naturally quantized. Our work provides an information-theoretic perspective for understanding the nature of black hole entropy.

scholarly journals Information-Theoretic Measures and Modeling Stock Market Volatility: A Comparative Approach

Risks ◽  
2021 ◽  
Vol 9 (5) ◽  
pp. 89
Author(s):  
Muhammad Sheraz ◽  
Imran Nasir
Keyword(s):  
Stock Market ◽  
Stock Returns ◽  
Stock Exchange ◽  
Approximate Entropy ◽  
Market Volatility ◽  
Comparative Approach ◽  
Stock Market Volatility ◽  
Information Theoretic ◽  
Information Theoretic Measures ◽  
Garch Modeling

The volatility analysis of stock returns data is paramount in financial studies. We investigate the dynamics of volatility and randomness of the Pakistan Stock Exchange (PSX-100) and obtain insights into the behavior of investors during and before the coronavirus disease (COVID-19 pandemic). The paper aims to present the volatility estimations and quantification of the randomness of PSX-100. The methodology includes two approaches: (i) the implementation of EGARCH, GJR-GARCH, and TGARCH models to estimate the volatilities; and (ii) analysis of randomness in volatilities series, return series, and PSX-100 closing prices for pre-pandemic and pandemic period by using Shannon’s, Tsallis, approximate and sample entropies. Volatility modeling suggests the existence of the leverage effect in both the underlying periods of study. The results obtained using GARCH modeling reveal that the stock market volatility has increased during the pandemic period. However, information-theoretic results based on Shannon and Tsallis entropies do not suggest notable variation in the estimated volatilities series and closing prices. We have examined regularity and randomness based on the approximate entropy and sample entropy. We have noticed both entropies are extremely sensitive to choices of the parameters.

Measuring the Complexity of Additive Manufacturing Supply Chains

2017 ◽  
Author(s):  
Ardeshir Raihanian Mashhadi ◽  
Sara Behdad
Keyword(s):  
Supply Chain ◽  
Additive Manufacturing ◽  
Supply Chains ◽  
Product Complexity ◽  
Information Theoretic ◽  
Manufacturing Method ◽  
Management Domain ◽  
Information Theoretic Measures ◽  
Production Cycles ◽  
Supply Planning

Complexity has been one of the focal points of attention in the supply chain management domain, as it deteriorates the performance of the supply chain and makes controlling it problematic. The complexity of supply chains has been significantly increased over the past couple of decades. Meanwhile, Additive Manufacturing (AM) not only revolutionizes the way that the products are made, but also brings a paradigm shift to the whole production system. The influence of AM extends to product design and supply chain as well. The unique capabilities of AM suggest that this manufacturing method can significantly affect the supply chain complexity. More product complexity and demand heterogeneity, faster production cycles, higher levels of automation and shorter supply paths are among the features of additive manufacturing that can directly influence the supply chain complexity. Comparison of additive manufacturing supply chain complexity to its traditional counterpart requires a profound comprehension of the transformative effects of AM on the supply chain. This paper first extracts the possible effects of AM on the supply chain and then tries to connect these effects to the drivers of complexity under three main categories of 1) market, 2) manufacturing technology, and 3) supply, planning and infrastructure. Possible impacts of additive manufacturing adoption on the supply chain complexity have been studied using information theoretic measures. An Agent-based Simulation (ABS) model has been developed to study and compare two different supply chain configurations. The findings of this study suggest that the adoption of AM can decrease the supply chain complexity, particularly when product customization is considered.

Sign in / Sign up

Export Citation Format

Share Document