Tight Bounds on the Rényi Entropy via Majorization with Applications to Guessing and Compression

This paper provides tight bounds on the Rényi entropy of a function of a discrete random variable with a finite number of possible values, where the considered function is not one to one. To that end, a tight lower bound on the Rényi entropy of a discrete random variable with a finite support is derived as a function of the size of the support, and the ratio of the maximal to minimal probability masses. This work was inspired by the recently published paper by Cicalese et al., which is focused on the Shannon entropy, and it strengthens and generalizes the results of that paper to Rényi entropies of arbitrary positive orders. In view of these generalized bounds and the works by Arikan and Campbell, non-asymptotic bounds are derived for guessing moments and lossless data compression of discrete memoryless sources.

Download Full-text

Dynamics of non-stationary processes that follow the maximum of the Rényi entropy principle

Proceedings of The Royal Society A Mathematical Physical and Engineering Sciences ◽

10.1098/rspa.2015.0324 ◽

2016 ◽

Vol 472 (2185) ◽

pp. 20150324 ◽

Cited By ~ 7

Author(s):

Dmitry S. Shalymov ◽

Alexander L. Fradkov

Keyword(s):

Limit Distribution ◽

Mass Conservation ◽

Random Variable ◽

Renyi Entropy ◽

Rényi Entropy ◽

Stationary Processes ◽

Entropy Principle ◽

Gradient Principle ◽

Probability Density Function Pdf ◽

Speed Gradient

We propose dynamics equations which describe the behaviour of non-stationary processes that follow the maximum Rényi entropy principle. The equations are derived on the basis of the speed-gradient principle originated in the control theory. The maximum of the Rényi entropy principle is analysed for discrete and continuous cases, and both a discrete random variable and probability density function (PDF) are used. We consider mass conservation and energy conservation constraints and demonstrate the uniqueness of the limit distribution and asymptotic convergence of the PDF for both cases. The coincidence of the limit distribution of the proposed equations with the Rényi distribution is examined.

Download Full-text

The Rényi entropy H2 as a rigorous, measurable lower bound for the entropy of the interaction region in multi-particle production processes

The European Physical Journal C ◽

10.1140/epjc/s2006-02623-6 ◽

2006 ◽

Vol 48 (1) ◽

pp. 215-221 ◽

Cited By ~ 2

Author(s):

A. Bialas ◽

W. Czyz ◽

K. Zalewski

Keyword(s):

Lower Bound ◽

Particle Production ◽

Interaction Region ◽

Renyi Entropy ◽

Rényi Entropy ◽

Production Processes

Download Full-text

Asymptotic bounds for the fluid queue fed by sub-exponential On/Off sources

Advances in Applied Probability ◽

10.1017/s0001867800009861 ◽

2000 ◽

Vol 32 (01) ◽

pp. 244-255 ◽

Cited By ~ 7

Author(s):

V. Dumas ◽

A. Simonian

Keyword(s):

Lower Bound ◽

Finite Number ◽

Upper Bound ◽

Content Distribution ◽

Upper Bounds ◽

Lower And Upper Bounds ◽

Fluid Queue ◽

Asymptotic Bounds ◽

Multiplicative Factor ◽

Buffer Content

We consider a fluid queue fed by a superposition of a finite number of On/Off sources, the distribution of the On period being subexponential for some of them and exponential for the others. We provide general lower and upper bounds for the tail of the stationary buffer content distribution in terms of the so-called minimal subsets of sources. We then show that this tail decays at exponential or subexponential speed according as a certain parameter is smaller or larger than the ouput rate. If we replace the subexponential tails by regularly varying tails, the upper bound and the lower bound are sharp in that they differ only by a multiplicative factor.

Download Full-text

Study of Quantile-Based Cumulative Renyi Information Measure

Statistics Optimization & Information Computing ◽

10.19139/soic-2310-5070-1034 ◽

2020 ◽

Vol 9 (4) ◽

pp. 886-909

Author(s):

Rekha ◽

Vikas Kumar

Keyword(s):

Order Statistic ◽

Random Variable ◽

Information Measure ◽

Renyi Entropy ◽

Rényi Entropy ◽

Reliability Measure ◽

Extreme Order Statistic ◽

The Relationship

In this paper, we proposed a quantile version of cumulative Renyi entropy for residual and past lifetimes and study their properties. We also study quantile-based cumulative Renyi entropy for extreme order statistic when random variable untruncated or truncated in nature. Some characterization results are studied using the relationship between proposed information measure and reliability measure. We also examine it in relation to some applied problems such as weighted and equillibrium models.

Download Full-text

Divisor function inequalities, entropy, and the chance of being below average

Mathematical Proceedings of the Cambridge Philosophical Society ◽

10.1017/s0305004117000147 ◽

2017 ◽

Vol 163 (3) ◽

pp. 547-560

Author(s):

ZARATHUSTRA BRADY

Keyword(s):

Lower Bound ◽

Natural Condition ◽

Geometric Distribution ◽

Random Variable ◽

Divisor Function ◽

Finite Support ◽

Fixed Power

AbstractWe extend a lower bound of Munshi on sums over divisors of a number n which are less than a fixed power of n from the squarefree case to the general case. In the process we prove a lower bound on the entropy of a geometric distribution with finite support, as well as a lower bound on the probability that a random variable is less than its mean given that it satisfies a natural condition related to its third cumulant.

Download Full-text

Asymptotic bounds for the fluid queue fed by sub-exponential On/Off sources

Advances in Applied Probability ◽

10.1239/aap/1013540032 ◽

2000 ◽

Vol 32 (1) ◽

pp. 244-255 ◽

Cited By ~ 9

Author(s):

V. Dumas ◽

A. Simonian

Keyword(s):

Lower Bound ◽

Finite Number ◽

Upper Bound ◽

Content Distribution ◽

Upper Bounds ◽

Lower And Upper Bounds ◽

Fluid Queue ◽

Asymptotic Bounds ◽

Multiplicative Factor ◽

Buffer Content

Download Full-text

Excited state Rényi entropy and subsystem distance in two-dimensional non-compact bosonic theory. Part I. Single-particle states

Journal of High Energy Physics ◽

10.1007/jhep12(2020)160 ◽

2020 ◽

Vol 2020 (12) ◽

Author(s):

Jiaju Zhang ◽

M.A. Rajabpour

Keyword(s):

Excited States ◽

Single Particle ◽

Renyi Entropy ◽

Rényi Entropy ◽

Large Momentum ◽

Particle State ◽

Two Dimensional ◽

Universal Form ◽

Conformal Field ◽

Harmonic Chain

Abstract We investigate the Rényi entropy of the excited states produced by the current and its derivatives in the two-dimensional free massless non-compact bosonic theory, which is a two-dimensional conformal field theory. We also study the subsystem Schatten distance between these states. The two-dimensional free massless non-compact bosonic theory is the continuum limit of the finite periodic gapless harmonic chains with the local interactions. We identify the excited states produced by current and its derivatives in the massless bosonic theory as the single-particle excited states in the gapless harmonic chain. We calculate analytically the second Rényi entropy and the second Schatten distance in the massless bosonic theory. We then use the wave functions of the excited states and calculate the second Rényi entropy and the second Schatten distance in the gapless limit of the harmonic chain, which match perfectly with the analytical results in the massless bosonic theory. We verify that in the large momentum limit the single-particle state Rényi entropy takes a universal form. We also show that in the limit of large momenta and large momentum difference the subsystem Schatten distance takes a universal form but it is replaced by a new corrected form when the momentum difference is small. Finally we also comment on the mutual Rényi entropy of two disjoint intervals in the excited states of the two-dimensional free non-compact bosonic theory.

Download Full-text

Testing homogeneity of the galaxy distribution in the SDSS using Renyi entropy

Journal of Cosmology and Astroparticle Physics ◽

10.1088/1475-7516/2021/07/019 ◽

2021 ◽

Vol 2021 (07) ◽

pp. 019

Author(s):

Biswajit Pandey ◽

Suman Sarkar

Keyword(s):

Renyi Entropy ◽

Rényi Entropy ◽

Galaxy Distribution ◽

The Galaxy ◽

Testing Homogeneity

Download Full-text

Conditional Rényi Entropy and the Relationships between Rényi Capacities

Entropy ◽

10.3390/e22050526 ◽

2020 ◽

Vol 22 (5) ◽

pp. 526

Author(s):

Gautam Aishwarya ◽

Mokshay Madiman

Keyword(s):

Mutual Information ◽

Renyi Entropy ◽

Rényi Entropy ◽

Basic Properties ◽

Reference Measure ◽

Definition Of ◽

Discrete Setting

The analogues of Arimoto’s definition of conditional Rényi entropy and Rényi mutual information are explored for abstract alphabets. These quantities, although dependent on the reference measure, have some useful properties similar to those known in the discrete setting. In addition to laying out some such basic properties and the relations to Rényi divergences, the relationships between the families of mutual informations defined by Sibson, Augustin-Csiszár, and Lapidoth-Pfister, as well as the corresponding capacities, are explored.

Download Full-text