scholarly journals Renyi entropy driven hierarchical graph clustering

2021 ◽  
Vol 7 ◽  
pp. e366
Author(s):  
Frédérique Oggier ◽  
Anwitaman Datta

This article explores a graph clustering method that is derived from an information theoretic method that clusters points in ${{\mathbb{R}}^{n}}$ relying on Renyi entropy, which involves computing the usual Euclidean distance between these points. Two view points are adopted: (1) the graph to be clustered is first embedded into ${\mathbb{R}}^{d}$ for some dimension d so as to minimize the distortion of the embedding, then the resulting points are clustered, and (2) the graph is clustered directly, using as distance the shortest path distance for undirected graphs, and a variation of the Jaccard distance for directed graphs. In both cases, a hierarchical approach is adopted, where both the initial clustering and the agglomeration steps are computed using Renyi entropy derived evaluation functions. Numerical examples are provided to support the study, showing the consistency of both approaches (evaluated in terms of F-scores).


Atoms ◽  
2019 ◽  
Vol 7 (3) ◽  
pp. 70 ◽  
Author(s):  
Jen-Hao Ou ◽  
Yew Kam Ho

Knowledge of the electronic structures of atomic and molecular systems deepens our understanding of the desired system. In particular, several information-theoretic quantities, such as Shannon entropy, have been applied to quantify the extent of electron delocalization for the ground state of various systems. To explore excited states, we calculated Shannon entropy and two of its one-parameter generalizations, Rényi entropy of order α and Tsallis entropy of order α , and Onicescu Information Energy of order α for four low-lying singly excited states (1s2s 1 S e , 1s2s 3 S e , 1s3s 1 S e , and 1s3s 3 S e states) of helium. This paper compares the behavior of these three quantities of order 0.5 to 9 for the ground and four excited states. We found that, generally, a higher excited state had a larger Rényi entropy, larger Tsallis entropy, and smaller Onicescu information energy. However, this trend was not definite and the singlet–triplet reversal occurred for Rényi entropy, Tsallis entropy and Onicescu information energy at a certain range of order α .



Author(s):  
Moonyoung Chung ◽  
Woong-Kee Loh

AbstractIn spatial database and road network applications, the search for the nearest neighbor (NN) from a given query object q is the most fundamental and important problem. Aggregate nearest neighbor (ANN) search is an extension of the NN search with a set of query objects $$Q = \{ q_0, \dots , q_{M-1} \}$$ Q = { q 0 , ⋯ , q M - 1 } and finds the object $$p^*$$ p ∗ that minimizes $$g \{ d(p^*, q_i), q_i \in Q \}$$ g { d ( p ∗ , q i ) , q i ∈ Q } , where g (max or sum) is an aggregate function and d() is a distance function between two objects. Flexible aggregate nearest neighbor (FANN) search is an extension of the ANN search with the introduction of a flexibility factor $$\phi \, (0 < \phi \le 1)$$ ϕ ( 0 < ϕ ≤ 1 ) and finds the object $$p^*$$ p ∗ and the set of query objects $$Q^*_\phi $$ Q ϕ ∗ that minimize $$g \{ d(p^*, q_i), q_i \in Q^*_\phi \}$$ g { d ( p ∗ , q i ) , q i ∈ Q ϕ ∗ } , where $$Q^*_\phi $$ Q ϕ ∗ can be any subset of Q of size $$\phi |Q|$$ ϕ | Q | . This study proposes an efficient $$\alpha $$ α -probabilistic FANN search algorithm in road networks. The state-of-the-art FANN search algorithm in road networks, which is known as IER-$$k\hbox {NN}$$ k NN , used the Euclidean distance based on the two-dimensional coordinates of objects when choosing an R-tree node that most potentially contains $$p^*$$ p ∗ . However, since the Euclidean distance is significantly different from the actual shortest-path distance between objects, IER-$$k\hbox {NN}$$ k NN looks up many unnecessary nodes, thereby incurring many calculations of ‘expensive’ shortest-path distances and eventually performance degradation. The proposed algorithm transforms road network objects into k-dimensional Euclidean space objects while preserving the distances between them as much as possible using landmark multidimensional scaling (LMDS). Since the Euclidean distance after LMDS transformation is very close to the shortest-path distance, the lookup of unnecessary R-tree nodes and the calculation of expensive shortest-path distances are reduced significantly, thereby greatly improving the search performance. As a result of performance comparison experiments conducted for various real road networks and parameters, the proposed algorithm always achieved higher performance than IER-$$k\hbox {NN}$$ k NN ; the performance (execution time) of the proposed algorithm was improved by up to 10.87 times without loss of accuracy.



Mathematics ◽  
2021 ◽  
Vol 9 (2) ◽  
pp. 146
Author(s):  
Salah Abid ◽  
Uday Quaez ◽  
Javier Contreras-Reyes

Shannon and Rényi entropies are two important measures of uncertainty for data analysis. These entropies have been studied for multivariate Student-t and skew-normal distributions. In this paper, we extend the Rényi entropy to multivariate skew-t and finite mixture of multivariate skew-t (FMST) distributions. This class of flexible distributions allows handling asymmetry and tail weight behavior simultaneously. We find upper and lower bounds of Rényi entropy for these families. Numerical simulations illustrate the results for several scenarios: symmetry/asymmetry and light/heavy-tails. Finally, we present applications of our findings to a swordfish length-weight dataset to illustrate the behavior of entropies of the FMST distribution. Comparisons with the counterparts—the finite mixture of multivariate skew-normal and normal distributions—are also presented.



Fractals ◽  
1996 ◽  
Vol 04 (03) ◽  
pp. 355-367 ◽  
Author(s):  
A. BUNDE ◽  
J. DRÄGER ◽  
S. HAVLIN ◽  
H. E. ROMAN ◽  
St. RUSS

We review analytical and numerical results for the vibrational amplitudes of localized excitations, the probability distribution of random walks and the distribution of linear polymers (modeled by self-avoiding walks of N steps) on percolation structures at criticality. Our numerical results show that the fluctuations of these quantities, at fixed shortest-path distance (“chemical length”) l from the center of localization, are considerably smaller than at fixed Euclidean distance r from the center. Using this fact, we derive via convolutional integrals explicit expressions for the averaged functions in r-space, and show analytically and numerically that three different localization regimes occur. In the short-distance regime, remarkably, the averages show a universal spatial decay behavior, with the same exponent for both fractons and random walks, while in the asymptotic regime, the averages depend explicitly on the number of configurations considered.



2019 ◽  
Vol 28 (07) ◽  
pp. 1950091 ◽  
Author(s):  
Clifford V. Johnson

We present a new type of generalization of the Rényi entropy that follows naturally from its representation as a thermodynamic quantity. We apply it to the case of [Formula: see text]-dimensional conformal field theories (CFTs) reduced on a region bounded by a sphere. It is known how to compute their Rényi entropy as an integral of the thermal entropy of hyperbolic black holes in [Formula: see text]-dimensional anti-de Sitter spacetime. We show how this integral fits into the framework of extended gravitational thermodynamics, and then point out the natural generalization of the Rényi entropy that suggests itself in that light. In the field theory terms, the new generalization employs aspects of the physics of Renormalization Group (RG) flow to define a refined version of the reduced vacuum density matrix. For [Formula: see text], it can be derived directly in terms of twist operators in field theory. The framework presented here may have applications beyond this context, perhaps in studies of both quantum and classical information theoretic properties of a variety of systems.



2010 ◽  
Vol 20 (01) ◽  
pp. 69-87
Author(s):  
HEE-KAP AHN ◽  
MOHAMMAD FARSHI ◽  
CHRISTIAN KNAUER ◽  
MICHIEL SMID ◽  
YAJUN WANG

Consider a geometric network G in the plane. The dilation between any two vertices x and y in G is the ratio of the shortest path distance between x and y in G to the Euclidean distance between them. The maximum dilation over all pairs of vertices in G is called the dilation of G. In this paper, a randomized algorithm is presented which, when given a polygonal cycle C on n vertices in the plane, computes in O(n log 3 n) expected time, the edge of C whose removal results in a polygonal path of smallest possible dilation. It is also shown that the edge whose removal gives a polygonal path of largest possible dilation can be computed in O(n log n) time. If C is a convex polygon, the running time for the latter problem becomes O(n). Finally, it is shown that a (1 - ϵ)-approximation to the dilation of every path C \{e}, for all edges e of C, can be computed in O(n log n) total time.



Author(s):  
Cecilia Viscardi ◽  
Michele Boreale ◽  
Fabio Corradi

AbstractWe consider the problem of sample degeneracy in Approximate Bayesian Computation. It arises when proposed values of the parameters, once given as input to the generative model, rarely lead to simulations resembling the observed data and are hence discarded. Such “poor” parameter proposals do not contribute at all to the representation of the parameter’s posterior distribution. This leads to a very large number of required simulations and/or a waste of computational resources, as well as to distortions in the computed posterior distribution. To mitigate this problem, we propose an algorithm, referred to as the Large Deviations Weighted Approximate Bayesian Computation algorithm, where, via Sanov’s Theorem, strictly positive weights are computed for all proposed parameters, thus avoiding the rejection step altogether. In order to derive a computable asymptotic approximation from Sanov’s result, we adopt the information theoretic “method of types” formulation of the method of Large Deviations, thus restricting our attention to models for i.i.d. discrete random variables. Finally, we experimentally evaluate our method through a proof-of-concept implementation.



2020 ◽  
Vol 2020 (12) ◽  
Author(s):  
Jiaju Zhang ◽  
M.A. Rajabpour

Abstract We investigate the Rényi entropy of the excited states produced by the current and its derivatives in the two-dimensional free massless non-compact bosonic theory, which is a two-dimensional conformal field theory. We also study the subsystem Schatten distance between these states. The two-dimensional free massless non-compact bosonic theory is the continuum limit of the finite periodic gapless harmonic chains with the local interactions. We identify the excited states produced by current and its derivatives in the massless bosonic theory as the single-particle excited states in the gapless harmonic chain. We calculate analytically the second Rényi entropy and the second Schatten distance in the massless bosonic theory. We then use the wave functions of the excited states and calculate the second Rényi entropy and the second Schatten distance in the gapless limit of the harmonic chain, which match perfectly with the analytical results in the massless bosonic theory. We verify that in the large momentum limit the single-particle state Rényi entropy takes a universal form. We also show that in the limit of large momenta and large momentum difference the subsystem Schatten distance takes a universal form but it is replaced by a new corrected form when the momentum difference is small. Finally we also comment on the mutual Rényi entropy of two disjoint intervals in the excited states of the two-dimensional free non-compact bosonic theory.





Entropy ◽  
2020 ◽  
Vol 22 (5) ◽  
pp. 526
Author(s):  
Gautam Aishwarya ◽  
Mokshay Madiman

The analogues of Arimoto’s definition of conditional Rényi entropy and Rényi mutual information are explored for abstract alphabets. These quantities, although dependent on the reference measure, have some useful properties similar to those known in the discrete setting. In addition to laying out some such basic properties and the relations to Rényi divergences, the relationships between the families of mutual informations defined by Sibson, Augustin-Csiszár, and Lapidoth-Pfister, as well as the corresponding capacities, are explored.



Sign in / Sign up

Export Citation Format

Share Document