scholarly journals Evaluation of Advanced Routing Strategies with Information-Theoretic Complexity Measures

Author(s):  
Michele Amoretti ◽  
Stefano Cagnoni
2008 ◽  
Vol 18 (05) ◽  
pp. 389-403 ◽  
Author(s):  
THOMAS D. JORGENSEN ◽  
BARRY P. HAYNES ◽  
CHARLOTTE C. F. NORLUND

This paper describes a new method for pruning artificial neural networks, using a measure of the neural complexity of the neural network. This measure is used to determine the connections that should be pruned. The measure computes the information-theoretic complexity of a neural network, which is similar to, yet different from previous research on pruning. The method proposed here shows how overly large and complex networks can be reduced in size, whilst retaining learnt behaviour and fitness. The technique proposed here helps to discover a network topology that matches the complexity of the problem it is meant to solve. This novel pruning technique is tested in a robot control domain, simulating a racecar. It is shown, that the proposed pruning method is a significant improvement over the most commonly used pruning method Magnitude Based Pruning. Furthermore, some of the pruned networks prove to be faster learners than the benchmark network that they originate from. This means that this pruning method can also help to unleash hidden potential in a network, because the learning time decreases substantially for a pruned a network, due to the reduction of dimensionality of the network.


Complexity ◽  
2017 ◽  
Vol 2017 ◽  
pp. 1-12 ◽  
Author(s):  
Mikołaj Morzy ◽  
Tomasz Kajdanowicz ◽  
Przemysław Kazienko

One of the most popular methods of estimating the complexity of networks is to measure the entropy of network invariants, such as adjacency matrices or degree sequences. Unfortunately, entropy and all entropy-based information-theoretic measures have several vulnerabilities. These measures neither are independent of a particular representation of the network nor can capture the properties of the generative process, which produces the network. Instead, we advocate the use of the algorithmic entropy as the basis for complexity definition for networks. Algorithmic entropy (also known as Kolmogorov complexity or K-complexity for short) evaluates the complexity of the description required for a lossless recreation of the network. This measure is not affected by a particular choice of network features and it does not depend on the method of network representation. We perform experiments on Shannon entropy and K-complexity for gradually evolving networks. The results of these experiments point to K-complexity as the more robust and reliable measure of network complexity. The original contribution of the paper includes the introduction of several new entropy-deceiving networks and the empirical comparison of entropy and K-complexity as fundamental quantities for constructing complexity measures for networks.


2011 ◽  
Vol 11 (7&8) ◽  
pp. 649-676
Author(s):  
Julien Degorre ◽  
Marc Kaplan ◽  
Sophie Laplante ◽  
J\'er\'emie Roland

We study a model of communication complexity that encompasses many well-studied problems, including classical and quantum communication complexity, the complexity of simulating distributions arising from bipartite measurements of shared quantum states, and XOR games. In this model, Alice gets an input $x$, Bob gets an input $y$, and their goal is to each produce an output $a,b$ distributed according to some pre-specified joint distribution $p(a,b|x,y)$. Our results apply to any non-signaling distribution, that is, those where Alice's marginal distribution does not depend on Bob's input, and vice versa.~~~By taking a geometric view of the non-signaling distributions, we introduce a simple new technique based on affine combinations of lower-complexity distributions, and we give the first general technique to apply to all these settings, with elementary proofs and very intuitive interpretations. Specifically, we introduce two complexity measures, one which gives lower bounds on classical communication, and one for quantum communication. These measures can be expressed as convex optimization problems. We show that the dual formulations have a striking interpretation, since they coincide with maximum violations of Bell and Tsirelson inequalities. The dual expressions are closely related to the winning probability of XOR games. Despite their apparent simplicity, these lower bounds subsume many known communication complexity lower bound methods, most notably the recent lower bounds of Linial and Shraibman for the special case of Boolean functions. We show that as in the case of Boolean functions, the gap between the quantum and classical lower bounds is at most linear in the size of the support of the distribution, and does not depend on the size of the inputs. This translates into a bound on the gap between maximal Bell and Tsirelson inequality violations, which was previously known only for the case of distributions with Boolean outcomes and uniform marginals. It also allows us to show that for some distributions, information theoretic methods are necessary to prove strong lower bounds. ~~~ Finally, we give an exponential upper bound on quantum and classical communication complexity in the simultaneous messages model, for any non-signaling distribution. One consequence of this is a simple proof that any quantum distribution can be approximated with a constant number of bits of communication.


PLoS ONE ◽  
2009 ◽  
Vol 4 (12) ◽  
pp. e8057 ◽  
Author(s):  
Matthias Dehmer ◽  
Nicola Barbarini ◽  
Kurt Varmuza ◽  
Armin Graber

Entropy ◽  
2021 ◽  
Vol 23 (11) ◽  
pp. 1488
Author(s):  
Angel Ricardo Plastino ◽  
Diana Monteoliva ◽  
Angelo Plastino

Finite quantum many fermion systems are essential for our current understanding of Nature. They are at the core of molecular, atomic, and nuclear physics. In recent years, the application of information and complexity measures to the study of diverse types of many-fermion systems has opened a line of research that elucidates new aspects of the structure and behavior of this class of physical systems. In this work we explore the main features of information and information-based complexity indicators in exactly soluble many-fermion models of the Lipkin kind. Models of this kind have been extremely useful in shedding light on the intricacies of quantum many body physics. Models of the Lipkin kind play, for finite systems, a role similar to the one played by the celebrated Hubbard model of solid state physics. We consider two many fermion systems and show how their differences can be best appreciated by recourse to information theoretic tools. We appeal to information measures as tools to compare the structural details of different fermion systems. We will discover that few fermion systems are endowed by a much larger complexity-degree than many fermion ones. The same happens with the coupling-constants strengths. Complexity augments as they decrease, without reaching zero. Also, the behavior of the two lowest lying energy states are crucial in evaluating the system’s complexity.


Sign in / Sign up

Export Citation Format

Share Document