To overcome the deficits of entropy as a measure for disorder when the number of states available to a system can change, Landsberg defined “disorder” as the entropy normalized to the maximum entropy. In the simplest cases, the maximum entropy is that of the equiprobable distribution, corresponding to a completely random system. However, depending on the question being asked and on system constraints, this absolute maximum entropy may not be the proper maximum entropy. To assess the effects of interactions on the “disorder” of a 1-dimensional spin system, the correct maximum entropy is that of the paramagnet (no interactions) with the same net magnetization; for a non-equilibrium system the proper maximum entropy may be that of the corresponding equilibrium system; and for hierarchical structures, an appropriate maximum entropy for a given level of the hierarchy is that of the system which is maximally random, subject to constraints deriving from the next lower level. Considerations of these examples leads us to introduce the “equivalent random system”: that system which is maximally random consistent with any constraints and with the question being asked. It is the entropy of the “equivalent random system” which should be taken as the maximum entropy in Landsberg's “disorder”.
An Introduction to the Non-Equilibrium Steady States of Maximum Entropy Spike Trains
