CNN is an acronym for either Cellular Neural Network when used in the context of brain science, or Cellular Nonlinear Network when used in the context of coupled dynamical systems. A CNN is defined by two mathematical constructs: 1. A spatially discrete collection of continuous nonlinear dynamical systems called cells, where information can be encrypted into each cell via three independent variables called input, threshold, and initial state. 2. A coupling law relating one or more relevant variables of each cell Cij to all neighbor cells Ckl located within a prescribed sphere of influence Sij(r) of radius r, centered at Cij. In the special case where the CNN consists of a homogeneous array, and where its cells have no inputs, no thresholds, and no outputs, and where the sphere of influence extends only to the nearest neighbors (i.e. r = 1), the CNN reduces to the familiar concept of a nonlinear lattice. The bulk of this three-part exposition is devoted to the standard CNN equation [Formula: see text] where xij, yij, uij and zij are scalars called state, output, input, and threshold of cell Cij; akl and bkl are scalars called synaptic weights, and Sij(r) is the sphere of influence of radius r. In the special case where r = 1, a standard CNN is uniquely defined by a string of "19" real numbers (a uniform thresholdzkl = z, nine feedback synaptic weights akl, and nine control synaptic weights bkl) called a CNN gene because it completely determines the properties of the CNN. The universe of all CNN genes is called the CNN genome. Many applications from image processing, pattern recognition, and brain science can be easily implemented by a CNN "program" defined by a string of CNN genes called a CNN chromosome. The first new result presented in this exposition asserts that every Boolean function of the neighboring-cell inputs can be explicitly synthesized by a CNN chromosome. This general theorem implies that every cellular automata (with binary states) is a CNN chromosome. In particular, a constructive proof is given which shows that the game-of-life cellular automata can be realized by a CNN chromosome made of only three CNN genes. Consequently, this "game-of-life" CNN chromosome is a universal Turing machine, and is capable of self-replication in the Von Neumann sense [Berlekamp et al., 1982]. One of the new concepts presented in this exposition is that of a generalized cellular automata (GCA), which is outside the framework of classic cellular (Von Neumann) automata because it cannot be defined by local rules: It is simply defined by iterating a CNN gene, or chromosome, in a "CNN DO LOOP". This new class of generalized cellular automata includes not only global Boolean maps, but also continuum-state cellular automata where the initial state configuration and its iterates are real numbers, not just a finite number of states as in classical (von Neumann) cellular automata. Another new result reported in this exposition is the successful implementation of an analog input analog output CNN universal machine, called a CNN universal chip, on a single silicon chip. This chip is a complete dynamic array stored-program computer where a CNN chromosome (i.e. a CNN algorithm or flow chart) can be programmed and executed on the chip at an extremely high speed of 1 Tera (1012) analog instructions per second (based on a 100 × 100 chip). The CNN universal chip is based entirely on nonlinear dynamics and therefore differs from a digital computer in its fundamental operating principles. Part II of this exposition is devoted to the important subclass of autonomous CNNs where the cells have no inputs. This class of CNNs can exhibit a great variety of complex phenomena, including pattern formation, Turing patterns, knots, auto waves, spiral waves, scroll waves, and spatiotemporal chaos. It provides a unified paradigm for complexity, as well as an alternative paradigm for simulating nonlinear partial differential equations (PDE's). In this context, rather than regarding the autonomous CNN as an approximation of nonlinear PDE's, we advocate the more provocative point of view that nonlinear PDE's are merely idealizations of CNNs, because while nonlinear PDE's can be regarded as a limiting form of autonomous CNNs, only a small class of CNNs has a limiting PDE representation. Part III of this exposition is rather short but no less significant. It contains in fact the potentially most important original results of this exposition. In particular, it asserts that all of the phenomena described in the complexity literature under various names and headings (e.g. synergetics, dissipative structures, self-organization, cooperative and competitive phenomena, far-from-thermodynamic equilibrium phenomena, edge of chaos, etc.) are merely qualitative manifestations of a more fundamental and quantitative principle called the local activity dogma. It is quantitative in the sense that it not only has a precise definition but can also be explicitly tested by computing whether a certain explicitly defined expression derived from the CNN paradigm can assume a negative value or not. Stated in words, the local activity dogma asserts that in order for a system or model to exhibit any form of complexity, such as those cited above, the associated CNN parameters must be chosen so that either the cells or their couplings are locally active.
A Dynamical Systems-Based Hierarchy for Shannon, Metric and Topological Entropy
