How a Brain Sees: Constructing Reality

The distinction between seeing and knowing, and why our brains even bother to see, are discussed using vivid perceptual examples, including image features without visible qualia that can nonetheless be consciously recognized, The work of Helmholtz and Kanizsa exemplify these issues, including examples of the paradoxical facts that “all boundaries are invisible”, and that brighter objects look closer. Why we do not see the big holes in, and occluders of, our retinas that block light from reaching our photoreceptors is explained, leading to the realization that essentially all percepts are visual illusions. Why they often look real is also explained. The computationally complementary properties of boundary completion and surface filling-in are introduced and their unifying explanatory power is illustrated, including that “all conscious qualia are surface percepts”. Neon color spreading provides a vivid example, as do self-luminous, glary, and glossy percepts. How brains embody general-purpose self-organizing architectures for solving modal problems, more general than AI algorithms, but less general than digital computers, is described. New concepts and mechanisms of such architectures are explained, including hierarchical resolution of uncertainty. Examples from the visual arts and technology are described to illustrate them, including paintings of Baer, Banksy, Bleckner, da Vinci, Gene Davis, Hawthorne, Hensche, Matisse, Monet, Olitski, Seurat, and Stella. Paintings by different artists and artistic schools instinctively emphasize some brain processes over others. These choices exemplify their artistic styles. The role of perspective, T-junctions, and end gaps are used to explain how 2D pictures can induce percepts of 3D scenes.

How a Brain Sees: Neural Mechanisms

Multiple paradoxical visual percepts are explained using boundary completion and surface filling-in properties, including discounting the illuminant; brightness constancy, contrast, and assimilation; the Craik-O’Brien-Cornsweet Effect; and Glass patterns. Boundaries act as both generators and barriers to filling-in using specific cooperative and competitive interactions. Oriented local contrast detectors, like cortical simple cells, create uncertainties that are resolved using networks of simple, complex, and hypercomplex cells, leading to unexpected insights such as why Roman typeface letter fonts use serifs. Further uncertainties are resolved by interactions with bipole grouping cells. These simple-complex-hypercomplex-bipole networks form a double filter and grouping network that provides unified explanations of texture segregation, hyperacuity, and illusory contour strength. Discounting the illuminant suppresses illumination contaminants so that feature contours can hierarchically induce surface filling-in. These three hierarchical resolutions of uncertainty explain neon color spreading. Why groupings do not penetrate occluding objects is explained, as are percepts of DaVinci stereopsis, the Koffka-Benussi and Kanizsa-Minguzzi rings, and pictures of graffiti artists and Mooney faces. The property of analog coherence is achieved by laminar neocortical circuits. Variations of a shared canonical laminar circuit have explained data about vision, speech, and cognition. The FACADE theory of 3D vision and figure-ground separation explains much more data than a Bayesian model can. The same cortical process that assures consistency of boundary and surface percepts, despite their complementary laws, also explains how figure-ground separation is triggered. It is also explained how cortical areas V2 and V4 regulate seeing and recognition without forcing all occluders to look transparent.

A detailed mathematical theory of thalamic and cortical microcircuits based on inference in a generative vision model

Understanding the information processing roles of cortical circuits is an outstanding problem in neuroscience and artificial intelligence. Theory-driven efforts will be required to tease apart the functional logic of cortical circuits from the vast amounts of experimental data on cortical connectivity and physiology. Although the theoretical setting of Bayesian inference has been suggested as a framework for understanding cortical computation, making precise and falsifiable biological mappings need models that tackle the challenge of real world tasks. Based on a recent generative model, Recursive Cortical Networks, that demonstrated excellent performance on visual task benchmarks, we derive a family of anatomically instantiated and functional cortical circuit models. Efficient inference and generalization guided the representational choices in the original computational model. The cortical circuit model is derived by systematically comparing the computational requirements of this model with known anatomical constraints. The derived model suggests precise functional roles for the feed-forward, feedback, and lateral connections observed in different laminae and columns, assigns a computational role for the path through the thalamus, predicts the interactions between blobs and inter-blobs, and offers an algorithmic explanation for the innate inter-laminar connectivity between clonal neurons within a cortical column. The model also explains several visual phenomena, including the subjective contour effect, and neon-color spreading effect, with circuit-level precision. Our work paves a new path forward in understanding the logic of cortical and thalamic circuits.

Explaining Visual Cortex Phenomena using Recursive Cortical Network

The connectivity and information pathways of visual cortex are well studied, as are observed physiological phenomena, yet a cohesive model for explaining visual cortex processes remains an open problem. For a comprehensive understanding, we need to build models of the visual cortex that are capable of robust real-world performance, while also being able to explain psychophysical and physiological observations. To this end, we demonstrate how the Recursive Cortical Network (George et al., 2017) can be used as a computational model to reproduce and explain subjective contours, neon color spreading, occlusion vs. deletion, and the border-ownership competition phenomena observed in the visual cortex.

Color Assimilation

Color assimilation, also known as the Von Bezold spreading effect, is the phenomenon in which the perceived color of a region shifts toward that of its neighbor. This chapter describes the traditional form of color assimilation as well as three “special cases” where the effects are particularly dramatic: the chromatic White’s Effect, Monnier and Shevell’s ring patterns, and neon-color spreading. Three potential causes of color assimilation are discussed: neural blurring, contrast normalization, and perceptual layer decomposition. All three of these could contribute to White’s Effect, and their relation to the other two cases are also discussed. Discussion on assimilation versus contrast and the effect of simulation contrast is included, and several figures are provided that illustrate the concepts.