Implementation of Biologically Inspired Components in Embedded Vision Systems

Studies in the area of computational vision have shown the capability of visual attention (VA) processing in aiding various visual tasks by providing a means for simplifying complex data handling and supporting action decisions using readily available low-level features. Due to the inclusion of computational biological vision components to mimic the mechanism of the human visual system, VA processing is computationally complex with heavy memory requirements and is often found implemented in workstations with unapplied resource constraints. In embedded systems, the computational capacity and memory resources are of a primary concern. To allow VA processing in such systems, the chapter presents a low complexity, low memory VA model based on an established mainstream VA model that addresses critical factors in terms of algorithm complexity, memory requirements, computational speed, and salience prediction performance to ensure the reliability of the VA processing in an environment with limited resources. Lastly, a custom softcore microprocessor-based hardware implementation on a Field-Programmable Gate Array (FPGA) is used to verify the implementation feasibility of the presented low complexity, low memory VA model.

Influence of Movement Expertise on Visual Perception of Objects, Events and Motor Action

Various computational models of visual attention rely on the extraction of salient points or proto-objects, i.e., discrete units of attention, computed from bottom-up image features. In recent years, different solutions integrating top-down mechanisms were implemented, as research has shown that although eye movements initially are solely influenced by bottom-up information, after some time goal driven (high-level) processes dominate the guidance of visual attention towards regions of interest (Hwang, Higgins & Pomplun, 2009). However, even these improved modeling approaches are unlikely to generalize to a broader range of application contexts, because basic principles of visual attention, such as cognitive control, learning and expertise, have thus far not sufficiently been taken into account (Tatler, Hayhoe, Land & Ballard, 2011). In some recent work, the authors showed the functional role and representational nature of long-term memory structures for human perceptual skills and motor control. Based on these findings, the chapter extends a widely applied saliency-based model of visual attention (Walther & Koch, 2006) in two ways: first, it computes the saliency map using the cognitive visual attention approach (CVA) that shows a correspondence between regions of high saliency values and regions of visual interest indicated by participants’ eye movements (Oyekoya & Stentiford, 2004). Second, it adds an expertise-based component (Schack, 2012) to represent the influence of the quality of mental representation structures in long-term memory (LTM) and the roles of learning on the visual perception of objects, events, and motor actions.

Visual Behavior Based Bio-Inspired Polarization Techniques in Computer Vision and Robotics

For long time, it was thought that the sensing of polarization by animals is invariably related to their behavior, such as navigation and orientation. Recently, it was found that polarization can be part of a high-level visual perception, permitting a wide area of vision applications. Polarization vision can be used for most tasks of color vision including object recognition, contrast enhancement, camouflage breaking, and signal detection and discrimination. The polarization based visual behavior found in the animal kingdom is briefly covered. Then, the authors go in depth with the bio-inspired applications based on polarization in computer vision and robotics. The aim is to have a comprehensive survey highlighting the key principles of polarization based techniques and how they are biologically inspired.

Modeling Binocular and Motion Transparency Processing by Local Center-Surround Interactions

Binocular transparency is perceived if two surfaces are seen in the same spatial location, but at different depths. Similarly, motion transparency occurs if two surfaces move differently over the same spatial location. Most models of motion or stereo processing incorporate uniqueness assumptions to resolve ambiguities of disparity or motion estimates and, thus, can not represent multiple features at the same spatial location. Unlike these previous models, the authors of this chapter suggest a model with local center-surround interaction that operates upon analogs of cell populations in velocity or disparity domain of the ventral second visual area (V2) and dorsal medial middle temporal area (MT) in primates, respectively. These modeled cell populations can encode motion and binocular transparency. Model simulations demonstrate the successful processing of scenes with opaque and transparent materials, not previously reported. Results suggest that motion and stereo processing both employ local center-surround interactions to resolve noisy and ambiguous disparity or motion input from initial correlations.

Replicating the Role of the Human Retina for a Cortical Visual Neuroprosthesis

Neuroengineering is an emerging research field combining the latest findings from neuroscience with developments in a variety of engineering disciplines to create artificial devices, mainly for therapeutical purposes. In this chapter, an application of this field to the development of a visual neuroprosthesis for the blind is described. Electrical stimulation of the visual cortex in blind subjects elicits the perception of visual sensations called phosphenes, a finding that encourages the development of future electronic visual prostheses. However, direct stimulation of the visual cortex would miss a significant degree of image processing that is carried out by the retina. The authors describe a biologically-inspired retina-like processor designed to drive the implanted stimulator using visual inputs from one or two cameras. This includes dynamic response modeling with minimal latency. The outputs of the retina-like processor are comparable to those recorded in biological retinas that are exposed to the same stimuli and allow estimation of the original scene

Attention in Stereo Vision

The stereo correspondence problem is a topic that has been the subject of considerable research effort. What has not yet been considered is an analogue of stereo correspondence in the domain of attention. In this chapter, the authors bring this problem to light, revealing important implications for computational models of attention, and in particular, how these implications constrain the problem of computational modeling of attention. A model is described which addresses attention in the stereo domain, and it is revealed that a variety of behaviors observed in binocular rivalry experiments are consistent with the model’s behavior. Finally, the authors consider how constraints imposed by stereo vision may suggest analogous constraints in other non-stereo feature domains with significant consequence to computational models of attention.

Mathematical Foundations Modeled after Neo-Cortex for Discovery and Understanding of Structures in Data

Even though there are distinct areas for different functionalities in the mammalian neo-cortex, it seems to use the same algorithm to understand a large variety of input modalities. In addition, it appears that the neo-cortex effortlessly identifies the correlation among many sensor modalities and fuses information obtained from them. The question then is, can we discover the brain’s learning algorithm and approximate it for problems such as computer vision and automatic speech recognition that the mammalian brain is so good at? The answer is: it is an orders of magnitude problem, i.e., not a simple task. However, we can attempt to develop mathematical foundations based on the understanding of how a human brain learns. This chapter is focused along that direction. In particular, it is focused on the ventral stream – the “what pathway” - and describes common algorithms that can be used for representation and classification of signals from different sensor modalities such as auditory and visual. These common algorithms are based on dictionary learning with a beta process, hierarchical graphical models, and embedded hidden Markov models.

A Measure of Localization of Brain Activity for the Motion Aperture Problem Using Electroencephalograms

When viewed through a limited-sized aperture, bars appear to move in a direction normal to their orientation. This motion aperture problem is an important rubric for analyzing the early stages of visual processing particularly with respect to the perceptual completion of motion sampled across two or more apertures. In the present study, a circular aperture was displayed in the center of the visual field. While the baseline bar moved within the aperture, two additional circular apertures appeared; within each aperture, a “flanker bar” appeared to move. For upwards movement of the flanker lines, subjects perceived the flanker bar to be connected to the base bar, and all three parts to move upward. The authors investigated the motion perception of the moving bars by changing the line speeds, radii of the apertures, and distances between the circular apertures and then analyzed spatio-temporal brain activities by electroencephalograms (EEGs). Latencies in the brain were estimated by using equivalent current dipole source (ECD) localization for one subject. Soon after the flankers appear, ECDs, assumed to be generated by the recognition of the aperture’s form, were localized along the ventral pathway. After the bars moved, the ECDs were localized along the dorsal pathway, presumably in response to motion of the bars. In addition, for the perception of grouped motion and not normal motion, ECDs were localized to the middle frontal gyrus and the inferior frontal gyrus.

The Roles of Endstopped and Curvature Tuned Computations in a Hierarchical Representation of 2D Shape

Computational models of visual processes are of interest in fields such as cybernetics, robotics, computer vision, and others. This chapter argues for the importance of intermediate representation layers in the visual cortex that have direct impact on the next generation of object recognition strategies in computer vision. Biological inspiration - and even biological realism - is currently of great interest in the computer vision community. The authors propose that endstopping and curvature cells are of great importance for shape selectivity and show how their combination can lead to shape selective neurons, providing an approach that does not require learning between early stages based on Gabor or Difference of Gaussian filters and later stages closer to object representations.

Local Constraints for the Perception of Binocular 3D Motion

The authors discuss local constraints for the perception of three-dimensional (3D) binocular motion in a geometric-probabilistic framework. It is shown that Bayesian models of binocular 3D motion can explain perceptual bias under uncertainty and predict perceived velocity under ambiguity. The models exploit biologically plausible constraints of local motion and disparity processing in a binocular viewing geometry. Results from computer simulations and psychophysical experiments support the idea that local constraints of motion and disparity processing are combined late in the visual processing hierarchy to establish perceived 3D motion direction.