A Two-Stage Approach to Important Area Detection in Gathering Place Using a Novel Multi-Input Attention Network

An important area in a gathering place is a region attracting the constant attention of people and has evident visual features, such as a flexible stage or an open-air show. Finding such areas can help security supervisors locate the abnormal regions automatically. The existing related methods lack an efficient means to find important area candidates from a scene and have failed to judge whether or not a candidate attracts people’s attention. To realize the detection of an important area, this study proposes a two-stage method with a novel multi-input attention network (MAN). The first stage, called important area candidate generation, aims to generate candidate important areas with an image-processing algorithm (i.e., K-means++, image dilation, median filtering, and the RLSA algorithm). The candidate areas can be selected automatically for further analysis. The second stage, called important area candidate classification, aims to detect an important area from candidates with MAN. In particular, MAN is designed as a multi-input network structure, which fuses global and local image features to judge whether or not an area attracts people’s attention. To enhance the representation of candidate areas, two modules (i.e., channel attention and spatial attention modules) are proposed on the basis of the attention mechanism. These modules are mainly based on multi-layer perceptron and pooling operation to reconstruct the image feature and provide considerably efficient representation. This study also contributes to a new dataset called gathering place important area detection for testing the proposed two-stage method. Lastly, experimental results show that the proposed method has good performance and can correctly detect an important area.

Phase of neural oscillations as a reference frame for attention-based routing in visual cortex

Selective attention allows the brain to efficiently process the image projected onto the retina, selectively focusing neural processing resources on behaviorally relevant visual information. While previous studies have documented the crucial role of the action potential rate of single neurons in relaying such information, little is known about how the activity of single neurons relative to their neighboring network contributes to the efficient representation of attended stimuli and transmission of this information to downstream areas. Here, we show in the dorsal visual pathway of monkeys (medial superior temporal (MST) area) that neurons fire spikes preferentially at a specific phase of the ongoing population beta (~20 Hz) oscillations of the surrounding local network. This preferred spiking phase shifts towards a later phase when monkeys selectively attend towards (rather than away from) the receptive field of the neuron. This shift of the locking phase is positively correlated with the speed at which animals report a visual change. Furthermore, our computational modelling suggests that neural networks can manipulate the preferred phase of coupling by imposing differential synaptic delays on postsynaptic potentials. This distinction between the locking phase of neurons activated by the spatially attended stimulus vs. that of neurons activated by the unattended stimulus, may enable the neural system to discriminate relevant from irrelevant sensory inputs and consequently filter out distracting stimuli information by aligning the spikes which convey relevant/irrelevant information to distinct phases linked to periods of better/worse perceptual sensitivity for higher cortices. This strategy may be used to reserve the narrow windows of highest perceptual efficacy to the processing of the most behaviorally relevant information, ensuring highly efficient responses to attended sensory events.

Deep heterogeneous superpixel neural networks for image analysis and feature extraction

Lately, deep convolutional neural networks are rapidly transforming and enhancing computer vision accuracy and performance, and pursuing higher-level and interpretable object recognition. Superpixel-based methodologies have been used in conventional computer vision research where their efficient representation has superior effects. In contemporary computer vision research driven by deep neural networks, superpixel-based approaches mainly rely on oversegmentation to provide a more efficient representation of the imagery data, especially when the computation is too expensive in time or memory to perform in pairwise similarity regularization or complex graphical probabilistic inference. In this dissertation, we proposed a novel superpixel-enabled deep neural network paradigm by relaxing some of the prior assumptions in the conventional superpixel-based methodologies and exploring its capabilities in the context of advanced deep convolutional neural networks. This produces novel neural network architectures that can achieve higher-level object relation modeling, weakly supervised segmentation, high explainability, and facilitate insightful visualizations. This approach has the advantage of being an efficient representation of the visual signal and has the capability to dissect out relevant object components from other background noise by spatially re-organizing visual features. Specifically, we have created superpixel models that join graphical neural network techniques and multiple-instance learning to achieve weakly supervised object detection and generate precise object bounding without pixel-level training labels. This dissection and the subsequent learning by the architecture promotes explainable models, whereby the human users of the models can see the parts of the objects that have led to recognition. Most importantly, this neural design's natural result goes beyond abstract rectangular bounds of an object occurrence (e.g., bounding box or image chip), but instead approaches efficient parts-based segmented recognition. It has been tested on commercial remote sensing satellite imagery and achieved success. Additionally, We have developed highly efficient monocular indoor depth estimation based on superpixel feature extraction. Furthermore, we have demonstrated state-of-theart weakly supervised object detection performance on two contemporary benchmark data sets, MS-COCO and VOC 2012. In the future, deep learning techniques based on superpixel-enabled image analysis can be further optimized in accuracy and computational performance; and it will also be interesting to evaluate in other research domains, such as those involving medical imagery, infrared imagery, or hyperspectral imagery.