Ultra-short periods of wakeful rest promote the viewpoint-invariance of entities mnemonic representations

Humans can recognize thousands of visual objects after a single exposure, even against highly confusable objects, and despite viewpoint changes between learning and recognition. Memory consolidation processes like those taking place during wakeful rest contribute to such a feat, possibly by protecting the fine details of objects’ representations. However, whether rest-related consolidation promotes the viewpoint invariance of mnemonic representations for individual objects remains unexplored.Fifteen participants underwent a speeded visual recognition memory task tapping on familiarity-based recognition of individual objects, across four conditions manipulating post- encoding rest. Viewpoints of target items were modified between study and test while controlling study-test perceptual distance, and targets and lures shared the same subordinate category, making recognition independent from perceptual and conceptual fluency. Performance was very accurate, even without post-encoding rest, which did not enhance memory. However, rest uniquely made target detection immune to study-test perceptual distance.These findings suggest that very short periods of wakeful rest (down to 2-sec post-stimulus) suffice to achieve complete mnemonic viewpoint-invariance, pushing forward the strength of post-encoding rest in learning and memory. They also strongly argue for a holistic, viewpoint- invariant, mnemonic representation of visual objects.

3D Affine: An Embedding of Local Image Features for Viewpoint Invariance Using RGB-D Sensor Data

Local image features are invariant to in-plane rotations and robust to minor viewpoint changes. However, the current detectors and descriptors for local image features fail to accommodate out-of-plane rotations larger than 25°–30°. Invariance to such viewpoint changes is essential for numerous applications, including wide baseline matching, 6D pose estimation, and object reconstruction. In this study, we present a general embedding that wraps a detector/descriptor pair in order to increase viewpoint invariance by exploiting input depth maps. The proposed embedding locates smooth surfaces within the input RGB-D images and projects them into a viewpoint invariant representation, enabling the detection and description of more viewpoint invariant features. Our embedding can be utilized with different combinations of descriptor/detector pairs, according to the desired application. Using synthetic and real-world objects, we evaluated the viewpoint invariance of various detectors and descriptors, for both standalone and embedded approaches. While standalone local image features fail to accommodate average viewpoint changes beyond 33.3°, our proposed embedding boosted the viewpoint invariance to different levels, depending on the scene geometry. Objects with distinct surface discontinuities were on average invariant up to 52.8°, and the overall average for all evaluated datasets was 45.4°. Similarly, out of a total of 140 combinations involving 20 local image features and various objects with distinct surface discontinuities, only a single standalone local image feature exceeded the goal of 60° viewpoint difference in just two combinations, as compared with 19 different local image features succeeding in 73 combinations when wrapped in the proposed embedding. Furthermore, the proposed approach operates robustly in the presence of input depth noise, even that of low-cost commodity depth sensors, and well beyond.

Effect of silhouetting and inversion on view invariance in the monkey inferotemporal cortex

We effortlessly recognize objects across changes in viewpoint, but we know relatively little about the features that underlie viewpoint invariance in the brain. Here, we set out to characterize how viewpoint invariance in monkey inferior temporal (IT) neurons is influenced by two image manipulations—silhouetting and inversion. Reducing an object into its silhouette removes internal detail, so this would reveal how much viewpoint invariance depends on the external contours. Inverting an object retains but rearranges features, so this would reveal how much viewpoint invariance depends on the arrangement and orientation of features. Our main findings are 1) view invariance is weakened by silhouetting but not by inversion; 2) view invariance was stronger in neurons that generalized across silhouetting and inversion; 3) neuronal responses to natural objects matched early with that of silhouettes and only later to that of inverted objects, indicative of coarse-to-fine processing; and 4) the impact of silhouetting and inversion depended on object structure. Taken together, our results elucidate the underlying features and dynamics of view-invariant object representations in the brain. NEW & NOTEWORTHY We easily recognize objects across changes in viewpoint, but the underlying features are unknown. Here, we show that view invariance in the monkey inferotemporal cortex is driven mainly by external object contours and is not specialized for object orientation. We also find that the responses to natural objects match with that of their silhouettes early in the response, and with inverted versions later in the response—indicative of a coarse-to-fine processing sequence in the brain.

Good Appearance and 3D Shape Descriptors for Object Category Recognition

For the problem of object category recognition, we have studied different families of descriptors exploiting RGB and 3D information. We have proven practically that 3D shape-based descriptors are more suitable for this type of recognition due to low shape intra-class variance, as opposed to texture-based. Performance evaluation on training-set subsampling, suggests that the viewpoint invariance characteristics of 3D descriptors, favors significantly these descriptors while invariant SIFT descriptors can be ambiguous. In addition, we have also shown how an efficient Naive Bayes Nearest Neighbor (NBNN) classifier can scale to a large hierarchical RGB-D Object Dataset and achieve, with a single descriptor type, an accuracy close to state-of-the-art learning-based approaches using combined descriptors.