GourmetNet: Food Segmentation Using Multi-Scale Waterfall Features with Spatial and Channel Attention

We propose GourmetNet, a single-pass, end-to-end trainable network for food segmentation that achieves state-of-the-art performance. Food segmentation is an important problem as the first step for nutrition monitoring, food volume and calorie estimation. Our novel architecture incorporates both channel attention and spatial attention information in an expanded multi-scale feature representation using our advanced Waterfall Atrous Spatial Pooling module. GourmetNet refines the feature extraction process by merging features from multiple levels of the backbone through the two attention modules. The refined features are processed with the advanced multi-scale waterfall module that combines the benefits of cascade filtering and pyramid representations without requiring a separate decoder or post-processing. Our experiments on two food datasets show that GourmetNet significantly outperforms existing current state-of-the-art methods.

Impact of microsaccades on visual shape processing

Sensitivity to subtle changes in the shape of visual objects has been attributed to the existence of global pooling mechanisms that integrate local form information across space. While global pooling is typically demonstrated under steady fixation, other work suggests prolonged fixation can lead to a collapse of global structure. Here we ask whether small ballistic eye movements that naturally occur during periods of fixation affect the global processing of radial frequency (RF) patterns - closed contours created by sinusoidally modulating the radius of a circle. Observers were asked to discriminate the shapes of circular and RF modulated patterns while fixational eye movements were recorded binocularly at 500Hz. Microsaccades were detected using a velocity-based algorithm, allowing trials to be sorted according to the relative timing of stimulus and microsaccade onset. Results revealed clear peri-saccadic changes in shape discrimination thresholds. Performance was impaired when microsaccades occurred close to stimulus onset, but facilitated when they occurred shortly afterwards. In contrast, global integration of shape was unaffected by the timing of microsaccades. These findings suggest that microsaccades alter the discrimination sensitivity to briefly presented shapes but do not disrupt the spatial pooling of local form signals.

Densely Dilated Spatial Pooling Convolutional Network Using Benign Loss Functions for Imbalanced Volumetric Prostate Segmentation

Background: The high incidence rate of prostate disease poses a requirement of accurate early detection. Magnetic Resonance Imaging (MRI) is one of the main imaging methods used for prostate cancer detection so far, but it has problems of imbalance and variation in appearance, therefore, automated prostate segmentation is still challenging. Objective: Aiming to accurately segment the prostate from MRI, the focus was on designing a unique network with benign loss functions. Methods: A novel Densely Dilated Spatial Pooling Convolutional Network (DDSP ConNet) in an encoderdecoder structure, with a unique DDSP block was proposed. By densely combining dilated convolution and global pooling layers, the DDSP block supplies coarse segmentation results and preserves hierarchical contextual information. Meanwhile, the DSC and Jaccard loss were adopted to train the DDSP ConNet. And it was proved theoretically that they have benign properties, including symmetry, continuity, and differentiability on the parameters of the network. Results: Extensive experiments have been conducted to corroborate the effectiveness of the DDSP ConNet with DSC and Jaccard loss on the MICCAI PROMISE12 challenge dataset. In the test dataset, the DDSP ConNet achieved a score of 85.78. Conclusion: In the conducted experiments, DDSP network with DSC and Jaccard loss outperformed most of the other competitors on the PROMISE12 dataset. Therefore, it has a better ability to extract hierarchical features and solve the imbalanced medical image problem.

Uniform spatial pooling explains topographic organization and deviation from receptive-field scale invariance in primate V1

Receptive field (RF) size and preferred spatial frequency (SF) vary greatly across the primary visual cortex (V1), increasing in a scale invariant fashion with eccentricity. Recent studies reveal that preferred SF also forms a fine-scale periodic map. A fundamental open question is how local variability in preferred SF is tied to the overall spatial RF. Here, we use two-photon imaging to simultaneously measure maps of RF size, phase selectivity, SF bandwidth, and orientation bandwidth—all of which were found to be topographically organized and correlate with preferred SF. Each of these newly characterized inter-map relationships strongly deviate from scale invariance, yet reveal a common motif—they are all accounted for by a model with uniform spatial pooling from scale invariant inputs. Our results and model provide novel and quantitative understanding of the output from V1 to downstream circuits.

Changes in annual extremes of daily temperature and precipitation in CMIP6 models

This study presents an analysis of daily temperature and precipitation extremes with return periods ranging from 2 to 50 years in the Coupled Model Intercomparison Project Phase 6 (CMIP6) multi-model ensemble of simulations. Judged by similarity with reanalyses, the new-generation models simulate the present-day temperature and precipitation extremes reasonably well. In line with previous CMIP simulations, the new simulations continue to project a large-scale picture of more frequent and more intense hot temperature extremes and precipitation extremes and vanishing cold extremes under continued global warming. Changes in temperature extremes outpace changes in global annual mean surface air temperature (GSAT) over most land masses, while changes in precipitation extremes follow changes in GSAT globally at roughly the Clausius-Clapeyron rate of ∼7%/°C. Changes in temperature and precipitation extremes normalized with respect to GSAT do not depend strongly on the choice of forcing scenario or model climate sensitivity, and do not vary strongly over time, but with notable regional variations. Over the majority of land regions, the projected intensity increases and relative frequency increases tend to be larger for more extreme hot temperature and precipitation events than for weaker events. To obtain robust estimates of these changes at local scales, large initial-condition ensemble simulations are needed. Appropriate spatial pooling of data from neighboring grid cells within individual simulations can, to some extent, reduce the needed ensemble size.

Deep Semantic-Preserving Reconstruction Hashing for Unsupervised Cross-Modal Retrieval

Deep hashing is the mainstream algorithm for large-scale cross-modal retrieval due to its high retrieval speed and low storage capacity, but the problem of reconstruction of modal semantic information is still very challenging. In order to further solve the problem of unsupervised cross-modal retrieval semantic reconstruction, we propose a novel deep semantic-preserving reconstruction hashing (DSPRH). The algorithm combines spatial and channel semantic information, and mines modal semantic information based on adaptive self-encoding and joint semantic reconstruction loss. The main contributions are as follows: (1) We introduce a new spatial pooling network module based on tensor regular-polymorphic decomposition theory to generate rank-1 tensor to capture high-order context semantics, which can assist the backbone network to capture important contextual modal semantic information. (2) Based on optimization perspective, we use global covariance pooling to capture channel semantic information and accelerate network convergence. In feature reconstruction layer, we use two bottlenecks auto-encoding to achieve visual-text modal interaction. (3) In metric learning, we design a new loss function to optimize model parameters, which can preserve the correlation between image modalities and text modalities. The DSPRH algorithm is tested on MIRFlickr-25K and NUS-WIDE. The experimental results show that DSPRH has achieved better performance on retrieval tasks.