Neural correlates of integration processes during dynamic face perception

Integrating the spatiotemporal information acquired from the highly dynamic world around us is essential to navigate, reason, and decide properly. Although this is particularly important in a face-to-face conversation, very little research to date has specifically examined the neural correlates of temporal integration in dynamic face perception. Here we present statistically robust observations regarding the brain activations measured via electroencephalography (EEG) that are specific to the temporal integration. To that end, we generate videos of neutral faces of individuals and non-face objects, modulate the contrast of the even and odd frames at two specific frequencies ($$f_1$$ f 1 and $$f_2$$ f 2 ) in an interlaced manner, and measure the steady-state visual evoked potential as participants view the videos. Then, we analyze the intermodulation components (IMs: ($$nf_1\pm mf_2$$ n f 1 ± m f 2 ), a linear combination of the fundamentals with integer multipliers) that consequently reflect the nonlinear processing and indicate temporal integration by design. We show that electrodes around the medial temporal, inferior, and medial frontal areas respond strongly and selectively when viewing dynamic faces, which manifests the essential processes underlying our ability to perceive and understand our social world. The generation of IMs is only possible if even and odd frames are processed in succession and integrated temporally, therefore, the strong IMs in our frequency spectrum analysis show that the time between frames (1/60 s) is sufficient for temporal integration.

Expectation violations produce error signals in mouse V1

Repeated exposure to visual sequences changes the form of evoked activity in the primary visual cortex (V1). Predictive coding theory provides a potential explanation for this, namely that plasticity shapes cortical circuits to encode spatiotemporal predictions and that subsequent responses are modulated by the degree to which actual inputs match these expectations. Here we use a recently developed statistical modeling technique called Model-Based Targeted Dimensionality Reduction (MbTDR) to study visually-evoked dynamics in mouse V1 in context of a previously described experimental paradigm called "sequence learning". We report that evoked spiking activity changed significantly with training, in a manner generally consistent with the predictive coding framework. Neural responses to expected stimuli were suppressed in a late window (100-150ms) after stimulus onset following training, while responses to novel stimuli were not. Omitting predictable stimuli led to increased firing at the expected time of stimulus onset, but only in trained mice. Substituting a novel stimulus for a familiar one led to changes in firing that persisted for at least 300ms. In addition, we show that spiking data can be used to accurately decode time within the sequence. Our findings are consistent with the idea that plasticity in early visual circuits is involved in coding spatiotemporal information.

Advances in an Event-Based Spatiotemporal Data Modeling

Spatiotemporal data are vitally important for the national economy and defense modernization since it is not only an important component of human society and geographical information of the environment but also a key carrier of spatiotemporal information. An event-based spatiotemporal data model and its improvements are employed to model spatiotemporal objects, change history, and change relation, which is the main approach to resolve the spatiotemporal change modeling and has been comprehensively developed in modeling theory and applications. This manuscript studies the event-based spatiotemporal data modeling theory based on three aspects of the cognitive theory, which are the spatiotemporal object, the concept of the spatiotemporal dynamic object, and the spatiotemporal object relationship. Then, the implementation characteristics of the models were analyzed regarding the management of cadastral information, analog natural disaster phenomena, and reasoning. Finally, the key points and difficulties of an event-based spatiotemporal data modeling and prospective developmental trends were discussed to provide insights with spatiotemporal data modeling.

Deep Transfer Learning for Land Use and Land Cover Classification: A Comparative Study

Efficiently implementing remote sensing image classification with high spatial resolution imagery can provide significant value in land use and land cover (LULC) classification. The new advances in remote sensing and deep learning technologies have facilitated the extraction of spatiotemporal information for LULC classification. Moreover, diverse disciplines of science, including remote sensing, have utilised tremendous improvements in image classification involving convolutional neural networks (CNNs) with transfer learning. In this study, instead of training CNNs from scratch, the transfer learning was applied to fine-tune pre-trained networks Visual Geometry Group (VGG16) and Wide Residual Networks (WRNs), by replacing the final layers with additional layers, for LULC classification using the red–green–blue version of the EuroSAT dataset. Moreover, the performance and computational time are compared and optimised with techniques such as early stopping, gradient clipping, adaptive learning rates, and data augmentation. The proposed approaches have addressed the limited-data problem, and very good accuracies were achieved. The results show that the proposed method based on WRNs outperformed the previous best results in terms of computational efficiency and accuracy, by achieving 99.17%.

Dynamic monitoring of urban built-up object expansion trajectories in Karachi, Pakistan with time series images and the LandTrendr algorithm

In the complex process of urbanization, retrieving its dynamic expansion trajectories with an efficient method is challenging, especially for urban regions that are not clearly distinguished from the surroundings in arid regions. In this study, we propose a framework for extracting spatiotemporal change information on urban disturbances. First, the urban built-up object areas in 2000 and 2020 were obtained using object-oriented segmentation method. Second, we applied LandTrendr (LT) algorithm and multiple bands/indices to extract annual spatiotemporal information. This process was implemented effectively with the support of the cloud computing platform of Earth Observation big data. The overall accuracy of time information extraction, the kappa coefficient, and average detection error were 83.76%, 0.79, and 0.57 a, respectively. These results show that Karachi expanded continuously during 2000–2020, with an average annual growth rate of 4.7%. However, this expansion was not spatiotemporally balanced. The coastal area developed quickly within a shorter duration, whereas the main newly added urban regions locate in the northern and eastern inland areas. This study demonstrated an effective framework for extract the dynamic spatiotemporal change information of urban built-up objects and substantially eliminate the salt-and-pepper effect based on pixel detection. Methods used in our study are of general promotion significance in the monitoring of other disturbances caused by natural or human activities.

An Analytic Method for Improving the Reliability of Models Based on a Histogram for Prediction of Companion Dogs’ Behaviors

Dogs and cats tend to show their conditions and desires through their behaviors. In companion animal behavior recognition, behavior data obtained by attaching a wearable device or sensor to a dog’s body are mostly used. However, differences occur in the output values of the sensor when the dog moves violently. A tightly coupled RGB time tensor network (TRT-Net) is proposed that minimizes the loss of spatiotemporal information by reflecting the three components (x-, y-, and z-axes) of the skeleton sequences in the corresponding three channels (red, green, and blue) for the behavioral classification of dogs. This paper introduces the YouTube-C7B dataset consisting of dog behaviors in various environments. Based on a method that visualizes the Conv-layer filters in analyzable feature maps, we add reliability to the results derived by the model. We can identify the joint parts, i.e., those represented as rows of input images showing behaviors, learned by the proposed model mainly for making decisions. Finally, the performance of the proposed method is compared to those of the LSTM, GRU, and RNN models. The experimental results demonstrate that the proposed TRT-Net method classifies dog behaviors more effectively, with improved accuracy and F1 scores of 7.9% and 7.3% over conventional models.

Significance of GABAA Receptor for Cognitive Function and Hippocampal Pathology

The hippocampus is a primary area for contextual memory, known to process spatiotemporal information within a specific episode. Long-term strengthening of glutamatergic transmission is a mechanism of contextual learning in the dorsal cornu ammonis 1 (CA1) area of the hippocampus. CA1-specific immobilization or blockade of α-amino-3-hydroxyl-5-methyl-4-isoxazole-propionate (AMPA) receptor delivery can impair learning performance, indicating a causal relationship between learning and receptor delivery into the synapse. Moreover, contextual learning also strengthens GABAA (gamma-aminobutyric acid) receptor-mediated inhibitory synapses onto CA1 neurons. Recently we revealed that strengthening of GABAA receptor-mediated inhibitory synapses preceded excitatory synaptic plasticity after contextual learning, resulting in a reduced synaptic excitatory/inhibitory (E/I) input balance that returned to pretraining levels within 10 min. The faster plasticity at inhibitory synapses may allow encoding a contextual memory and prevent cognitive dysfunction in various hippocampal pathologies. In this review, we focus on the dynamic changes of GABAA receptor mediated-synaptic currents after contextual learning and the intracellular mechanism underlying rapid inhibitory synaptic plasticity. In addition, we discuss that several pathologies, such as Alzheimer’s disease, autism spectrum disorders and epilepsy are characterized by alterations in GABAA receptor trafficking, synaptic E/I imbalance and neuronal excitability.