Spatiotemporal Features Learning from Song for Emotions Recognition with Time Distributed CNN

The paper proposes an effective continuous gesture recognition method, which includes two modules: segmentation and recognition. In the segmentation module, the video frames are divided into gesture frames and transitional frames by using the information of hand motion and appearance, and continuous gesture sequences are segmented into isolated sequences. In the recognition module, our method exploits the spatiotemporal information embedded in RGB and depth sequences. For the RGB modality, our method adopts Convolutional Long Short-Term Memory Networks to learn long-term spatiotemporal features from short-term spatiotemporal features obtained from a 3D convolutional neural network. For the depth modality, our method converts a sequence into Dynamic Images and Motion Dynamic Images through weighted rank pooling and feed them into Convolutional Neural Networks, respectively. Our method has been evaluated on both ChaLearn LAP Large-scale Continuous Gesture Dataset and Montalbano Gesture Dataset and achieved state-of-the-art performance.

Download Full-text

Video-based Indoor Human Gait Recognition Using Depth Imaging and Hidden Markov Model: A Smart System for Smart Home

Indoor and Built Environment ◽

10.1177/1420326x10391140 ◽

2010 ◽

Vol 20 (1) ◽

pp. 120-128 ◽

Cited By ~ 11

Author(s):

Md. Zia Uddin ◽

Tae-Seong Kim ◽

Jeong Tai Kim

Keyword(s):

Hidden Markov ◽

Gait Recognition ◽

Smart Homes ◽

The Elderly ◽

Recognition System ◽

Home Healthcare ◽

Human Gait ◽

Depth Imaging ◽

Smart System ◽

Spatiotemporal Features

Smart homes that are capable of home healthcare and e-Health services are receiving much attention due to their potential for better care of the elderly and disabled in an indoor environment. Recently the Center for Sustainable Healthy Buildings at Kyung Hee University has developed a novel indoor human activity recognition methodology based on depth imaging of a user’s activities. This system utilizes Independent Component Analysis to extract spatiotemporal features from a series of depth silhouettes of various activities. To recognise the activities from the spatiotemporal features, trained Hidden Markov Models of the activities would be used. In this study, this technique has been extended to recognise human gaits (including normal and abnormal). Since this system could be of great significance for the caring of the elderly, to promote and preserve their health and independence, the gait recognition system would be considered a primary function of the smart system for smart homes. The indoor gait recognition system is trained to detect abnormal gait patterns and generate warnings. The system works in real-time and is aimed to be installed at smart homes. This paper provides the information for further development of the system for their application in the future.

Download Full-text

Evaluation of a Nonrigid Motion Compensation Technique Based on Spatiotemporal Features for Small Lesion Detection in Breast MRI

Advances in Artificial Neural Systems ◽

10.1155/2012/808602 ◽

2012 ◽

Vol 2012 ◽

pp. 1-10 ◽

Cited By ~ 1

Author(s):

F. Steinbruecker ◽

A. Meyer-Baese ◽

T. Schlossbauer ◽

D. Cremers

Keyword(s):

Breast Cancer ◽

Motion Compensation ◽

Rigid Motion ◽

Breast Mri ◽

Lesion Detection ◽

Nonrigid Motion ◽

Small Lesion ◽

Spatiotemporal Features ◽

Optical Flow Method ◽

Compensation Technique

Motion-induced artifacts represent a major problem in detection and diagnosis of breast cancer in dynamic contrast-enhanced magnetic resonance imaging. The goal of this paper is to evaluate the performance of a new nonrigid motion correction algorithm based on the optical flow method. For each of the small lesions, we extracted morphological and dynamical features describing both global and local shape, and kinetics behavior. In this paper, we compare the performance of each extracted feature set under consideration of several 2D or 3D motion compensation parameters for the differential diagnosis of enhancing lesions in breast MRI. Based on several simulation results, we determined the optimal motion compensation parameters. Our results have shown that motion compensation can improve the classification results. The results suggest that the computerized analysis system based on the non-rigid motion compensation technique and spatiotemporal features has the potential to increase the diagnostic accuracy of MRI mammography for small lesions and can be used as a basis for computer-aided diagnosis of breast cancer with MR mammography.

Download Full-text

Characterizing the spatiotemporal dynamics of soil water stable isotopes on a karst hillslope in southwestern China

10.5194/egusphere-egu21-640 ◽

2021 ◽

Author(s):

Qin Liu ◽

Tiejun Wang ◽

Cong-qiang Liu ◽

Xi Chen

Keyword(s):

Soil Water ◽

Temporal Stability ◽

Spatiotemporal Dynamics ◽

Spatial Variations ◽

Spatiotemporal Variability ◽

Strong Negative Correlation ◽

Mean Values ◽

Spatiotemporal Features ◽

Field Evidence ◽

Karst Hillslope

Soil water stable isotope compositions (SWSI; i.e., &#948;D and &#948;18O) and soil moisture content (SMC) are widely used to illuminate water exchange processes across the atmosphere-land interface. Thus, the knowledge of spatiotemporal dynamics of these two variables is critical to help our understanding of relevant ecohydrological processes. However, in comparison to the efforts for elucidating the spatiotemporal variability in SMC, much less attention was paid to understand the spatiotemporal variability in SWSI, which also raises the question as to whether SWSI and SMC share similar spatiotemporal features. To this end, the spatiotemporal dynamics of SWSI and SMC were jointly investigated on a karst hillslope with eight sampling campaigns among two years. The method of temporal stability analysis (TSA) was adopted to evaluate the spatiotemporal patterns of SWSI and SMC in this study. Generally, both &#948;D and &#948;18O exhibited considerable temporal and spatial variations; meanwhile, the variations in &#948;D and &#948;18O values were relatively smaller than that of SMC. In addition, in comparison with the spatial pattern of SMC, there were no clear relationships between the standard deviation (SD) and the spatial mean of &#948;D or &#948;18O. However, the SD of line-conditioned excess (lc-excess) and its mean values displayed a strong negative correlation, indicating that the spatial variations in lc-excess increased with soil evaporation. Moreover, SWSI displayed weaker temporal stability than SMC and no clear controlling factors were identified, suggesting that the spatiotemporal dynamics of SWSI might be more complex than that of SMC. This study provided comprehensive field evidence that there existed profound spatiotemporal variability in SWSI and its spatiotemporal features were different from SMC, highlighting that the spatiotemporal variability in SWSI needs to be considered in isotope-based estimations and it should be investigated separately from the spatiotemporal characteristics of SMC in future studies.

Download Full-text