scholarly journals Sensing-HH: A Deep Hybrid Attention Model for Footwear Recognition

Electronics ◽  
2020 ◽  
Vol 9 (9) ◽  
pp. 1552
Author(s):  
Yumin Yao ◽  
Ya Wen ◽  
Jianxin Wang
Keyword(s):  
Short Term Memory ◽  
Gait Recognition ◽  
Recognition Task ◽  
Gait Pattern ◽  
Human Gait ◽  
Smart Devices ◽  
Motion Sensors ◽  
Attention Model ◽  
Young Females ◽  
Using Data

The human gait pattern is an emerging biometric trait for user identification of smart devices. However, one of the challenges in this biometric domain is the gait pattern change caused by footwear, especially if the users are wearing high heels (HH). Wearing HH puts extra stress and pressure on various parts of the human body and it alters the wearer’s common gait pattern, which may cause difficulties in gait recognition. In this paper, we propose the Sensing-HH, a deep hybrid attention model for recognizing the subject’s shoes, flat or different types of HH, using smartphone’s motion sensors. In this model, two streams of convolutional and bidirectional long short-term memory (LSTM) networks are designed as the backbone, which extract the hierarchical spatial and temporal representations of accelerometer and gyroscope individually. We also introduce a spatio attention mechanism into the stacked convolutional layers to scan the crucial structure of the data. This mechanism enables the hybrid neural networks to capture extra information from the signal and thus it is able to significantly improve the discriminative power of the classifier for the footwear recognition task. To evaluate Sensing-HH, we built a dataset with 35 young females, each of whom walked for 4 min wearing shoes with varied heights of the heels. We conducted extensive experiments and the results demonstrated that the Sensing-HH outperformed the baseline models on leave-one-subject-out cross-validation (LOSO-CV). The Sensing-HH achieved the best Fm score, which was 0.827 when the smartphone was attached to the waist. This outperformed all the baseline methods at least by more than 14%. Meanwhile, the F1 Score of the Ultra HH was as high as 0.91. The results suggest the proposed model has made the footwear recognition more efficient and automated. We hope the findings from this study paves the way for a more sophisticated application using data from motion sensors, as well as lead to a path to a more robust biometric system based on gait pattern.

scholarly journals Multimodal Few-Shot Learning for Gait Recognition

2020 ◽  
Vol 10 (21) ◽  
pp. 7619
Author(s):  
Jucheol Moon ◽  
Nhat Anh Le ◽  
Nelson Hebert Minaya ◽  
Sang-Il Choi
Keyword(s):  
Neural Network ◽  
Time Series ◽  
Gait Recognition ◽  
Gait Pattern ◽  
Recognition Problem ◽  
Human Gait ◽  
Training Dataset ◽  
Support Vector ◽  
Open Set ◽  
Latent Space

A person’s gait is a behavioral trait that is uniquely associated with each individual and can be used to recognize the person. As information about the human gait can be captured by wearable devices, a few studies have led to the proposal of methods to process gait information for identification purposes. Despite recent advances in gait recognition, an open set gait recognition problem presents challenges to current approaches. To address the open set gait recognition problem, a system should be able to deal with unseen subjects who have not included in the training dataset. In this paper, we propose a system that learns a mapping from a multimodal time series collected using insole to a latent (embedding vector) space to address the open set gait recognition problem. The distance between two embedding vectors in the latent space corresponds to the similarity between two multimodal time series. Using the characteristics of the human gait pattern, multimodal time series are sliced into unit steps. The system maps unit steps to embedding vectors using an ensemble consisting of a convolutional neural network and a recurrent neural network. To recognize each individual, the system learns a decision function using a one-class support vector machine from a few embedding vectors of the person in the latent space, then the system determines whether an unknown unit step is recognized as belonging to a known individual. Our experiments demonstrate that the proposed framework recognizes individuals with high accuracy regardless they have been registered or not. If we could have an environment in which all people would be wearing the insole, the framework would be used for user verification widely.

scholarly journals Human Gait Recognition Based on Frame-by-Frame Gait Energy Images and Convolutional Long Short-Term Memory

2019 ◽  
Vol 30 (01) ◽  
pp. 1950027 ◽  
Author(s):  
Xiuhui Wang ◽  
Wei Qi Yan
Keyword(s):  
Short Term Memory ◽  
Gait Recognition ◽  
Large Population ◽  
Recognition Rate ◽  
Human Gait ◽  
Short Term ◽  
Term Memory ◽  
Cross Covariance ◽  
Novel Method ◽  
Long Short Term Memory

Human gait recognition is one of the most promising biometric technologies, especially for unobtrusive video surveillance and human identification from a distance. Aiming at improving recognition rate, in this paper we study gait recognition using deep learning and propose a novel method based on convolutional Long Short-Term Memory (Conv-LSTM). First, we present a variation of Gait Energy Images, i.e. frame-by-frame GEI (ff-GEI), to expand the volume of available Gait Energy Images (GEI) data and relax the constraints of gait cycle segmentation required by existing gait recognition methods. Second, we demonstrate the effectiveness of ff-GEI by analyzing the cross-covariance of one person’s gait data. Then, making use of the temporality of our human gait, we design a novel gait recognition model using Conv-LSTM. Finally, the proposed method is evaluated extensively based on the CASIA Dataset B for cross-view gait recognition, furthermore the OU-ISIR Large Population Dataset is employed to verify its generalization ability. Our experimental results show that the proposed method outperforms other algorithms based on these two datasets. The results indicate that the proposed ff-GEI model using Conv-LSTM, coupled with the new gait representation, can effectively solve the problems related to cross-view gait recognition.

scholarly journals Human Gait Data Augmentation and Trajectory Prediction for Lower-Limb Rehabilitation Robot Control Using GANs and Attention Mechanism

Machines ◽  
2021 ◽  
Vol 9 (12) ◽  
pp. 367
Author(s):  
Yan Wang ◽  
Zhikang Li ◽  
Xin Wang ◽  
Hongnian Yu ◽  
Wudai Liao ◽  
...  
Keyword(s):  
Data Augmentation ◽  
Control Strategies ◽  
Gait Pattern ◽  
Attention Mechanism ◽  
Human Gait ◽  
Generative Adversarial Networks ◽  
Joint Angles ◽  
Two Stage ◽  
The Real ◽  
Attention Model

To date, several alterations in the gait pattern can be treated through rehabilitative approaches and robot assisted therapy (RAT). Gait data and gait trajectories are essential in specific exoskeleton control strategies. Nevertheless, the scarcity of human gait data due to the high cost of data collection or privacy concerns can hinder the performance of controllers or models. This paper thus first creates a GANs-based (Generative Adversarial Networks) data augmentation method to generate synthetic human gait data while still retaining the dynamics of the real gait data. Then, both the real collected and the synthesized gait data are fed to our constructed two-stage attention model for gait trajectories prediction. The real human gait data are collected with the five healthy subjects recruited from an optical motion capture platform. Experimental results indicate that the created GANs-based data augmentation model can synthesize realistic-looking multi-dimensional human gait data. Also, the two-stage attention model performs better compared with the LSTM model; the attention mechanism shows a higher capacity of learning dependencies between the historical gait data to accurately predict the current values of the hip joint angles and knee joint angles in the gait trajectory. The predicted gait trajectories depending on the historical gait data can be further used for gait trajectory tracking strategies.

Dimensionality and Signal Detection Effects on a Short-Term Memory Task with Mentally Retarded Children

1977 ◽  
Vol 40 (2) ◽  
pp. 421-422 ◽  
Author(s):  
H. Lee Swanson
Keyword(s):  
Short Term Memory ◽  
Recognition Performance ◽  
A Priori ◽  
Memory Task ◽  
Recognition Task ◽  
Mentally Retarded ◽  
Mentally Retarded Children ◽  
Using Data ◽  
Post Hoc ◽  
Main Effects

The purpose was to compare a priori and d′ measures using data from Swanson and Watson's (1976) study with 10 mentally retarded children on a serial recognition task. Treatment main effects were similar using both measures. Significant primacy and recency effects were found with d′ measures, however, this was not found in a post hoc analysis of proportion correct scores. The results suggested that future investigations of serial recognition performance with emotionally mentally retarded children use an unbiased index of memory strength.

scholarly journals The Role of Entropy in Construct Specification Equations (CSE) to Improve the Validity of Memory Tests

Entropy ◽  
2021 ◽  
Vol 23 (2) ◽  
pp. 212
Author(s):  
Jeanette Melin ◽  
Stefan Cano ◽  
Leslie Pendrill
Keyword(s):  
Rating Scales ◽  
Short Term Memory ◽  
Item Difficulty ◽  
Digit Span ◽  
Reliability And Validity ◽  
Practice Research ◽  
Theory Approach ◽  
Memory Tests ◽  
Using Data ◽  
Block Test

Commonly used rating scales and tests have been found lacking reliability and validity, for example in neurodegenerative diseases studies, owing to not making recourse to the inherent ordinality of human responses, nor acknowledging the separability of person ability and item difficulty parameters according to the well-known Rasch model. Here, we adopt an information theory approach, particularly extending deployment of the classic Brillouin entropy expression when explaining the difficulty of recalling non-verbal sequences in memory tests (i.e., Corsi Block Test and Digit Span Test): a more ordered task, of less entropy, will generally be easier to perform. Construct specification equations (CSEs) as a part of a methodological development, with entropy-based variables dominating, are found experimentally to explain (r=R2 = 0.98) and predict the construct of task difficulty for short-term memory tests using data from the NeuroMET (n = 88) and Gothenburg MCI (n = 257) studies. We propose entropy-based equivalence criteria, whereby different tasks (in the form of items) from different tests can be combined, enabling new memory tests to be formed by choosing a bespoke selection of items, leading to more efficient testing, improved reliability (reduced uncertainties) and validity. This provides opportunities for more practical and accurate measurement in clinical practice, research and trials.

scholarly journals Prediction of Head Movement in 360-Degree Videos Using Attention Model

Sensors ◽  
2021 ◽  
Vol 21 (11) ◽  
pp. 3678
Author(s):  
Dongwon Lee ◽  
Minji Choi ◽  
Joohyun Lee
Keyword(s):  
Machine Learning ◽  
Short Term Memory ◽  
Moving Average ◽  
The Other ◽  
Learning Models ◽  
Short Term ◽  
Term Memory ◽  
Attention Model ◽  
Long Short Term Memory ◽  
Machine Learning Models

In this paper, we propose a prediction algorithm, the combination of Long Short-Term Memory (LSTM) and attention model, based on machine learning models to predict the vision coordinates when watching 360-degree videos in a Virtual Reality (VR) or Augmented Reality (AR) system. Predicting the vision coordinates while video streaming is important when the network condition is degraded. However, the traditional prediction models such as Moving Average (MA) and Autoregression Moving Average (ARMA) are linear so they cannot consider the nonlinear relationship. Therefore, machine learning models based on deep learning are recently used for nonlinear predictions. We use the Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) neural network methods, originated in Recurrent Neural Networks (RNN), and predict the head position in the 360-degree videos. Therefore, we adopt the attention model to LSTM to make more accurate results. We also compare the performance of the proposed model with the other machine learning models such as Multi-Layer Perceptron (MLP) and RNN using the root mean squared error (RMSE) of predicted and real coordinates. We demonstrate that our model can predict the vision coordinates more accurately than the other models in various videos.

scholarly journals Higher Handgrip Strength Is Linked to Better Cognitive Performance in Chinese Adults with Hypertension

2021 ◽  
Vol 11 (8) ◽  
pp. 985
Author(s):  
Shenghua Lu ◽  
Fabian Herold ◽  
Yanjie Zhang ◽  
Yuruo Lei ◽  
Arthur F. Kramer ◽  
...  
Keyword(s):  
Cognitive Performance ◽  
Short Term Memory ◽  
Handgrip Strength ◽  
Adult Health ◽  
Linear Regression Models ◽  
Chinese Adults ◽  
Short Term ◽  
Term Memory ◽  
Using Data ◽  
The Relationship

Objective: There is growing evidence that in adults, higher levels of handgrip strength (HGS) are linked to better cognitive performance. However, the relationship between HGS and cognitive performance has not been sufficiently investigated in special cohorts, such as individuals with hypertension who have an intrinsically higher risk of cognitive decline. Thus, the purpose of this study was to examine the relationship between HGS and cognitive performance in adults with hypertension using data from the Global Ageing and Adult Health Survey (SAGE). Methods: A total of 4486 Chinese adults with hypertension from the SAGE were included in this study. Absolute handgrip strength (aHGS in kilograms) was measured using a handheld electronic dynamometer, and cognitive performance was assessed in the domains of short-term memory, delayed memory, and language ability. Multiple linear regression models were fitted to examine the association between relative handgrip strength (rHGS; aHGS divided by body mass index) and measures of cognitive performance. Results: Overall, higher levels of rHGS were associated with higher scores in short-term memory (β = 0.20) and language (β = 0.63) compared with the lowest tertiles of rHGS. In male participants, higher HGS was associated with higher scores in short-term memory (β = 0.31), language (β = 0.64), and delayed memory (β = 0.22). There were no associations between rHGS and cognitive performance measures in females. Conclusion: We observed that a higher level of rHGS was associated with better cognitive performance among hypertensive male individuals. Further studies are needed to investigate the neurobiological mechanisms, including sex-specific differences driving the relationship between measures of HGS and cognitive performance in individuals with hypertension.

Video-based Indoor Human Gait Recognition Using Depth Imaging and Hidden Markov Model: A Smart System for Smart Home

2010 ◽  
Vol 20 (1) ◽  
pp. 120-128 ◽  
Author(s):  
Md. Zia Uddin ◽  
Tae-Seong Kim ◽  
Jeong Tai Kim
Keyword(s):  
Hidden Markov ◽  
Gait Recognition ◽  
Smart Homes ◽  
The Elderly ◽  
Recognition System ◽  
Home Healthcare ◽  
Human Gait ◽  
Depth Imaging ◽  
Smart System ◽  
Spatiotemporal Features

Smart homes that are capable of home healthcare and e-Health services are receiving much attention due to their potential for better care of the elderly and disabled in an indoor environment. Recently the Center for Sustainable Healthy Buildings at Kyung Hee University has developed a novel indoor human activity recognition methodology based on depth imaging of a user’s activities. This system utilizes Independent Component Analysis to extract spatiotemporal features from a series of depth silhouettes of various activities. To recognise the activities from the spatiotemporal features, trained Hidden Markov Models of the activities would be used. In this study, this technique has been extended to recognise human gaits (including normal and abnormal). Since this system could be of great significance for the caring of the elderly, to promote and preserve their health and independence, the gait recognition system would be considered a primary function of the smart system for smart homes. The indoor gait recognition system is trained to detect abnormal gait patterns and generate warnings. The system works in real-time and is aimed to be installed at smart homes. This paper provides the information for further development of the system for their application in the future.

Sign in / Sign up

Export Citation Format

Share Document