3D-CNN-Based Fused Feature Maps with LSTM Applied to Action Recognition

Human activity recognition is an active field of research in computer vision with numerous applications. Recently, deep convolutional networks and recurrent neural networks (RNN) have received increasing attention in multimedia studies, and have yielded state-of-the-art results. In this research work, we propose a new framework which intelligently combines 3D-CNN and LSTM networks. First, we integrate discriminative information from a video into a map called a ‘motion map’ by using a deep 3-dimensional convolutional network (C3D). A motion map and the next video frame can be integrated into a new motion map, and this technique can be trained by increasing the training video length iteratively; then, the final acquired network can be used for generating the motion map of the whole video. Next, a linear weighted fusion scheme is used to fuse the network feature maps into spatio-temporal features. Finally, we use a Long-Short-Term-Memory (LSTM) encoder-decoder for final predictions. This method is simple to implement and retains discriminative and dynamic information. The improved results on benchmark public datasets prove the effectiveness and practicability of the proposed method.

Students’ approaches to solving R-FCI tasks observed by eye-tracking method

This study sought to assess the representational format of task options in the representational variant of the force concept Inventory (R-FCI) test, namely its impact on students’ problem-solving approaches. This was done with the help of eye-tracking equipment. 35 high-school students solved four tasks, mainly from the R-FCI test, which sought to assess the student’s understanding of Newton’s 1st and 2nd Law of Motion. As they were trying to solve the problems, their gazes were tracked by TobiiTX300. A comparison between students who provided the correct and incorrect answer was subsequently carried out. The correctly answering students very quickly found the correct solution both in verbal and graph representation. For motion map representation, they usually compared and made decision between two options. The incorrectly answering students did not show any consistent strategy except they paid the least attention to the correct answer. Moreover, two case stud studies of correctly and incorrectly answering students were described.