A CNN-based Human Activity Recognition System Combining a Laser Feedback Interferometry Eye Movement Sensor and an IMU for Context-aware Smart Glasses

Smart glasses are considered the next breakthrough in wearables. As the successor of smart watches and smart ear wear, they promise to extend reality by immersive embedding of content in the user's field of view. While advancements in display technology seems to fulfill this promises, interaction concepts are derived from established wearable concepts like touch interaction or voice interaction, preventing full immersion as they require the user to frequently interact with the glasses. To minimize interactions, we propose to add context-awareness to smart glasses through human activity recognition (HAR) by combining head- and eye movement features to recognize a wide range of activities. To measure eye movements in unobtrusive way, we propose laser feedback interferometry (LFI) sensors. These tiny low power sensors are highly robust to ambient light. We combine LFI sensors and an IMU to collect eye and head movement features from 15 participants performing 7 cognitive and physical activities, leading to a unique data set. To recognize activities we propose a 1D-CNN model and apply transfer learning to personalize the classification, leading to an outstanding macro-F1 score of 88.15 % which outperforms state of the art methods. Finally, we discuss the applicability of the proposed system in a smart glasses setup.

Deep Learning based Speech Emotion Recognition System

Emotions are elementary for humans, impacting perception and everyday activities like communication, learning and decision-making. Speech emotion Recognition (SER) systems aim to facilitate the natural interaction with machines by direct voice interaction rather than exploitation ancient devices as input to know verbal content and build it straightforward for human listeners to react. During this SER system primarily composed of 2 sections called feature extraction and feature classification phase. SER implements on bots to speak with humans during a non-lexical manner. The speech emotion recognition algorithm here is predicated on the Convolutional Neural Network (CNN) model, which uses varied modules for emotion recognition and classifiers to differentiate feelings like happiness, calm, anger, neutral state, sadness, and fear. The accomplishment of classification is predicated on extracted features. Finally, the emotion of a speech signal will be determined.

Towards Contactless Learning Activities during Pandemics Using Autonomous Service Robots

The COVID-19 pandemic has had a significant impact worldwide, impacting schools, undergraduate, and graduate university education. More than half a million lives have been lost due to COVID-19. Moving towards contactless learning activities has become a research area due to the rapid advancement of technology, particularly in artificial intelligence and robotics. This paper proposes an autonomous service robot for handling multiple teaching assistant duties in the educational field to move towards contactless learning activities during pandemics. We use SLAM to map and navigate the environment to proctor an exam. We also propose a human–robot voice interaction and an academic content personalization algorithm. Our results show that our robot can navigate the environment to proctor students avoiding any static or dynamic obstacles. Our cheating detection system obtained a testing accuracy of 86.85%. Our image-based exam paper scanning system can scan, extract, and process exams with high accuracy.

Design and Implementation of a Robotic Arm Assistant with Voice Interaction Using Machine Vision

It is evident that the technological growth of the last few decades has signaled the development of several application domains. One application domain that has expanded massively in recent years is robotics. The usage and spread of robotic systems in commercial and non-commercial environments resulted in increased productivity, efficiency, and higher quality of life. Many researchers have developed systems that improve many aspects of people’s lives, based on robotics. Most of the engineers use high-cost robotic arms, which are usually out of the reach of typical consumers. We fill this gap by presenting a low-cost and high-accuracy project to be used as a robotic assistant for every consumer. Our project aims to further improve people’s quality of life, and more specifically people with physical and mobility impairments. The robotic system is based on the Niryo-One robotic arm, equipped with a USB (Universal Serial Bus) HD (High Definition) camera on the end-effector. To achieve high accuracy, we modified the YOLO algorithm by adding novel features and additional computations to be used in the kinematic model. We evaluated the proposed system by conducting experiments using PhD students of our laboratory and demonstrated its effectiveness. The experimental results indicate that the robotic arm can detect and deliver the requested object in a timely manner with a 96.66% accuracy.

Robust voice user interface for internet-of-things

Internet of things (IoT) plays significant role in the fourth industrial revolution and attracts an increasing interest due to the rapid development of smart devices. IoT comprises factors of twofold. Firstly, a set of things (i.e., appliances, devices, vehicles, etc.) connected together via network. Secondly, human-device interaction to communicate with these things. Speech is the most natural methodology of interaction that can enrich user experience. In this paper, we propose a novel and effective approach for building customized voice interaction for controlling smart devices in IoT environments (i.e., Smart home). The proposed approach is based on extracting customized tiny decoding graph from a large graph constructed using weighted finite sates transducers. Experimental results showed that tiny decoding graphs are very efficient in terms of computational resources and recognition accuracy in clean and noisy conditions. To emphasize the effectiveness of the proposed approach, the standard Resources Management (RM1) dataset was employed and promising results were achieved when compared with four competitive approaches.