scholarly journals Mimicking speaker’s lip movement on a 3D head model using cosine function fitting

2017 ◽  
Vol 65 (5) ◽  
pp. 733-739
Author(s):  
I. Lüsi ◽  
G. Anbarjafari
Keyword(s):  
Research Work ◽  
Cosine Function ◽  
Microsoft Kinect ◽  
Head Model ◽  
Depth Information ◽  
Facial Movement ◽  
Function Fitting ◽  
Lip Movement ◽  
3D Head Model ◽  
Movement Tracking

Abstract Real-time mimicking of human facial movement on a 3D head model is a challenge which has attracted attention of many researchers. In this research work we propose a new method for enhancing the capturing of the shape of lips. We present an automatic lip movement tracking method which employs a cosine function to interpolate between extracted lip features in order to make the detection more accurate. In order to test the proposed method, mimicking lip movements of a speaker on a 3D head model is studied. Microsoft Kinect II is used in order to capture videos and both RGB and depth information are used to locate the mouth of a speaker followed by fitting a cosine function in order to track the changes of the features extracted from the lips.

scholarly journals CNN-Based Facial Expression Recognition from Annotated RGB-D Images for Human–Robot Interaction

2019 ◽  
Vol 16 (04) ◽  
pp. 1941002 ◽  
Author(s):  
Jing Li ◽  
Yang Mi ◽  
Gongfa Li ◽  
Zhaojie Ju
Keyword(s):  
Facial Expression ◽  
Facial Expression Recognition ◽  
Recognition Task ◽  
Recognition System ◽  
Human Robot Interaction ◽  
Microsoft Kinect ◽  
Depth Information ◽  
Expression Recognition ◽  
Stream Network ◽  
Depth Images

Facial expression recognition has been widely used in human computer interaction (HCI) systems. Over the years, researchers have proposed different feature descriptors, implemented different classification methods, and carried out a number of experiments on various datasets for automatic facial expression recognition. However, most of them used 2D static images or 2D video sequences for the recognition task. The main limitations of 2D-based analysis are problems associated with variations in pose and illumination, which reduce the recognition accuracy. Therefore, an alternative way is to incorporate depth information acquired by 3D sensor, because it is invariant in both pose and illumination. In this paper, we present a two-stream convolutional neural network (CNN)-based facial expression recognition system and test it on our own RGB-D facial expression dataset collected by Microsoft Kinect for XBOX in unspontaneous scenarios since Kinect is an inexpensive and portable device to capture both RGB and depth information. Our fully annotated dataset includes seven expressions (i.e., neutral, sadness, disgust, fear, happiness, anger, and surprise) for 15 subjects (9 males and 6 females) aged from 20 to 25. The two individual CNNs are identical in architecture but do not share parameters. To combine the detection results produced by these two CNNs, we propose the late fusion approach. The experimental results demonstrate that the proposed two-stream network using RGB-D images is superior to that of using only RGB images or depth images.

scholarly journals Artificial Neural Network for Vibration Frequency Measurement Using Kinect V2

2019 ◽  
Vol 2019 ◽  
pp. 1-16
Author(s):  
Jiantao Liu ◽  
Xiaoxiang Yang
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Vibration Frequency ◽  
Optical Measurement ◽  
Frequency Measurement ◽  
Microsoft Kinect ◽  
Depth Information ◽  
Research Attention ◽  
Full Field ◽  
Artificial Neural

Optical measurement can substantially reduce the required amount of labor and simplify the measurement process. Furthermore, the optical measurement method can provide full-field measurement results of the target object without affecting the physical properties of the measurement target, such as stiffness, mass, or damping. The advent of consumer grade depth cameras, such as the Microsoft Kinect, Intel RealSence, and ASUS Xtion, has attracted significant research attention owing to their availability and robustness in sampling depth information. This paper presents an effective method employing the Kinect sensor V2 and an artificial neural network for vibration frequency measurement. Experiments were conducted to verify the performance of the proposed method. The proposed method can provide good frequency prediction within acceptable accuracy compared to an industrial vibrometer, with the advantages of contactless process and easy pipeline implementation.

Towards a Real-Time Fall Detection System using Kinect Sensor

2016 ◽  
Vol 6 (1) ◽  
pp. 41-58 ◽  
Author(s):  
Nadia Baha ◽  
Eden Beloudah ◽  
Mehdi Ousmer
Keyword(s):  
Real Time ◽  
Detection System ◽  
Fall Detection ◽  
Microsoft Kinect ◽  
False Alarms ◽  
Depth Information ◽  
Detection Rates ◽  
Major Health Problem ◽  
Major Health ◽  
Time Fall

Falls are the major health problem among older people who live alone in their home. In the past few years, several studies have been proposed to solve the dilemma especially those which exploit video surveillance. In this paper, in order to allow older adult to safely continue living in home environments, the authors propose a method which combines two different configurations of the Microsoft Kinect: The first one is based on the person's depth information and his velocity (Ceiling mounted Kinect). The second one is based on the variation of bounding box parameters and its velocity (Frontal Kinect). Experimental results on real datasets are conducted and a comparative evaluation of the obtained results relative to the state-of-art methods is presented. The results show that the authors' method is able to accurately detect several types of falls in real-time as well as achieving a significant reduction in false alarms and improves detection rates.

Development of precise 3D head model : A sixteen million model

2003 ◽  
Vol VIII.03.1 (0) ◽  
pp. 95-96
Author(s):  
Tetsuya NISHIMOTO ◽  
Susumu EJIMA ◽  
Shigeyuki MURAKAMI ◽  
Hiroyuki TAKAO ◽  
Kohei TOMONAGA ◽  
...  
Keyword(s):  
Head Model ◽  
3D Head Model

scholarly journals Real Time 3D Facial Movement Tracking Using a Monocular Camera

Sensors ◽  
2016 ◽  
Vol 16 (8) ◽  
pp. 1157 ◽  
Author(s):  
Yanchao Dong ◽  
Yanming Wang ◽  
Jiguang Yue ◽  
Zhencheng Hu
Keyword(s):  
Real Time ◽  
Facial Movement ◽  
Monocular Camera ◽  
Movement Tracking

scholarly journals Imputation of Rainfall Data Using the Sine Cosine Function Fitting Neural Network

2021 ◽  
Vol 6 (7) ◽  
pp. 39
Author(s):  
Po Chan Chiu ◽  
Ali Selamat ◽  
Ondrej Krejcar ◽  
King Kuok Kuok ◽  
Enrique Herrera-Viedma ◽  
...  
Keyword(s):  
Neural Network ◽  
Rainfall Data ◽  
Cosine Function ◽  
Function Fitting

scholarly journals Is sensor space analysis good enough? Spatial patterns as a tool for assessing spatial mixing of EEG/MEG rhythms

2021 ◽  
Author(s):  
Natalie Schaworonkow ◽  
Vadim V Nikulin
Keyword(s):  
Basic Research ◽  
General Idea ◽  
Head Model ◽  
Mu Rhythm ◽  
Volume Conduction ◽  
Space Analysis ◽  
Non Invasive ◽  
Individual Contributions ◽  
The Individual ◽  
3D Head Model

Analyzing non-invasive recordings of electroencephalography (EEG) and magnetoencephalography (MEG) directly in sensor space, using the signal from individual sensors, is a convenient and standard way of working with this type of data. However, volume conduction introduces considerable challenges for sensor space analysis. While the general idea of signal mixing due to volume conduction in EEG/MEG is recognized, the implications have not yet been clearly exemplified. Here, we illustrate how different types of activity overlap on the level of individual sensors. We show spatial mixing in the context of alpha rhythms, which are known to have generators in different areas of the brain. Using simulations with a realistic 3D head model and lead field and data analysis of a large resting-state EEG dataset, we show that electrode signals can be differentially affected by spatial mixing by computing a sensor complexity measure. While prominent occipital alpha rhythms result in less heterogeneous spatial mixing on posterior electrodes, central electrodes show a diversity of rhythms present. This makes the individual contributions, such as the sensorimotor mu-rhythm and temporal alpha rhythms, hard to disentangle from the dominant occipital alpha. Additionally, we show how strong occipital rhythms rhythms can contribute the majority of activity to frontal channels, potentially compromising analyses that are solely conducted in sensor space. We also outline specific consequences of signal mixing for frequently used assessment of power, power ratios and connectivity profiles in basic research and for neurofeedback application. With this work, we hope to illustrate the effects of volume conduction in a concrete way, such that the provided practical illustrations may be of use to EEG researchers to in order to evaluate whether sensor space is an appropriate choice for their topic of investigation.

Sign in / Sign up

Export Citation Format

Share Document