scholarly journals Deep4D: A Compact Generative Representation for Volumetric Video

2021 ◽  
Vol 2 ◽  
Author(s):  
João Regateiro ◽  
Marco Volino ◽  
Adrian Hilton
Keyword(s):  
Generative Model ◽  
Character Animation ◽  
Video Sequences ◽  
Motion Capture Data ◽  
High Quality ◽  
Motion Graphs ◽  
Video Acquisition ◽  
Latent Space ◽  
Multiple Sequences ◽  
Dynamic Shape

This paper introduces Deep4D a compact generative representation of shape and appearance from captured 4D volumetric video sequences of people. 4D volumetric video achieves highly realistic reproduction, replay and free-viewpoint rendering of actor performance from multiple view video acquisition systems. A deep generative network is trained on 4D video sequences of an actor performing multiple motions to learn a generative model of the dynamic shape and appearance. We demonstrate the proposed generative model can provide a compact encoded representation capable of high-quality synthesis of 4D volumetric video with two orders of magnitude compression. A variational encoder-decoder network is employed to learn an encoded latent space that maps from 3D skeletal pose to 4D shape and appearance. This enables high-quality 4D volumetric video synthesis to be driven by skeletal motion, including skeletal motion capture data. This encoded latent space supports the representation of multiple sequences with dynamic interpolation to transition between motions. Therefore we introduce Deep4D motion graphs, a direct application of the proposed generative representation. Deep4D motion graphs allow real-tiome interactive character animation whilst preserving the plausible realism of movement and appearance from the captured volumetric video. Deep4D motion graphs implicitly combine multiple captured motions from a unified representation for character animation from volumetric video, allowing novel character movements to be generated with dynamic shape and appearance detail.

Detecting “DeepFakes” in H.264 Video Data Using Compression Ghost Artifacts

2020 ◽  
Vol 2020 (4) ◽  
pp. 116-1-116-7
Author(s):  
Raphael Antonius Frick ◽  
Sascha Zmudzinski ◽  
Martin Steinebach
Keyword(s):  
Image Forensics ◽  
Video Data ◽  
Experimental Results ◽  
Video Sequences ◽  
The Internet ◽  
Video Content ◽  
High Quality ◽  
The Public

In recent years, the number of forged videos circulating on the Internet has immensely increased. Software and services to create such forgeries have become more and more accessible to the public. In this regard, the risk of malicious use of forged videos has risen. This work proposes an approach based on the Ghost effect knwon from image forensics for detecting forgeries in videos that can replace faces in video sequences or change the mimic of a face. The experimental results show that the proposed approach is able to identify forgery in high-quality encoded video content.

scholarly journals AI-Sketcher : A Deep Generative Model for Producing High-Quality Sketches

2019 ◽  
Vol 33 ◽  
pp. 2564-2571 ◽  
Author(s):  
Nan Cao ◽  
Xin Yan ◽  
Yang Shi ◽  
Chaoran Chen
Keyword(s):  
Artificial Intelligence ◽  
Large Scale ◽  
State Of The Art ◽  
Positional Information ◽  
Generative Model ◽  
Training Data ◽  
Creative Design ◽  
High Quality ◽  
User Input ◽  
Variational Autoencoder

Sketch drawings play an important role in assisting humans in communication and creative design since ancient period. This situation has motivated the development of artificial intelligence (AI) techniques for automatically generating sketches based on user input. Sketch-RNN, a sequence-to-sequence variational autoencoder (VAE) model, was developed for this purpose and known as a state-of-the-art technique. However, it suffers from limitations, including the generation of lowquality results and its incapability to support multi-class generations. To address these issues, we introduced AI-Sketcher, a deep generative model for generating high-quality multiclass sketches. Our model improves drawing quality by employing a CNN-based autoencoder to capture the positional information of each stroke at the pixel level. It also introduces an influence layer to more precisely guide the generation of each stroke by directly referring to the training data. To support multi-class sketch generation, we provided a conditional vector that can help differentiate sketches under various classes. The proposed technique was evaluated based on two large-scale sketch datasets, and results demonstrated its power in generating high-quality sketches.

Supervised Determined Source Separation with Multichannel Variational Autoencoder

2019 ◽  
Vol 31 (9) ◽  
pp. 1891-1914 ◽  
Author(s):  
Hirokazu Kameoka ◽  
Li Li ◽  
Shota Inoue ◽  
Shoji Makino
Keyword(s):  
Source Separation ◽  
Generative Model ◽  
Separation Technique ◽  
Separation Performance ◽  
Unknown Parameters ◽  
Baseline Method ◽  
Latent Space ◽  
Variational Autoencoder ◽  
Training Examples ◽  
Class Labels

This letter proposes a multichannel source separation technique, the multichannel variational autoencoder (MVAE) method, which uses a conditional VAE (CVAE) to model and estimate the power spectrograms of the sources in a mixture. By training the CVAE using the spectrograms of training examples with source-class labels, we can use the trained decoder distribution as a universal generative model capable of generating spectrograms conditioned on a specified class index. By treating the latent space variables and the class index as the unknown parameters of this generative model, we can develop a convergence-guaranteed algorithm for supervised determined source separation that consists of iteratively estimating the power spectrograms of the underlying sources, as well as the separation matrices. In experimental evaluations, our MVAE produced better separation performance than a baseline method.

Evaluating motion graphs for character animation

2007 ◽  
Vol 26 (4) ◽  
pp. 18 ◽  
Author(s):  
Paul S. A. Reitsma ◽  
Nancy S. Pollard
Keyword(s):  
Character Animation ◽  
Motion Graphs
Sign in / Sign up

Export Citation Format

Share Document