A unified generative model using generative adversarial network for activity recognition

2020 ◽  
Author(s):  
Mang Hong Chan ◽  
Mohd Halim Mohd Noor
Keyword(s):  
Activity Recognition ◽  
Generative Model ◽  
Generative Adversarial Network ◽  
Adversarial Network

scholarly journals Generative Model for Skeletal Human Movements Based on Conditional DC-GAN Applied to Pseudo-Images

Algorithms ◽  
2020 ◽  
Vol 13 (12) ◽  
pp. 319
Author(s):  
Wang Xi ◽  
Guillaume Devineau ◽  
Fabien Moutarde ◽  
Jie Yang
Keyword(s):  
Data Augmentation ◽  
Human Movement ◽  
Generative Models ◽  
Generative Model ◽  
Great Success ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Human Movements ◽  
Action Type ◽  
Skeleton Image

Generative models for images, audio, text, and other low-dimension data have achieved great success in recent years. Generating artificial human movements can also be useful for many applications, including improvement of data augmentation methods for human gesture recognition. The objective of this research is to develop a generative model for skeletal human movement, allowing to control the action type of generated motion while keeping the authenticity of the result and the natural style variability of gesture execution. We propose to use a conditional Deep Convolutional Generative Adversarial Network (DC-GAN) applied to pseudo-images representing skeletal pose sequences using tree structure skeleton image format. We evaluate our approach on the 3D skeletal data provided in the large NTU_RGB+D public dataset. Our generative model can output qualitatively correct skeletal human movements for any of the 60 action classes. We also quantitatively evaluate the performance of our model by computing Fréchet inception distances, which shows strong correlation to human judgement. To the best of our knowledge, our work is the first successful class-conditioned generative model for human skeletal motions based on pseudo-image representation of skeletal pose sequences.

scholarly journals Extreme Low-Resolution Activity Recognition Using a Super-Resolution-Oriented Generative Adversarial Network

Micromachines ◽  
2021 ◽  
Vol 12 (6) ◽  
pp. 670
Author(s):  
Mingzheng Hou ◽  
Song Liu ◽  
Jiliu Zhou ◽  
Yi Zhang ◽  
Ziliang Feng
Keyword(s):  
Activity Recognition ◽  
Large Scale ◽  
Super Resolution ◽  
General Activity ◽  
Low Resolution ◽  
Generative Adversarial Network ◽  
Video Clips ◽  
Adversarial Network ◽  
Efficient Recognition ◽  
Low Resolution Images

Activity recognition is a fundamental and crucial task in computer vision. Impressive results have been achieved for activity recognition in high-resolution videos, but for extreme low-resolution videos, which capture the action information at a distance and are vital for preserving privacy, the performance of activity recognition algorithms is far from satisfactory. The reason is that extreme low-resolution (e.g., 12 × 16 pixels) images lack adequate scene and appearance information, which is needed for efficient recognition. To address this problem, we propose a super-resolution-driven generative adversarial network for activity recognition. To fully take advantage of the latent information in low-resolution images, a powerful network module is employed to super-resolve the extremely low-resolution images with a large scale factor. Then, a general activity recognition network is applied to analyze the super-resolved video clips. Extensive experiments on two public benchmarks were conducted to evaluate the effectiveness of our proposed method. The results demonstrate that our method outperforms several state-of-the-art low-resolution activity recognition approaches.

scholarly journals Generative Enhancement of 3D Image Classifiers

2020 ◽  
Author(s):  
Michal Varga ◽  
Ján Jadlovský ◽  
Slávka Jadlovská
Keyword(s):  
Knowledge Sharing ◽  
Classification Accuracy ◽  
Network Architecture ◽  
Classification Performance ◽  
Generative Model ◽  
3D Image ◽  
Accuracy Improvement ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Generative Modeling

Abstract In this paper, we propose a methodology for generative enhancement of existing 3D image classifiers. This methodology is based on combining the strengths of both non-generative classifiers and generative modeling. Its purpose is to streamline the creation of new classifiers by embedding existing compatible classifiers in a generative network architecture. The demonstration of this process and evaluation of its effects is performed using a 3D convolutional classifier and its generative equivalent - a conditional generative adversarial network classifier. The results show that the generative model achieves greater classification performance, gaining a relative classification accuracy improvement of 7.43%. Improvement of accuracy is also present when compared to a plain convolutional classifier trained on a dataset augmented with examples produced by a trained generator. This suggests there is a desirable knowledge sharing within the hybrid discriminator-classifier network.

scholarly journals Generative Model for Skeletal Human Movements based on conditional DC-GAN applied to pseudo-images

2020 ◽  
Author(s):  
Wang Xi ◽  
Guillaume Devineau ◽  
Fabien Moutarde ◽  
Jie Yang
Keyword(s):  
Data Augmentation ◽  
Human Movement ◽  
Generative Models ◽  
Generative Model ◽  
Great Success ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Human Movements ◽  
Action Type ◽  
Skeleton Image

Generative models for images, audio, text and other low-dimension data have achieved great success in recent years. Generating artificial human movements can also be useful for many applications, including improvement of data augmentation methods for human gesture recognition. The object of this research is to develop a generative model for skeletal human movement, allowing to control the action type of generated motion while keeping the authenticity of the result and the natural style variability of gesture execution. We propose to use a conditional Deep Convolutional Generative Adversarial Network (DC-GAN) applied to pseudo-images representing skeletal pose sequences using Tree Structure Skeleton Image format. We evaluate our approach on the 3D-skeleton data provided in the large NTU RGB+D public dataset. Our generative model can output qualitatively correct skeletal human movements for any of its 60 action classes. We also quantitatively evaluate the performance of our model by computing Frechet Inception Distances, which shows strong correlation to human judgement. Up to our knowledge, our work is the first successful class-conditioned generative model for human skeletal motions based on pseudo-image representation of skeletal pose sequences.

scholarly journals BEGAN v3: Avoiding Mode Collapse in GANs Using Variational Inference

Electronics ◽  
2020 ◽  
Vol 9 (4) ◽  
pp. 688
Author(s):  
Sung-Wook Park ◽  
Jun-Ho Huh ◽  
Jong-Chan Kim
Keyword(s):  
Image Synthesis ◽  
Generative Model ◽  
Generative Adversarial Networks ◽  
Generative Adversarial Network ◽  
Compression Performance ◽  
Adversarial Network ◽  
Adversarial Networks ◽  
Latent Space ◽  
Boundary Equilibrium ◽  
Space Compression

In the field of deep learning, the generative model did not attract much attention until GANs (generative adversarial networks) appeared. In 2014, Google’s Ian Goodfellow proposed a generative model called GANs. GANs use different structures and objective functions from the existing generative model. For example, GANs use two neural networks: a generator that creates a realistic image, and a discriminator that distinguishes whether the input is real or synthetic. If there are no problems in the training process, GANs can generate images that are difficult even for experts to distinguish in terms of authenticity. Currently, GANs are the most researched subject in the field of computer vision, which deals with the technology of image style translation, synthesis, and generation, and various models have been unveiled. The issues raised are also improving one by one. In image synthesis, BEGAN (Boundary Equilibrium Generative Adversarial Network), which outperforms the previously announced GANs, learns the latent space of the image, while balancing the generator and discriminator. Nonetheless, BEGAN also has a mode collapse wherein the generator generates only a few images or a single one. Although BEGAN-CS (Boundary Equilibrium Generative Adversarial Network with Constrained Space), which was improved in terms of loss function, was introduced, it did not solve the mode collapse. The discriminator structure of BEGAN-CS is AE (AutoEncoder), which cannot create a particularly useful or structured latent space. Compression performance is not good either. In this paper, this characteristic of AE is considered to be related to the occurrence of mode collapse. Thus, we used VAE (Variational AutoEncoder), which added statistical techniques to AE. As a result of the experiment, the proposed model did not cause mode collapse but converged to a better state than BEGAN-CS.

scholarly journals Range-GAN: Range-Constrained Generative Adversarial Network for Conditioned Design Synthesis

2021 ◽  
Author(s):  
Amin Heyrani Nobari ◽  
Wei Chen ◽  
Faez Ahmed
Keyword(s):  
Generative Model ◽  
Inverse Design ◽  
Data Driven ◽  
Generative Adversarial Network ◽  
Design Problems ◽  
Design Synthesis ◽  
Adversarial Network ◽  
Average Improvement ◽  
Inverse Design Problem ◽  
Alternative Designs

Abstract Typical engineering design tasks require the effort to modify designs iteratively until they meet certain constraints, i.e., performance or attribute requirements. Past work has proposed ways to solve the inverse design problem, where desired designs are directly generated from specified requirements, thus avoid the trial and error process. Among those approaches, the conditional deep generative model shows great potential since 1) it works for complex high-dimensional designs and 2) it can generate multiple alternative designs given any condition. In this work, we propose a conditional deep generative model, Range-GAN, to achieve automatic design synthesis subject to range constraints. The proposed model addresses the sparse conditioning issue in data-driven inverse design problems by introducing a label-aware self-augmentation approach. We also propose a new uniformity loss to ensure generated designs evenly cover the given requirement range. Through a real-world example of constrained 3D shape generation, we show that the label-aware self-augmentation leads to an average improvement of 14% on the constraint satisfaction for generated 3D shapes, and the uniformity loss leads to a 125% average increase on the uniformity of generated shapes’ attributes. This work laid the foundation for data-driven inverse design problems where we consider range constraints and there are sparse regions in the condition space. For further information and code for this paper please refer to http://decode.mit.edu/projects/rangegan/.

scholarly journals Conditional LSTM-GAN for Melody Generation from Lyrics

2021 ◽  
Vol 17 (1) ◽  
pp. 1-20
Author(s):  
Yi Yu ◽  
Abhishek Srivastava ◽  
Simon Canales
Keyword(s):  
Short Term Memory ◽  
Generative Model ◽  
Research Progress ◽  
Short Term ◽  
Large Dataset ◽  
Generative Adversarial Network ◽  
Term Memory ◽  
Adversarial Network ◽  
Challenging Research ◽  
Long Short Term Memory

Melody generation from lyrics has been a challenging research issue in the field of artificial intelligence and music, which enables us to learn and discover latent relationships between interesting lyrics and accompanying melodies. Unfortunately, the limited availability of a paired lyrics–melody dataset with alignment information has hindered the research progress. To address this problem, we create a large dataset consisting of 12,197 MIDI songs each with paired lyrics and melody alignment through leveraging different music sources where alignment relationship between syllables and music attributes is extracted. Most importantly, we propose a novel deep generative model, conditional Long Short-Term Memory (LSTM)–Generative Adversarial Network for melody generation from lyrics, which contains a deep LSTM generator and a deep LSTM discriminator both conditioned on lyrics. In particular, lyrics-conditioned melody and alignment relationship between syllables of given lyrics and notes of predicted melody are generated simultaneously. Extensive experimental results have proved the effectiveness of our proposed lyrics-to-melody generative model, where plausible and tuneful sequences can be inferred from lyrics.

scholarly journals A Domain-Independent Generative Adversarial Network for Activity Recognition Using WiFi CSI Data

Sensors ◽  
2021 ◽  
Vol 21 (23) ◽  
pp. 7852
Author(s):  
Augustinas Zinys ◽  
Bram van Berlo ◽  
Nirvana Meratnia
Keyword(s):  
Data Collection ◽  
Activity Recognition ◽  
Vital Signs ◽  
Learning Task ◽  
Model Complexity ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Processing Module ◽  
Device Free ◽  
Domain Independent

Over the past years, device-free sensing has received considerable attention due to its unobtrusiveness. In this regard, context recognition using WiFi Channel State Information (CSI) data has gained popularity, and various techniques have been proposed that combine unobtrusive sensing and deep learning to accurately detect various contexts ranging from human activities to gestures. However, research has shown that the performance of these techniques significantly degrades due to change in various factors including sensing environment, data collection configuration, diversity of target subjects, and target learning task (e.g., activities, gestures, emotions, vital signs). This problem, generally known as the domain change problem, is typically addressed by collecting more data and learning the data distribution that covers multiple factors impacting the performance. However, activity recognition data collection is a very labor-intensive and time consuming task, and there are too many known and unknown factors impacting WiFi CSI signals. In this paper, we propose a domain-independent generative adversarial network for WiFi CSI based activity recognition in combination with a simplified data pre-processing module. Our evaluation results show superiority of our proposed approach compared to the state of the art in terms of increased robustness against domain change, higher accuracy of activity recognition, and reduced model complexity.

scholarly journals AOGAN: A generative adversarial network for screen space ambient occlusion

2022 ◽  
Author(s):  
Lei Ren ◽  
Ying Song
Keyword(s):  
Light Intensity ◽  
Real Time ◽  
Generative Model ◽  
Attention Mechanism ◽  
Generative Adversarial Network ◽  
Learning Framework ◽  
Adversarial Network ◽  
Ambient Occlusion ◽  
Complex Scenes ◽  
Screen Space

AbstractAmbient occlusion (AO) is a widely-used real-time rendering technique which estimates light intensity on visible scene surfaces. Recently, a number of learning-based AO approaches have been proposed, which bring a new angle to solving screen space shading via a unified learning framework with competitive quality and speed. However, most such methods have high error for complex scenes or tend to ignore details. We propose an end-to-end generative adversarial network for the production of realistic AO, and explore the importance of perceptual loss in the generative model to AO accuracy. An attention mechanism is also described to improve the accuracy of details, whose effectiveness is demonstrated on a wide variety of scenes.

Sign in / Sign up

Export Citation Format

Share Document