Generation of Pedestrian Crossing Scenarios Using Ped-Cross Generative Adversarial Network

The safety of vulnerable road users is of paramount importance as transport moves towards fully automated driving. The richness of real-world data required for testing autonomous vehicles is limited and furthermore, available data do not present a fair representation of different scenarios and rare events. Before deploying autonomous vehicles publicly, their abilities must reach a safety threshold, not least with regards to vulnerable road users, such as pedestrians. In this paper, we present a novel Generative Adversarial Networks named the Ped-Cross GAN. Ped-Cross GAN is able to generate crossing sequences of pedestrians in the form of human pose sequences. The Ped-Cross GAN is trained with the Pedestrian Scenario dataset. The novel Pedestrian Scenario dataset, derived from existing datasets, enables training on richer pedestrian scenarios. We demonstrate an example of its use through training and testing the Ped-Cross GAN. The results show that the Ped-Cross GAN is able to generate new crossing scenarios that are of the same distribution from those contained in the Pedestrian Scenario dataset. Having a method with these capabilities is important for the future of transport, as it will allow for the adequate testing of Connected and Autonomous Vehicles on how they correctly perceive the intention of pedestrians crossing the street, ultimately leading to fewer pedestrian casualties on our roads.

Download Full-text

Learning Representations of Natural Language Texts with Generative Adversarial Networks at Document, Sentence, and Aspect Level

Algorithms ◽

10.3390/a11100164 ◽

2018 ◽

Vol 11 (10) ◽

pp. 164 ◽

Cited By ~ 4

Author(s):

Aggeliki Vlachostergiou ◽

George Caridakis ◽

Phivos Mylonas ◽

Andreas Stafylopatis

Keyword(s):

Natural Language ◽

Generative Models ◽

Generative Adversarial Networks ◽

Real World Data ◽

Generative Adversarial Network ◽

Feature Representations ◽

Adversarial Network ◽

Adversarial Networks ◽

Learning Tasks ◽

Potential Applications

The ability to learn robust, resizable feature representations from unlabeled data has potential applications in a wide variety of machine learning tasks. One way to create such representations is to train deep generative models that can learn to capture the complex distribution of real-world data. Generative adversarial network (GAN) approaches have shown impressive results in producing generative models of images, but relatively little work has been done on evaluating the performance of these methods for the learning representation of natural language, both in supervised and unsupervised settings at the document, sentence, and aspect level. Extensive research validation experiments were performed by leveraging the 20 Newsgroups corpus, the Movie Review (MR) Dataset, and the Finegrained Sentiment Dataset (FSD). Our experimental analysis suggests that GANs can successfully learn representations of natural language texts at all three aforementioned levels.

Download Full-text

VRUNet: Multi-Task Learning Model for Intent Prediction of Vulnerable Road Users

Electronic Imaging ◽

10.2352/issn.2470-1173.2020.16.avm-109 ◽

2020 ◽

Vol 2020 (16) ◽

pp. 109-1-109-10 ◽

Cited By ~ 1

Author(s):

Adithya Ranga ◽

Filippo Giruzzi ◽

Jagdish Bhanushali ◽

Emilie Wirbel ◽

Patrick Pérez ◽

...

Keyword(s):

Autonomous Vehicles ◽

Learning Model ◽

Task Learning ◽

Road Users ◽

Vulnerable Road Users ◽

Naturalistic Driving ◽

Human Pose ◽

Intent Prediction ◽

Future Path

Advanced perception and path planning are at the core for any self-driving vehicle. Autonomous vehicles need to understand the scene and intentions of other road users for safe motion planning. For urban use cases it is very important to perceive and predict the intentions of pedestrians, cyclists, scooters, etc., classified as vulnerable road users (VRU). Intent is a combination of pedestrian activities and long term trajectories defining their future motion. In this paper we propose a multi-task learning model to predict pedestrian actions, crossing intent and forecast their future path from video sequences. We have trained the model on naturalistic driving open-source JAAD [1] dataset, which is rich in behavioral annotations and real world scenarios. Experimental results show state-of-the-art performance on JAAD dataset and how we can benefit from jointly learning and predicting actions and trajectories using 2D human pose features and scene context.

Download Full-text

ORGANIC (1).pdf

10.26434/chemrxiv.5309668.v1 ◽

2017 ◽

Author(s):

Benjamin Sanchez-Lengeling ◽

Carlos Outeiral ◽

Gabriel L. Guimaraes ◽

Alan Aspuru-Guzik

Keyword(s):

Machine Learning ◽

Learning Community ◽

Chemical Species ◽

Material Design ◽

Organic Photovoltaic ◽

Generative Adversarial Networks ◽

Generative Adversarial Network ◽

Adversarial Network ◽

Adversarial Networks ◽

Photovoltaic Material

Molecular discovery seeks to generate chemical species tailored to very specific needs. In this paper, we present ORGANIC, a framework based on Objective-Reinforced Generative Adversarial Networks (ORGAN), capable of producing a distribution over molecular space that matches with a certain set of desirable metrics. This methodology combines two successful techniques from the machine learning community: a Generative Adversarial Network (GAN), to create non-repetitive sensible molecular species, and Reinforcement Learning (RL), to bias this generative distribution towards certain attributes. We explore several applications, from optimization of random physicochemical properties to candidates for drug discovery and organic photovoltaic material design.

Download Full-text

Restoring Raindrops Using Attentive Generative Adversarial Networks

Applied Sciences ◽

10.3390/app11157034 ◽

2021 ◽

Vol 11 (15) ◽

pp. 7034

Author(s):

Hee-Deok Yang

Keyword(s):

Weather Conditions ◽

Recurrent Network ◽

Generative Adversarial Networks ◽

Navigation Systems ◽

Vision Systems ◽

Generative Adversarial Network ◽

Network Layers ◽

Adversarial Network ◽

Adversarial Networks ◽

Outdoor Vision

Artificial intelligence technologies and vision systems are used in various devices, such as automotive navigation systems, object-tracking systems, and intelligent closed-circuit televisions. In particular, outdoor vision systems have been applied across numerous fields of analysis. Despite their widespread use, current systems work well under good weather conditions. They cannot account for inclement conditions, such as rain, fog, mist, and snow. Images captured under inclement conditions degrade the performance of vision systems. Vision systems need to detect, recognize, and remove noise because of rain, snow, and mist to boost the performance of the algorithms employed in image processing. Several studies have targeted the removal of noise resulting from inclement conditions. We focused on eliminating the effects of raindrops on images captured with outdoor vision systems in which the camera was exposed to rain. An attentive generative adversarial network (ATTGAN) was used to remove raindrops from the images. This network was composed of two parts: an attentive-recurrent network and a contextual autoencoder. The ATTGAN generated an attention map to detect rain droplets. A de-rained image was generated by increasing the number of attentive-recurrent network layers. We increased the number of visual attentive-recurrent network layers in order to prevent gradient sparsity so that the entire generation was more stable against the network without preventing the network from converging. The experimental results confirmed that the extended ATTGAN could effectively remove various types of raindrops from images.

Download Full-text

Prediction and analysis of multiple protein lysine modified sites based on conditional wasserstein generative adversarial networks

BMC Bioinformatics ◽

10.1186/s12859-021-04101-y ◽

2021 ◽

Vol 22 (1) ◽

Author(s):

Yingxi Yang ◽

Hui Wang ◽

Wen Li ◽

Xiaobo Wang ◽

Shizhao Wei ◽

...

Keyword(s):

Correlation Coefficient ◽

Sequence Data ◽

Rapid Development ◽

Pearson Correlation ◽

Structural Features ◽

Generative Adversarial Networks ◽

Post Translational Modification ◽

Generative Adversarial Network ◽

Data Imbalance ◽

Adversarial Network

Abstract Background Protein post-translational modification (PTM) is a key issue to investigate the mechanism of protein’s function. With the rapid development of proteomics technology, a large amount of protein sequence data has been generated, which highlights the importance of the in-depth study and analysis of PTMs in proteins. Method We proposed a new multi-classification machine learning pipeline MultiLyGAN to identity seven types of lysine modified sites. Using eight different sequential and five structural construction methods, 1497 valid features were remained after the filtering by Pearson correlation coefficient. To solve the data imbalance problem, Conditional Generative Adversarial Network (CGAN) and Conditional Wasserstein Generative Adversarial Network (CWGAN), two influential deep generative methods were leveraged and compared to generate new samples for the types with fewer samples. Finally, random forest algorithm was utilized to predict seven categories. Results In the tenfold cross-validation, accuracy (Acc) and Matthews correlation coefficient (MCC) were 0.8589 and 0.8376, respectively. In the independent test, Acc and MCC were 0.8549 and 0.8330, respectively. The results indicated that CWGAN better solved the existing data imbalance and stabilized the training error. Alternatively, an accumulated feature importance analysis reported that CKSAAP, PWM and structural features were the three most important feature-encoding schemes. MultiLyGAN can be found at https://github.com/Lab-Xu/MultiLyGAN. Conclusions The CWGAN greatly improved the predictive performance in all experiments. Features derived from CKSAAP, PWM and structure schemes are the most informative and had the greatest contribution to the prediction of PTM.

Download Full-text

Stochastic Restoration of Heavily Compressed Musical Audio Using Generative Adversarial Networks

Electronics ◽

10.3390/electronics10111349 ◽

2021 ◽

Vol 10 (11) ◽

pp. 1349

Author(s):

Stefan Lattner ◽

Javier Nistal

Keyword(s):

Data Storage ◽

Audio Signal ◽

Human Perception ◽

Generative Adversarial Networks ◽

Audio Signals ◽

Generative Adversarial Network ◽

Adversarial Network ◽

Extensive Evaluation ◽

Listening Tests ◽

Musical Audio

Lossy audio codecs compress (and decompress) digital audio streams by removing information that tends to be inaudible in human perception. Under high compression rates, such codecs may introduce a variety of impairments in the audio signal. Many works have tackled the problem of audio enhancement and compression artifact removal using deep-learning techniques. However, only a few works tackle the restoration of heavily compressed audio signals in the musical domain. In such a scenario, there is no unique solution for the restoration of the original signal. Therefore, in this study, we test a stochastic generator of a Generative Adversarial Network (GAN) architecture for this task. Such a stochastic generator, conditioned on highly compressed musical audio signals, could one day generate outputs indistinguishable from high-quality releases. Therefore, the present study may yield insights into more efficient musical data storage and transmission. We train stochastic and deterministic generators on MP3-compressed audio signals with 16, 32, and 64 kbit/s. We perform an extensive evaluation of the different experiments utilizing objective metrics and listening tests. We find that the models can improve the quality of the audio signals over the MP3 versions for 16 and 32 kbit/s and that the stochastic generators are capable of generating outputs that are closer to the original signals than those of the deterministic generators.

Download Full-text

Remote Sensing Image Dataset Expansion Based on Generative Adversarial Networks with Modified Shuffle Attention

Sensors ◽

10.3390/s21144867 ◽

2021 ◽

Vol 21 (14) ◽

pp. 4867

Author(s):

Lu Chen ◽

Hongjun Wang ◽

Xianghao Meng

Keyword(s):

Remote Sensing ◽

Neural Networks ◽

Image Processing ◽

Remote Sensing Image ◽

Generative Adversarial Networks ◽

Generative Adversarial Network ◽

Evaluation Indexes ◽

Adversarial Network ◽

Remote Sensing Image Processing ◽

Data Expansion

With the development of science and technology, neural networks, as an effective tool in image processing, play an important role in gradual remote-sensing image-processing. However, the training of neural networks requires a large sample database. Therefore, expanding datasets with limited samples has gradually become a research hotspot. The emergence of the generative adversarial network (GAN) provides new ideas for data expansion. Traditional GANs either require a large number of input data, or lack detail in the pictures generated. In this paper, we modify a shuffle attention network and introduce it into GAN to generate higher quality pictures with limited inputs. In addition, we improved the existing resize method and proposed an equal stretch resize method to solve the problem of image distortion caused by different input sizes. In the experiment, we also embed the newly proposed coordinate attention (CA) module into the backbone network as a control test. Qualitative indexes and six quantitative evaluation indexes were used to evaluate the experimental results, which show that, compared with other GANs used for picture generation, the modified Shuffle Attention GAN proposed in this paper can generate more refined and high-quality diversified aircraft pictures with more detailed features of the object under limited datasets.

Download Full-text

Enhanced network optimized generative adversarial network for image enhancement

Multimedia Tools and Applications ◽

10.1007/s11042-020-10310-z ◽

2021 ◽

Author(s):

Lingyu Yan ◽

Jiarun Fu ◽

Chunzhi Wang ◽

Zhiwei Ye ◽

Hongwei Chen ◽

...

Keyword(s):

Image Enhancement ◽

Image Recognition ◽

Generative Adversarial Networks ◽

Low Light ◽

Generative Adversarial Network ◽

Adversarial Network ◽

Adversarial Networks ◽

Enhancement Method ◽

New Space ◽

Traditional Image

AbstractWith the development of image recognition technology, face, body shape, and other factors have been widely used as identification labels, which provide a lot of convenience for our daily life. However, image recognition has much higher requirements for image conditions than traditional identification methods like a password. Therefore, image enhancement plays an important role in the process of image analysis for images with noise, among which the image of low-light is the top priority of our research. In this paper, a low-light image enhancement method based on the enhanced network module optimized Generative Adversarial Networks(GAN) is proposed. The proposed method first applied the enhancement network to input the image into the generator to generate a similar image in the new space, Then constructed a loss function and minimized it to train the discriminator, which is used to compare the image generated by the generator with the real image. We implemented the proposed method on two image datasets (DPED, LOL), and compared it with both the traditional image enhancement method and the deep learning approach. Experiments showed that our proposed network enhanced images have higher PNSR and SSIM, the overall perception of relatively good quality, demonstrating the effectiveness of the method in the aspect of low illumination image enhancement.

Download Full-text

CDL-GAN: Contrastive Distance Learning Generative Adversarial Network for Image Generation

Applied Sciences ◽

10.3390/app11041380 ◽

2021 ◽

Vol 11 (4) ◽

pp. 1380

Author(s):

Yingbo Zhou ◽

Pengcheng Zhao ◽

Weiqin Tong ◽

Yongxin Zhu

Keyword(s):

Distance Learning ◽

Feature Learning ◽

Image Synthesis ◽

Image Feature ◽

Generative Adversarial Networks ◽

Image Generation ◽

Generative Adversarial Network ◽

Feature Representations ◽

Adversarial Network ◽

Public Datasets

While Generative Adversarial Networks (GANs) have shown promising performance in image generation, they suffer from numerous issues such as mode collapse and training instability. To stabilize GAN training and improve image synthesis quality with diversity, we propose a simple yet effective approach as Contrastive Distance Learning GAN (CDL-GAN) in this paper. Specifically, we add Consistent Contrastive Distance (CoCD) and Characteristic Contrastive Distance (ChCD) into a principled framework to improve GAN performance. The CoCD explicitly maximizes the ratio of the distance between generated images and the increment between noise vectors to strengthen image feature learning for the generator. The ChCD measures the sampling distance of the encoded images in Euler space to boost feature representations for the discriminator. We model the framework by employing Siamese Network as a module into GANs without any modification on the backbone. Both qualitative and quantitative experiments conducted on three public datasets demonstrate the effectiveness of our method.

Download Full-text

Continuous missing data imputation with incomplete dataset by generative adversarial networks–based unsupervised learning for long-term bridge health monitoring

Structural Health Monitoring ◽

10.1177/14759217211021942 ◽

2021 ◽

pp. 147592172110219

Author(s):

Huachen Jiang ◽

Chunfeng Wan ◽

Kang Yang ◽

Youliang Ding ◽

Songtao Xue

Keyword(s):

Missing Data ◽

Health Monitoring ◽

Signal Transmission ◽

Imputation Accuracy ◽

Generative Adversarial Networks ◽

Data Imputation ◽

Sensor Failure ◽

Generative Adversarial Network ◽

Missing Data Imputation ◽

Adversarial Network

Wireless sensors are the key components of structural health monitoring systems. During the signal transmission, sensor failure is inevitable, among which, data loss is the most common type. Missing data problem poses a huge challenge to the consequent damage detection and condition assessment, and therefore, great importance should be attached. Conventional missing data imputation basically adopts the correlation-based method, especially for strain monitoring data. However, such methods often require delicate model selection, and the correlations for vehicle-induced strains are much harder to be captured compared with temperature-induced strains. In this article, a novel data-driven generative adversarial network (GAN) for imputing missing strain response is proposed. As opposed to traditional ways where correlations for inter-strains are explicitly modeled, the proposed method directly imputes the missing data considering the spatial–temporal relationships with other strain sensors based on the remaining observed data. Furthermore, the intact and complete dataset is not even necessary during the training process, which shows another great superiority over the model-based imputation method. The proposed method is implemented and verified on a real concrete bridge. In order to demonstrate the applicability and robustness of the GAN, imputation for single and multiple sensors is studied. Results show the proposed method provides an excellent performance of imputation accuracy and efficiency.

Download Full-text