Pixel to Stroke Sketch Generation Using Reinforcement Learning

2019 ◽  
Author(s):  
Haizhou Wang ◽  
Conrad Tucker
Keyword(s):  
Reinforcement Learning ◽  
Engineering Design ◽  
Random Noise ◽  
Color Images ◽  
Training Data ◽  
High Fidelity ◽  
Reference Images

Abstract Many engineering design tasks involve creating early conceptual sketches that do not require exact dimensions. Although some previous works focus on automatically generating sketches from reference images, many of them output exactly the same objects as the reference images. There are also models that generate sketches from scratch, which can be divided into pixel-based and stroke-based methods. Pixel-based methods generate sketches as a whole, without any information of the strokes, while stroke-based methods generate sketches by outputting strokes in a sequential manner. Pixel-based methods are frequently used to generate realistic color images. Although the pixel-based methods are more popular, stroke-based methods have the advantages to scale to a larger dimension without losing high fidelity. An image generated from stroke-based methods has only strokes on the canvas, resulting in no random noise in the blank areas of the canvas. However, one challenge in the engineering design community is that most of the sketches are saved as pixel-based images. Furthermore, many non-pixel-based methods rely on stroke-based training data, making them ill-suited for generating design conceptual sketches. In order to overcome these limitations, the authors proposed an agent that can learn from pixel-based images and generate stroke-based images. An advantage of such an agent is the ability to utilize pixel-based training data that is abundant in design repositories, to train stroke-based methods that are typically constrained by the lack of access to stroke-based training data.

Transfer Reinforcement Learning for Autonomous Driving

2021 ◽  
Vol 31 (3) ◽  
pp. 1-26
Author(s):  
Aravind Balakrishnan ◽  
Jaeyoung Lee ◽  
Ashish Gaurav ◽  
Krzysztof Czarnecki ◽  
Sean Sedwards
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Transfer Problem ◽  
Autonomous Driving ◽  
High Fidelity ◽  
Rule Based ◽  
High Level ◽  
Real Vehicle

Reinforcement learning (RL) is an attractive way to implement high-level decision-making policies for autonomous driving, but learning directly from a real vehicle or a high-fidelity simulator is variously infeasible. We therefore consider the problem of transfer reinforcement learning and study how a policy learned in a simple environment using WiseMove can be transferred to our high-fidelity simulator, W ise M ove . WiseMove is a framework to study safety and other aspects of RL for autonomous driving. W ise M ove accurately reproduces the dynamics and software stack of our real vehicle. We find that the accurately modelled perception errors in W ise M ove contribute the most to the transfer problem. These errors, when even naively modelled in WiseMove , provide an RL policy that performs better in W ise M ove than a hand-crafted rule-based policy. Applying domain randomization to the environment in WiseMove yields an even better policy. The final RL policy reduces the failures due to perception errors from 10% to 2.75%. We also observe that the RL policy has significantly less reliance on velocity compared to the rule-based policy, having learned that its measurement is unreliable.

scholarly journals Three-Dimensional Liver Image Segmentation Using Generative Adversarial Networks Based on Feature Restoration

2022 ◽  
Vol 8 ◽  
Author(s):  
Runnan He ◽  
Shiqi Xu ◽  
Yashu Liu ◽  
Qince Li ◽  
Yang Liu ◽  
...  
Keyword(s):  
Medical Imaging ◽  
Random Noise ◽  
Three Dimensional ◽  
Poor Quality ◽  
Training Data ◽  
Generative Adversarial Networks ◽  
Liver Segmentation ◽  
Deep Convolutional Neural Networks ◽  
Adversarial Networks ◽  
Liver Region

Medical imaging provides a powerful tool for medical diagnosis. In the process of computer-aided diagnosis and treatment of liver cancer based on medical imaging, accurate segmentation of liver region from abdominal CT images is an important step. However, due to defects of liver tissue and limitations of CT imaging procession, the gray level of liver region in CT image is heterogeneous, and the boundary between the liver and those of adjacent tissues and organs is blurred, which makes the liver segmentation an extremely difficult task. In this study, aiming at solving the problem of low segmentation accuracy of the original 3D U-Net network, an improved network based on the three-dimensional (3D) U-Net, is proposed. Moreover, in order to solve the problem of insufficient training data caused by the difficulty of acquiring labeled 3D data, an improved 3D U-Net network is embedded into the framework of generative adversarial networks (GAN), which establishes a semi-supervised 3D liver segmentation optimization algorithm. Finally, considering the problem of poor quality of 3D abdominal fake images generated by utilizing random noise as input, deep convolutional neural networks (DCNN) based on feature restoration method is designed to generate more realistic fake images. By testing the proposed algorithm on the LiTS-2017 and KiTS19 dataset, experimental results show that the proposed semi-supervised 3D liver segmentation method can greatly improve the segmentation performance of liver, with a Dice score of 0.9424 outperforming other methods.

scholarly journals Generating stable molecules using imitation and reinforcement learning

2021 ◽  
Author(s):  
Søren Ager Meldgaard ◽  
Jonas Köhler ◽  
Henrik Lund Mortensen ◽  
Mads-Peter Verner Christiansen ◽  
Frank Noé ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Chemical Space ◽  
Training Data ◽  
Graph Representation ◽  
Imitation Learning ◽  
Training Set ◽  
Machine Learning Methods ◽  
Multiple Copies ◽  
The Stability ◽  
3D Information

Abstract Chemical space is routinely explored by machine learning methods to discover interesting molecules, before time-consuming experimental synthesizing is attempted. However, these methods often rely on a graph representation, ignoring 3D information necessary for determining the stability of the molecules. We propose a reinforcement learning approach for generating molecules in cartesian coordinates allowing for quantum chemical prediction of the stability. To improve sample-efficiency we learn basic chemical rules from imitation learning on the GDB-11 database to create an initial model applicable for all stoichiometries. We then deploy multiple copies of the model conditioned on a specific stoichiometry in a reinforcement learning setting. The models correctly identify low energy molecules in the database and produce novel isomers not found in the training set. Finally, we apply the model to larger molecules to show how reinforcement learning further refines the imitation learning model in domains far from the training data.

Gaussian Process Emulation for Big Data in Data-Driven Metamaterials Design

2019 ◽  
Author(s):  
Ramin Bostanabad ◽  
Yu-Chin Chan ◽  
Liwei Wang ◽  
Ping Zhu ◽  
Wei Chen
Keyword(s):  
Big Data ◽  
Gaussian Process ◽  
Engineering Design ◽  
Training Data ◽  
Data Driven ◽  
Training Dataset ◽  
Massive Datasets ◽  
Unit Cells ◽  
Gaussian Process Emulation ◽  
Novel Method

Abstract Our main contribution is to introduce a novel method for Gaussian process (GP) modeling of massive datasets. The key idea is to build an ensemble of independent GPs that use the same hyperparameters but distribute the entire training dataset among themselves. This is motivated by our observation that estimates of the GP hyperparameters change negligibly as the size of the training data exceeds a certain level, which can be found in a systematic way. For inference, the predictions from all GPs in the ensemble are pooled to efficiently exploit the entire training dataset for prediction. We name our modeling approach globally approximate Gaussian process (GAGP), which, unlike most largescale supervised learners such as neural networks and trees, is easy to fit and can interpret the model behavior. These features make it particularly useful in engineering design with big data. We use analytical examples to demonstrate that GAGP achieves very high predictive power that matches or exceeds that of state-of-the-art machine learning methods. We illustrate the application of GAGP in engineering design with a problem on data-driven metamaterials design where it is used to link reduced-dimension geometrical descriptors of unit cells and their properties. Searching for new unit cell designs with desired properties is then accomplished by employing GAGP in inverse optimization.

Multi-Context Generation in Virtual Reality Environments Using Deep Reinforcement Learning

2020 ◽  
Author(s):  
James Cunningham ◽  
Christian Lopez ◽  
Omar Ashour ◽  
Conrad S. Tucker
Keyword(s):  
Virtual Reality ◽  
Reinforcement Learning ◽  
Virtual Environments ◽  
Probability Distributions ◽  
Automatic Generation ◽  
Grocery Store ◽  
Training Data ◽  
Learning Approaches ◽  
Common Concept

Abstract In this work, a Deep Reinforcement Learning (RL) approach is proposed for Procedural Content Generation (PCG) that seeks to automate the generation of multiple related virtual reality (VR) environments for enhanced personalized learning. This allows for the user to be exposed to multiple virtual scenarios that demonstrate a consistent theme, which is especially valuable in an educational context. RL approaches to PCG offer the advantage of not requiring training data, as opposed to other PCG approaches that employ supervised learning approaches. This work advances the state of the art in RL-based PCG by demonstrating the ability to generate a diversity of contexts in order to teach the same underlying concept. A case study is presented that demonstrates the feasibility of the proposed RL-based PCG method using examples of probability distributions in both manufacturing facility and grocery store virtual environments. The method demonstrated in this paper has the potential to enable the automatic generation of a variety of virtual environments that are connected by a common concept or theme.

Design-Oriented Multifidelity Fluid Simulation Using Machine Learned Fidelity Mapping

2019 ◽  
Author(s):  
Kazuko Fuchi ◽  
Eric M. Wolf ◽  
David S. Makhija ◽  
Nathan A. Wukie ◽  
Christopher R. Schrock ◽  
...  
Keyword(s):  
Machine Learning ◽  
Learning Algorithm ◽  
Fluid Simulation ◽  
Machine Learning Algorithms ◽  
Training Data ◽  
Supervised Machine Learning ◽  
High Fidelity ◽  
Computational Domain ◽  
Symmetry Properties ◽  
High Fidelity Simulations

Abstract A machine learning algorithm that performs multifidelity domain decomposition is introduced. While the design of complex systems can be facilitated by numerical simulations, the determination of appropriate physics couplings and levels of model fidelity can be challenging. The proposed method automatically divides the computational domain into subregions and assigns required fidelity level, using a small number of high fidelity simulations to generate training data and low fidelity solutions as input data. Unsupervised and supervised machine learning algorithms are used to correlate features from low fidelity solutions to fidelity assignment. The effectiveness of the method is demonstrated in a problem of viscous fluid flow around a cylinder at Re ≈ 20. Ling et al. built physics-informed invariance and symmetry properties into machine learning models and demonstrated improved model generalizability. Along these lines, we avoid using problem dependent features such as coordinates of sample points, object geometry or flow conditions as explicit inputs to the machine learning model. Use of pointwise flow features generates large data sets from only one or two high fidelity simulations, and the fidelity predictor model achieved 99.5% accuracy at training points. The trained model was shown to be capable of predicting a fidelity map for a problem with an altered cylinder radius. A significant improvement in the prediction performance was seen when inputs are expanded to include multiscale features that incorporate neighborhood information.

scholarly journals MetaLight: Value-Based Meta-Reinforcement Learning for Traffic Signal Control

2020 ◽  
Vol 34 (01) ◽  
pp. 1153-1160 ◽  
Author(s):  
Xinshi Zang ◽  
Huaxiu Yao ◽  
Guanjie Zheng ◽  
Nan Xu ◽  
Kai Xu ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Real World ◽  
Learning Algorithm ◽  
Traffic Signal ◽  
Training Data ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Individual Level ◽  
Real World Datasets ◽  
Reinforcement Learning Models

Using reinforcement learning for traffic signal control has attracted increasing interests recently. Various value-based reinforcement learning methods have been proposed to deal with this classical transportation problem and achieved better performances compared with traditional transportation methods. However, current reinforcement learning models rely on tremendous training data and computational resources, which may have bad consequences (e.g., traffic jams or accidents) in the real world. In traffic signal control, some algorithms have been proposed to empower quick learning from scratch, but little attention is paid to learning by transferring and reusing learned experience. In this paper, we propose a novel framework, named as MetaLight, to speed up the learning process in new scenarios by leveraging the knowledge learned from existing scenarios. MetaLight is a value-based meta-reinforcement learning workflow based on the representative gradient-based meta-learning algorithm (MAML), which includes periodically alternate individual-level adaptation and global-level adaptation. Moreover, MetaLight improves the-state-of-the-art reinforcement learning model FRAP in traffic signal control by optimizing its model structure and updating paradigm. The experiments on four real-world datasets show that our proposed MetaLight not only adapts more quickly and stably in new traffic scenarios, but also achieves better performance.

scholarly journals Spatial Grammar-Based Recurrent Neural Network for Design Form and Behavior Optimization

2019 ◽  
Vol 141 (12) ◽  
Author(s):  
Gary M. Stump ◽  
Simon W. Miller ◽  
Michael A. Yukish ◽  
Timothy W. Simpson ◽  
Conrad Tucker
Keyword(s):  
Reinforcement Learning ◽  
Degrees Of Freedom ◽  
Parametric Design ◽  
Control Policy ◽  
Training Data ◽  
High Performing ◽  
First Case ◽  
Spatial Grammar ◽  
And Behavior

Abstract A novel method has been developed to optimize both the form and behavior of complex systems. The method uses spatial grammars embodied in character-recurrent neural networks (char-RNNs) to define the system including actuator numbers and degrees of freedom, reinforcement learning to optimize actuator behavior, and physics-based simulation systems to determine performance and provide (re)training data for the char-RNN. Compared to parametric design optimization with fixed numbers of inputs, using grammars and char-RNNs allows for a more complex, combinatorial infinite design space. In the proposed method, the char-RNN is first trained to learn a spatial grammar that defines the assembly layout, component geometries, material properties, and arbitrary numbers and degrees of freedom of actuators. Next, generated designs are evaluated using a physics-based environment, with an inner optimization loop using reinforcement learning to determine the best control policy for the actuators. The resulting design is thus optimized for both form and behavior, generated by a char-RNN embodying a high-performing grammar. Two evaluative case studies are presented using the design of the modular sailing craft. The first case study optimizes the design without actuated surfaces, allowing the char-RNN to understand the semantics of high-performing designs. The second case study extends the first by incorporating controllable actuators requiring an inner loop behavioral optimization. The implications of the results are discussed along with the ongoing and future work.

Attenuation of random noise using denoising convolutional neural networks

2019 ◽  
Vol 7 (3) ◽  
pp. SE269-SE280
Author(s):  
Xu Si ◽  
Yijun Yuan ◽  
Tinghua Si ◽  
Shiwen Gao
Keyword(s):  
Neural Network ◽  
Data Processing ◽  
Seismic Data ◽  
Random Noise ◽  
Training Data ◽  
Prediction Errors ◽  
Network Learning ◽  
Deep Network ◽  
Residual Image ◽  
And Training

Random noise often contaminates seismic data and reduces its signal-to-noise ratio. Therefore, the removal of random noise has been an essential step in seismic data processing. The [Formula: see text]-[Formula: see text] predictive filtering method is one of the most widely used methods in suppressing random noise. However, when the subsurface structure becomes complex, this method suffers from higher prediction errors owing to the large number of different dip components that need to be predicted. Here, we used a denoising convolutional neural network (DnCNN) algorithm to attenuate random noise in seismic data. This method does not assume the linearity and stationarity of the signal in the conventional [Formula: see text]-[Formula: see text] domain prediction technique, and it involves creating a set of training data that are obtained by data processing, feeding the neural network with the training data obtained, and deep network learning and training. During deep network learning and training, the activation function and batch normalization are used to solve the gradient vanishing and gradient explosion problems, and the residual learning technique is used to improve the calculation precision, respectively. After finishing deep network learning and training, the network will have the ability to separate the residual image from the seismic data with noise. Then, clean images can be obtained by subtracting the residual image from the raw data with noise. Tests on the synthetic and real data demonstrate that the DnCNN algorithm is very effective for random noise attenuation in seismic data.

Improved Junction Body Flow Modeling Through Data-Driven Symbolic Regression

2019 ◽  
Vol 63 (4) ◽  
pp. 283-293 ◽  
Author(s):  
Jack Weatheritt ◽  
Richard David Sandberg
Keyword(s):  
Reynolds Number ◽  
Symbolic Regression ◽  
Basis Functions ◽  
Training Data ◽  
Data Driven ◽  
High Fidelity ◽  
Turbulent Stress ◽  
Stress Strain ◽  
New Model ◽  
Varying Coefficients

A novel data-driven turbulence modeling framework is presented and applied to the problem of junction body flow. In particular, a symbolic regression approach is used to find nonlinear analytical expressions of the turbulent stress‐strain coupling that are ready for implementation in computational fluid dynamics (CFD) solvers using Reynolds-averaged Navier‐Stokes (RANS) closures. Results from baseline linear RANS closure calculations of a finite square-mounted cylinder with a Reynolds number of <inline-graphic xlink:href="josr09180053inf1.tif"/>, based on diameter and freestream velocity, are shown to considerably overpredict the separated flow region downstream of the square cylinder, mainly because of the failure of the model to accurately represent the complex vortex structure generated by the junction flow. In the present study, a symbolic regression tool built on a gene expression programming technique is used to find a nonlinear constitutive stress‐strain relationship. In short, the algorithm finds the most appropriate linear combination of basis functions and spatially varying coefficients that approximate the turbulent stress tensor from high-fidelity data. Here, the high-fidelity data, or the so-called training data, were obtained from a hybrid RANS/Large Eddy Simulation (LES) calculation also developed with symbolic regression that showed excellent agreement with direct numerical simulation data. The present study, therefore, also demonstrates that training data required for RANS closure development can be obtained using computationally more affordable approaches, such as hybrid RANS/LES. A procedure is presented to evaluate which of the individual basis functions that are available for model development are most likely to produce a successful nonlinear closure. A new model is built using those basis functions only. This new model is then tested, i.e., an actual CFD calculation is performed, on the well-known periodic hills case and produces significantly better results than the linear baseline model, despite this test case being fundamentally different from the training case. Finally, the new model is shown to also improve predictive accuracy for a surface-mounted cube placed in a channel at a cube height Reynolds number of <inline-graphic xlink:href="josr09180053inf2.tif"/> over traditional linear RANS closures.

Sign in / Sign up

Export Citation Format

Share Document