scholarly journals Spatial Grammar-Based Recurrent Neural Network for Design Form and Behavior Optimization

2019 ◽  
Vol 141 (12) ◽  
Author(s):  
Gary M. Stump ◽  
Simon W. Miller ◽  
Michael A. Yukish ◽  
Timothy W. Simpson ◽  
Conrad Tucker
Keyword(s):  
Reinforcement Learning ◽  
Degrees Of Freedom ◽  
Parametric Design ◽  
Control Policy ◽  
Training Data ◽  
High Performing ◽  
First Case ◽  
Spatial Grammar ◽  
And Behavior

Abstract A novel method has been developed to optimize both the form and behavior of complex systems. The method uses spatial grammars embodied in character-recurrent neural networks (char-RNNs) to define the system including actuator numbers and degrees of freedom, reinforcement learning to optimize actuator behavior, and physics-based simulation systems to determine performance and provide (re)training data for the char-RNN. Compared to parametric design optimization with fixed numbers of inputs, using grammars and char-RNNs allows for a more complex, combinatorial infinite design space. In the proposed method, the char-RNN is first trained to learn a spatial grammar that defines the assembly layout, component geometries, material properties, and arbitrary numbers and degrees of freedom of actuators. Next, generated designs are evaluated using a physics-based environment, with an inner optimization loop using reinforcement learning to determine the best control policy for the actuators. The resulting design is thus optimized for both form and behavior, generated by a char-RNN embodying a high-performing grammar. Two evaluative case studies are presented using the design of the modular sailing craft. The first case study optimizes the design without actuated surfaces, allowing the char-RNN to understand the semantics of high-performing designs. The second case study extends the first by incorporating controllable actuators requiring an inner loop behavioral optimization. The implications of the results are discussed along with the ongoing and future work.

Multi-Context Generation in Virtual Reality Environments Using Deep Reinforcement Learning

2020 ◽  
Author(s):  
James Cunningham ◽  
Christian Lopez ◽  
Omar Ashour ◽  
Conrad S. Tucker
Keyword(s):  
Virtual Reality ◽  
Reinforcement Learning ◽  
Virtual Environments ◽  
Probability Distributions ◽  
Automatic Generation ◽  
Grocery Store ◽  
Training Data ◽  
Learning Approaches ◽  
Common Concept

Abstract In this work, a Deep Reinforcement Learning (RL) approach is proposed for Procedural Content Generation (PCG) that seeks to automate the generation of multiple related virtual reality (VR) environments for enhanced personalized learning. This allows for the user to be exposed to multiple virtual scenarios that demonstrate a consistent theme, which is especially valuable in an educational context. RL approaches to PCG offer the advantage of not requiring training data, as opposed to other PCG approaches that employ supervised learning approaches. This work advances the state of the art in RL-based PCG by demonstrating the ability to generate a diversity of contexts in order to teach the same underlying concept. A case study is presented that demonstrates the feasibility of the proposed RL-based PCG method using examples of probability distributions in both manufacturing facility and grocery store virtual environments. The method demonstrated in this paper has the potential to enable the automatic generation of a variety of virtual environments that are connected by a common concept or theme.

scholarly journals 3D Design Using Generative Adversarial Networks and Physics-Based Validation

2019 ◽  
Vol 142 (7) ◽  
Author(s):  
Dule Shu ◽  
James Cunningham ◽  
Gary Stump ◽  
Simon W. Miller ◽  
Michael A. Yukish ◽  
...  
Keyword(s):  
Evaluation Process ◽  
3D Models ◽  
Training Data ◽  
Generative Adversarial Networks ◽  
3D Design ◽  
Generative Adversarial Network ◽  
Data Set ◽  
High Performing ◽  
Adversarial Network

Abstract The authors present a generative adversarial network (GAN) model that demonstrates how to generate 3D models in their native format so that they can be either evaluated using complex simulation environments or realized using methods such as additive manufacturing. Once initially trained, the GAN can create additional training data itself by generating new designs, evaluating them in a physics-based virtual environment, and adding the high performing ones to the training set. A case study involving a GAN model that is initially trained on 4045 3D aircraft models is used for demonstration, where a training data set that has been updated with GAN-generated and evaluated designs results in enhanced model generation, in both the geometric feasibility and performance of the designs. Z-tests on the performance scores of the generated aircraft models indicate a statistically significant improvement in the functionality of the generated models after three iterations of the training-evaluation process. In the case study, a number of techniques are explored to structure the generate-evaluate process in order to balance the need to generate feasible designs with the need for innovative designs.

Learning from Case Studies: Timber Tower of Gospić Cathedral and Traditional Timber Church

2013 ◽  
Vol 778 ◽  
pp. 927-934
Author(s):  
Mislav Stepinac ◽  
Vlatka Rajčić ◽  
Dean Čizmar
Keyword(s):  
Cultural Heritage ◽  
Case Studies ◽  
Structural Behavior ◽  
Single Element ◽  
Historic Buildings ◽  
Timber Structures ◽  
Structural Elements ◽  
First Case ◽  
And Behavior

The conservation of historical timber structures requires both the knowledge of structural behavior of complete building and behavior of any single element in structure. Because of a poor communication between the institutions involved in the conservation and restoration of buildings and institutions that urgently require renovation of structures, renovation of historic buildings is often performed inappropriately due to engineering and architectural approach. Buildings are frequently subjected to inappropriate interventions, replacements of structural elements, or even worse, they are demolished. First case study in this paper is Cathedral from Gospic city which was subjected to inappropriate reconstruction in 1990s. Second case study is small traditionally constructed timber chapel in village Lijevi Stefanki which represents a valuable monument of rural cultural heritage.

scholarly journals Developmental Reinforcement Learning of Control Policy of a Quadcopter UAV With Thrust Vectoring Rotors

2020 ◽  
Author(s):  
Aditya M. Deshpande ◽  
Rumit Kumar ◽  
Ali A. Minai ◽  
Manish Kumar
Keyword(s):  
Reinforcement Learning ◽  
Degrees Of Freedom ◽  
Initial Conditions ◽  
Control Policy ◽  
Policy Transfer ◽  
Action Space ◽  
Thrust Vectoring ◽  
Tilt Rotor ◽  
Force Magnitude ◽  
Higher Dimensional

Abstract In this paper, we present a novel developmental reinforcement learning-based controller for a quadcopter with thrust vectoring capabilities. This multirotor UAV design has tilt-enabled rotors. It utilizes the rotor force magnitude and direction to achieve the desired state during flight. The control policy of this robot is learned using the policy transfer from the learned controller of the quadcopter (comparatively simple UAV design without thrust vectoring). This approach allows learning a control policy for systems with multiple inputs and multiple outputs. The performance of the learned policy is evaluated by physics-based simulations for the tasks of hovering and way-point navigation. The flight simulations utilize a flight controller based on reinforcement learning without any additional PID components. The results show faster learning with the presented approach as opposed to learning the control policy from scratch for this new UAV design created by modifications in a conventional quadcopter, i.e., the addition of more degrees of freedom (4-actuators in conventional quadcopter to 8-actuators in tilt-rotor quadcopter). We demonstrate the robustness of our learned policy by showing the recovery of the tilt-rotor platform in the simulation from various non-static initial conditions in order to reach a desired state. The developmental policy for the tilt-rotor UAV also showed superior fault tolerance when compared with the policy learned from the scratch. The results show the ability of the presented approach to bootstrap the learned behavior from a simpler system (lower-dimensional action-space) to a more complex robot (comparatively higher-dimensional action-space) and reach better performance faster.

scholarly journals A Survey of Uncertainty Quantification in Machine Learning for Space Weather Prediction

Geosciences ◽  
2022 ◽  
Vol 12 (1) ◽  
pp. 27
Author(s):  
Talha Siddique ◽  
Md Mahmud ◽  
Amy Keesee ◽  
Chigomezyo Ngwira ◽  
Hyunju Connor
Keyword(s):  
Machine Learning ◽  
Space Weather ◽  
Weather Prediction ◽  
Training Data ◽  
Series Data ◽  
Modern World ◽  
Great Promise ◽  
Model Parameters ◽  
First Case

With the availability of data and computational technologies in the modern world, machine learning (ML) has emerged as a preferred methodology for data analysis and prediction. While ML holds great promise, the results from such models are not fully unreliable due to the challenges introduced by uncertainty. An ML model generates an optimal solution based on its training data. However, if the uncertainty in the data and the model parameters are not considered, such optimal solutions have a high risk of failure in actual world deployment. This paper surveys the different approaches used in ML to quantify uncertainty. The paper also exhibits the implications of quantifying uncertainty when using ML by performing two case studies with space physics in focus. The first case study consists of the classification of auroral images in predefined labels. In the second case study, the horizontal component of the perturbed magnetic field measured at the Earth’s surface was predicted for the study of Geomagnetically Induced Currents (GICs) by training the model using time series data. In both cases, a Bayesian Neural Network (BNN) was trained to generate predictions, along with epistemic and aleatoric uncertainties. Finally, the pros and cons of both Gaussian Process Regression (GPR) models and Bayesian Deep Learning (DL) are weighed. The paper also provides recommendations for the models that need exploration, focusing on space weather prediction.

scholarly journals Reinforcement learning control of a biomechanical model of the upper extremity

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Florian Fischer ◽  
Miroslav Bachinski ◽  
Markus Klar ◽  
Arthur Fleig ◽  
Jörg Müller
Keyword(s):  
Reinforcement Learning ◽  
Upper Extremity ◽  
Degrees Of Freedom ◽  
State Of The Art ◽  
Movement Time ◽  
Biomechanical Model ◽  
Control Policy ◽  
Human Movement ◽  
Motor Noise ◽  
Skeletal Model

AbstractAmong the infinite number of possible movements that can be produced, humans are commonly assumed to choose those that optimize criteria such as minimizing movement time, subject to certain movement constraints like signal-dependent and constant motor noise. While so far these assumptions have only been evaluated for simplified point-mass or planar models, we address the question of whether they can predict reaching movements in a full skeletal model of the human upper extremity. We learn a control policy using a motor babbling approach as implemented in reinforcement learning, using aimed movements of the tip of the right index finger towards randomly placed 3D targets of varying size. We use a state-of-the-art biomechanical model, which includes seven actuated degrees of freedom. To deal with the curse of dimensionality, we use a simplified second-order muscle model, acting at each degree of freedom instead of individual muscles. The results confirm that the assumptions of signal-dependent and constant motor noise, together with the objective of movement time minimization, are sufficient for a state-of-the-art skeletal model of the human upper extremity to reproduce complex phenomena of human movement, in particular Fitts’ Law and the $$\frac{2}{3}$$ 2 3 Power Law. This result supports the notion that control of the complex human biomechanical system can plausibly be determined by a set of simple assumptions and can easily be learned.

scholarly journals Comparing Reinforcement Learning Methods for Real-Time Optimization of a Chemical Process

Processes ◽  
2020 ◽  
Vol 8 (11) ◽  
pp. 1497
Author(s):  
Titus Quah ◽  
Derek Machalek ◽  
Kody M. Powell
Keyword(s):  
Reinforcement Learning ◽  
Chemical Process ◽  
Optimization Method ◽  
Training Data ◽  
Economic Optimization ◽  
Artificial Neural Network Ann ◽  
Policy Optimization ◽  
Real Time Optimization ◽  
Operational Data

One popular method for optimizing systems, referred to as ANN-PSO, uses an artificial neural network (ANN) to approximate the system and an optimization method like particle swarm optimization (PSO) to select inputs. However, with reinforcement learning developments, it is important to compare ANN-PSO to newer algorithms, like Proximal Policy Optimization (PPO). To investigate ANN-PSO’s and PPO’s performance and applicability, we compare their methodologies, apply them on steady-state economic optimization of a chemical process, and compare their results to a conventional first principles modeling with nonlinear programming (FP-NLP). Our results show that ANN-PSO and PPO achieve profits nearly as high as FP-NLP, but PPO achieves slightly higher profits compared to ANN-PSO. We also find PPO has the fastest computational times, 10 and 10,000 times faster than FP-NLP and ANN-PSO, respectively. However, PPO requires more training data than ANN-PSO to converge to an optimal policy. This case study suggests PPO has better performance as it achieves higher profits and faster online computational times. ANN-PSO shows better applicability with its capability to train on historical operational data and higher training efficiency.

The Influence of Attitudes, Subjective, Norms, and Behavior Controls on Intention to Choose Islamic Banks With Islamic Religiosity as a Moderating Variable (Case Study at Bri Syariah Purwokerto Customers)

2018 ◽  
Vol 2 (1) ◽  
Author(s):  
Akhmad Khoyrun Najakh ◽  
Dwiwiyati Astogini ◽  
Sri Martini
Keyword(s):  
Behavioral Control ◽  
Sampling Technique ◽  
Subjective Norm ◽  
Subjective Norms ◽  
Islamic Banks ◽  
Control Behavior ◽  
Before And After ◽  
And Behavior ◽  
Number Of Customers

The purpose of this study was to analyze the influence of attitudes on the intention to choose Islamic banks, to analyze the effect of subjective norm on the intention to choose Islamic banks. to analyze the effect of the control behavior of the intention to choose the Islamic banks, to analyze the moderating influence of religiosity on the relationship attitudes, subjective norms and behavioral control of the intention to choose the Islamic banks . The method used is a survey with a sampling technique used purposive sampling with a sample size of this study was 100 respondents . Further analysis tools used in this study is multiple regression analysis using SPSS 16.0 software . Based on this study it can be concluded that the attitude does not affect to the intention of choose Bank BRISyariah. Subjective norm positive effect on intention choose Bank BRISyariah. Control behavior does not affect to the intention choose Bank BRISyariah. Relationship between Attitudes, Subjective Norms and Behavior Control with the intention to select Bank BRISyariah not moderated by religiosity.Based on these conclusions can be said that the Bank BRISyariah should improve understanding related to the subjective norm in order to increase the number of customers who use the services of Islamic Banking . Further research is recommended in order to follow up and develop this research to further explore the independent and dependent variables continued before and after behavioral intention or intention to perform a specific action .

Deep Reinforcement Learning for Multiparameter Optimization in de novo Drug Design

2019 ◽  
Author(s):  
Niclas Ståhl ◽  
Göran Falkman ◽  
Alexander Karlsson ◽  
Gunnar Mathiason ◽  
Jonas Boström
Keyword(s):  
Reinforcement Learning ◽  
Short Term Memory ◽  
De Novo ◽  
De Novo Drug Design ◽  
Generative Process ◽  
New Methods ◽  
Multiparameter Optimization ◽  
Long Short Term Memory ◽  
New Compounds

<p>In medicinal chemistry programs it is key to design and make compounds that are efficacious and safe. This is a long, complex and difficult multi-parameter optimization process, often including several properties with orthogonal trends. New methods for the automated design of compounds against profiles of multiple properties are thus of great value. Here we present a fragment-based reinforcement learning approach based on an actor-critic model, for the generation of novel molecules with optimal properties. The actor and the critic are both modelled with bidirectional long short-term memory (LSTM) networks. The AI method learns how to generate new compounds with desired properties by starting from an initial set of lead molecules and then improve these by replacing some of their fragments. A balanced binary tree based on the similarity of fragments is used in the generative process to bias the output towards structurally similar molecules. The method is demonstrated by a case study showing that 93% of the generated molecules are chemically valid, and a third satisfy the targeted objectives, while there were none in the initial set.</p>

Sign in / Sign up

Export Citation Format

Share Document