Using Q-Learning and Genetic Algorithms to Improve the Efficiency of Weight Adjustments for Optimal Control and Design Problems

2005 ◽  
Author(s):  
Kaivan Kamali ◽  
Lijun Jiang ◽  
John Yen ◽  
K. W. Wang
Keyword(s):  
Optimal Control ◽  
Genetic Algorithms ◽  
Convergence Rate ◽  
Cost Function ◽  
Design Problem ◽  
Numerical Experiments ◽  
Design Parameters ◽  
Design Problems ◽  
Q Learning ◽  
Layered Approach

In traditional optimal control and design problems, the control gains and design parameters are usually derived to minimize a cost function reflecting the system performance and control effort. One major challenge of such approaches is the selection of weighting matrices in the cost function, which are usually determined via trial and error and human intuition. While various techniques have been proposed to automate the weight selection process, they either can not address complex design problems or suffer from slow convergence rate and high computational costs. We propose a layered approach based on Q-learning, a reinforcement learning technique, on top of genetic algorithms (GA) to determine the best weightings for optimal control and design problems. The layered approach allows for reuse of knowledge. Knowledge obtained via Q-learning in a design problem can be used to speed up the convergence rate of a similar design problem. Moreover, the layered approach allows for solving optimizations that cannot be solved by GA alone. To test the proposed method, we perform numerical experiments on a sample active-passive hybrid vibration control problem, namely adaptive structures with active-passive hybrid piezoelectric networks (APPN). These numerical experiments show that the proposed Q-learning scheme is a promising approach for.

Using Q-Learning and Genetic Algorithms to Improve the Efficiency of Weight Adjustments for Optimal Control and Design Problems

2007 ◽  
Vol 7 (4) ◽  
pp. 302-308 ◽  
Author(s):  
Kaivan Kamali ◽  
L. J. Jiang ◽  
John Yen ◽  
K. W. Wang
Keyword(s):  
Optimal Control ◽  
Genetic Algorithms ◽  
Convergence Rate ◽  
Cost Function ◽  
Design Problem ◽  
Numerical Experiments ◽  
Design Problems ◽  
Q Learning ◽  
Complex Design ◽  
Layered Approach

In traditional optimal control and design problems, the control gains and design parameters are usually derived to minimize a cost function reflecting the system performance and control effort. One major challenge of such approaches is the selection of weighting matrices in the cost function, which are usually determined via trial-and-error and human intuition. While various techniques have been proposed to automate the weight selection process, they either can not address complex design problems or suffer from slow convergence rate and high computational costs. We propose a layered approach based on Q-learning, a reinforcement learning technique, on top of genetic algorithms (GA) to determine the best weightings for optimal control and design problems. The layered approach allows for reuse of knowledge. Knowledge obtained via Q-learning in a design problem can be used to speed up the convergence rate of a similar design problem. Moreover, the layered approach allows for solving optimizations that cannot be solved by GA alone. To test the proposed method, we perform numerical experiments on a sample active-passive hybrid vibration control problem, namely adaptive structures with active-passive hybrid piezoelectric networks. These numerical experiments show that the proposed Q-learning scheme is a promising approach for automation of weight selection for complex design problems.

Optimal Designs by Means of Genetic Algorithms

2021 ◽  
pp. 344-354
Author(s):  
Lata Nautiyal ◽  
Preeti Shivach ◽  
Mangey Ram
Keyword(s):  
Genetic Algorithms ◽  
Modeling And Simulation ◽  
Engineering Design ◽  
Design Problem ◽  
High Performance ◽  
Mechanical Engineering ◽  
Design Parameters ◽  
Design Problems ◽  
Simulation Tools ◽  
Engineering Design Problems

With the advancement in contemporary computational and modeling skills, engineering design completely depends upon on variety of computer modeling and simulation tools to hasten the design cycles and decrease the overall budget. The most difficult design problem will include various design parameters along with the tables. Finding out the design space and ultimate solutions to those problems are still biggest challenges for the area of complex systems. This chapter is all about suggesting the use of Genetic Algorithms to enhance maximum engineering design problems. The chapter recommended that Genetic Algorithms are highly useful to increase the High-Performance Areas for Engineering Design. This chapter is established to use Genetic Algorithms to large number of design areas and delivered a comprehensive conversation on the use, scope and its applications in mechanical engineering.

Optimal Designs by Means of Genetic Algorithms

2018 ◽  
pp. 151-161
Author(s):  
Lata Nautiyal ◽  
Preeti Shivach ◽  
Mangey Ram
Keyword(s):  
Genetic Algorithms ◽  
Modeling And Simulation ◽  
Engineering Design ◽  
Design Problem ◽  
High Performance ◽  
Mechanical Engineering ◽  
Design Parameters ◽  
Design Problems ◽  
Simulation Tools ◽  
Engineering Design Problems

With the advancement in contemporary computational and modeling skills, engineering design completely depends upon on variety of computer modeling and simulation tools to hasten the design cycles and decrease the overall budget. The most difficult design problem will include various design parameters along with the tables. Finding out the design space and ultimate solutions to those problems are still biggest challenges for the area of complex systems. This chapter is all about suggesting the use of Genetic Algorithms to enhance maximum engineering design problems. The chapter recommended that Genetic Algorithms are highly useful to increase the High-Performance Areas for Engineering Design. This chapter is established to use Genetic Algorithms to large number of design areas and delivered a comprehensive conversation on the use, scope and its applications in mechanical engineering.

Constraint Reordering for Multi-Objective Configuration Design

1998 ◽  
Author(s):  
Nathan J. Adams ◽  
Georges M. Fadel
Keyword(s):  
Genetic Algorithms ◽  
Design Problem ◽  
Iterative Procedure ◽  
Complex Configuration ◽  
Configuration Design ◽  
Spatial Constraints ◽  
Design Problems ◽  
Acceptable Solution ◽  
The Many ◽  
Combinatorial Methods

Abstract Configuration design is the process of placing components, without altering their shape or connectivity, into an available space, while satisfying various spatial constraints, such as no component overlap. Minimizing the volume occupied by the components and or maximizing the accessibility of the components are just two examples of the many objectives that can drive a configuration design problem. For complex configuration designs, there can be many objectives, which can impose spatial constraints among the components and increase the design complexity, cycle cost, and time. An iterative procedure becomes necessary to reconcile these spatial constraints. To reach solutions that are optimal, these constraints must be reordered else combinatorial methods such as Genetic Algorithms that are used for such problems do not converge. Successful reordering can make complex configuration design problems easier to solve by minimizing the iterations necessary to reach an acceptable solution. Minimizing iterations translates into faster convergence and thus savings on time and money. This paper presents a methodology that can manage the propagation of spatial constraints in complex configuration design problems. Representative examples are shown and results and conclusions are drawn.

From Genetic Evolution to Engineering Optimization

2019 ◽  
pp. 219-246
Author(s):  
Yaser Khalifa
Keyword(s):  
Genetic Algorithms ◽  
Evolutionary Algorithms ◽  
Design Problem ◽  
Music Composition ◽  
Design Parameters ◽  
Individual Member ◽  
Genetic Evolution ◽  
Optimum Solution ◽  
Survival Of The Fittest ◽  
Electronic Circuit Design

This chapter focuses on one of the main members of the evolutionary algorithms class called genetic algorithms (GAs). GAs mimic the process of sexual reproduction and survival of the fittest in nature. The process begins by coding the design parameters into a chromosome-like structure to form an individual member. Next, a prespecified number of individual members are generated with different genetic contents to form a population. For each member of the population, a measure of its fitness or success is calculated. The members “most fit” in the population are then allowed to mate and reproduce, to have children. The children create a new and better generation of population, and the process repeats until an optimum solution or solutions are produced. This chapter illustrates how GAs are used to solve the following two problems: electronic circuit design problem and music composition.

Advantages of surrogate models for architectural design optimization

2015 ◽  
Vol 29 (4) ◽  
pp. 471-481 ◽  
Author(s):  
Thomas Wortmann ◽  
Alberto Costa ◽  
Giacomo Nannicini ◽  
Thomas Schroepfer
Keyword(s):  
Genetic Algorithms ◽  
Design Problem ◽  
Architectural Design ◽  
Optimization Problems ◽  
Resource Depletion ◽  
Surrogate Models ◽  
Performance Criteria ◽  
Design Parameters ◽  
Metaheuristic Methods ◽  
The Impact

AbstractClimate change, resource depletion, and worldwide urbanization feed the demand for more energy and resource-efficient buildings. Increasingly, architectural designers and consultants analyze building designs with easy-to-use simulation tools. To identify design alternatives with good performance, designers often turn to optimization methods. Randomized, metaheuristic methods such as genetic algorithms are popular in the architectural design field. However, are metaheuristics the best approach for architectural design problems that often are complex and ill defined? Metaheuristics may find solutions for well-defined problems, but they do not contribute to a better understanding of a complex design problem. This paper proposes surrogate-based optimization as a method that promotes understanding of the design problem. The surrogate method interpolates a mathematical model from data that relate design parameters to performance criteria. Designers can interact with this model to explore the approximate impact of changing design variables. We apply the radial basis function method, a specific type of surrogate model, to two architectural daylight optimization problems. These case studies, along with results from computational experiments, serve to discuss several advantages of surrogate models. First, surrogate models not only propose good solutions but also allow designers to address issues outside of the formulation of the optimization problem. Instead of accepting a solution presented by the optimization process, designers can improve their understanding of the design problem by interacting with the model. Second, a related advantage is that designers can quickly construct surrogate models from existing simulation results and other knowledge they might possess about the design problem. Designers can thus explore the impact of different evaluation criteria by constructing several models from the same set of data. They also can create models from approximate data and later refine them with more precise simulations. Third, surrogate-based methods typically find global optima orders of magnitude faster than genetic algorithms, especially when the evaluation of design variants requires time-intensive simulations.

scholarly journals Optimal control techniques based on infection age for the study of the COVID-19 epidemic

2020 ◽  
Vol 15 ◽  
pp. 48
Author(s):  
J. Frédéric Bonnans ◽  
Justina Gianatti
Keyword(s):  
Optimal Control ◽  
Cost Function ◽  
Numerical Experiments ◽  
Age Classes ◽  
Control Techniques ◽  
Infection Age ◽  
Death Toll ◽  
Infected Population ◽  
The Future ◽  
The Cost

We propose a model for the COVID-19 epidemic where the population is partitioned into classes corresponding to ages (that remain constant during the epidemic). The main feature is to take into account the infection age of the infected population. This allows to better simulate the infection propagation that crucially depend on the infection age. We discuss how to estimate the coefficients from data available in the future, and introduce a confinement variable as control. The cost function is a compromise between a confinement term, the hospitalization peak and the death toll. Our numerical experiments allow to evaluate the interest of confinement varying with age classes.

scholarly journals An Adaptive Optimization Method Based on Learning Rate Schedule for Neural Networks

2021 ◽  
Vol 11 (2) ◽  
pp. 850
Author(s):  
Dokkyun Yi ◽  
Sangmin Ji ◽  
Jieun Park
Keyword(s):  
Artificial Intelligence ◽  
Cost Function ◽  
Numerical Experiments ◽  
Global Minimum ◽  
Optimization Method ◽  
Learning Method ◽  
Adaptive Optimization ◽  
The Cost ◽  
Proof Of Convergence ◽  
Learning Data

Artificial intelligence (AI) is achieved by optimizing the cost function constructed from learning data. Changing the parameters in the cost function is an AI learning process (or AI learning for convenience). If AI learning is well performed, then the value of the cost function is the global minimum. In order to obtain the well-learned AI learning, the parameter should be no change in the value of the cost function at the global minimum. One useful optimization method is the momentum method; however, the momentum method has difficulty stopping the parameter when the value of the cost function satisfies the global minimum (non-stop problem). The proposed method is based on the momentum method. In order to solve the non-stop problem of the momentum method, we use the value of the cost function to our method. Therefore, as the learning method processes, the mechanism in our method reduces the amount of change in the parameter by the effect of the value of the cost function. We verified the method through proof of convergence and numerical experiments with existing methods to ensure that the learning works well.

scholarly journals Optimal Pressure Boundary Control of Steady Multiscale Fluid-Structure Interaction Shell Model Derived from Koiter Equations

Fluids ◽  
2021 ◽  
Vol 6 (4) ◽  
pp. 149
Author(s):  
Andrea Chierici ◽  
Leonardo Chirco ◽  
Sandro Manservisi
Keyword(s):  
Optimal Control ◽  
Solid Wall ◽  
Fluid Structure Interaction ◽  
Computational Cost ◽  
Robin Boundary Condition ◽  
Design Parameters ◽  
Fluid Structure ◽  
Control Approach ◽  
Structure Interaction ◽  
Fluid Boundary

Fluid-structure interaction (FSI) problems are of great interest, due to their applicability in science and engineering. However, the coupling between large fluid domains and small moving solid walls presents numerous numerical difficulties and, in some configurations, where the thickness of the solid wall can be neglected, one can consider membrane models, which are derived from the Koiter shell equations with a reduction of the computational cost of the algorithm. With this assumption, the FSI simulation is reduced to the fluid equations on a moving mesh together with a Robin boundary condition that is imposed on the moving solid surface. In this manuscript, we are interested in the study of inverse FSI problems that aim to achieve an objective by changing some design parameters, such as forces, boundary conditions, or geometrical domain shapes. We study the inverse FSI membrane model by using an optimal control approach that is based on Lagrange multipliers and adjoint variables. In particular, we propose a pressure boundary optimal control with the purpose to control the solid deformation by changing the pressure on a fluid boundary. We report the results of some numerical tests for two-dimensional domains to demonstrate the feasibility and robustness of our method.

Sign in / Sign up

Export Citation Format

Share Document