Application of Q Learning-Based Self-Tuning PID with DRNN in the Strip Flatness and Gauge System

In view of the process of automatic flatness control and automatic gauge control that is a nonlinear system with multi-dimensions, multi-variables, strong coupling and time variation, a novel control method called self-tuning PID with diagonal recurrent neural network (DRNN-PID) based on Q learning is proposed. It is able to coordinate the coupling of flatness control and gauge control agents to get the satisfactory control requirements without decoupling directly and amend output control laws by DRNN-PID adaptively. Decomposition-coordination is utilized to establish a novel multi-agent system for coordination control including flatness agent, gauge agent and Q learning agent. Simulation result demonstrates the validity of our proposed method.

Download Full-text

Boltzmann Distributed Replicator Dynamics: Population Games in a Microgrid Context

Games ◽

10.3390/g12010008 ◽

2021 ◽

Vol 12 (1) ◽

pp. 8

Author(s):

Gustavo Chica-Pedraza ◽

Eduardo Mojica-Nava ◽

Ernesto Cadena-Muñoz

Keyword(s):

Control Method ◽

Optimization Problems ◽

Replicator Dynamics ◽

Boltzmann Distribution ◽

Full Information ◽

Multi Agent Systems ◽

Q Learning ◽

Population Games ◽

Distributed Approach ◽

Multi Agent

Multi-Agent Systems (MAS) have been used to solve several optimization problems in control systems. MAS allow understanding the interactions between agents and the complexity of the system, thus generating functional models that are closer to reality. However, these approaches assume that information between agents is always available, which means the employment of a full-information model. Some tendencies have been growing in importance to tackle scenarios where information constraints are relevant issues. In this sense, game theory approaches appear as a useful technique that use a strategy concept to analyze the interactions of the agents and achieve the maximization of agent outcomes. In this paper, we propose a distributed control method of learning that allows analyzing the effect of the exploration concept in MAS. The dynamics obtained use Q-learning from reinforcement learning as a way to include the concept of exploration into the classic exploration-less Replicator Dynamics equation. Then, the Boltzmann distribution is used to introduce the Boltzmann-Based Distributed Replicator Dynamics as a tool for controlling agents behaviors. This distributed approach can be used in several engineering applications, where communications constraints between agents are considered. The behavior of the proposed method is analyzed using a smart grid application for validation purposes. Results show that despite the lack of full information of the system, by controlling some parameters of the method, it has similar behavior to the traditional centralized approaches.

Download Full-text

Improved Q-Learning Method for Linear Discrete-Time Systems

Processes ◽

10.3390/pr8030368 ◽

2020 ◽

Vol 8 (3) ◽

pp. 368

Author(s):

Jian Chen ◽

Jinhua Wang ◽

Jie Huang

Keyword(s):

Optimal Control ◽

Linear Systems ◽

Discrete Time ◽

Control Method ◽

Learning Method ◽

Control Laws ◽

Q Learning ◽

Model Free ◽

Quadratic Optimal Control ◽

Time Linear

In this paper, the Q-learning method for quadratic optimal control problem of discrete-time linear systems is reconsidered. The theoretical results prove that the quadratic optimal controller cannot be solved directly due to the linear correlation of the data sets. The following corollaries have been made: (1) The correlation of data is the key factor in the success for the calculation of quadratic optimal control laws by Q-learning method; (2) The control laws for linear systems cannot be derived directly by the existing Q-learning method; (3) For nonlinear systems, there are some doubts about the data independence of current method. Therefore, it is necessary to discuss the probability of the controllers established by the existing Q-learning method. To solve this problem, based on the ridge regression, an improved model-free Q-learning quadratic optimal control method for discrete-time linear systems is proposed in this paper. Therefore, the computation process can be implemented correctly, and the effective controller can be solved. The simulation results show that the proposed method can not only overcome the problem caused by the data correlation, but also derive proper control laws for discrete-time linear systems.

Download Full-text

Fuzzy Self-Tuning PID Control Method Based on Multi-Agent Particle Swarm Algorithm

International Journal of Simulation Systems Science & Technology ◽

10.5013/ijssst.a.17.42.31 ◽

2016 ◽

Author(s):

Guest Editor Jianping Du

Keyword(s):

Pid Control ◽

Control Method ◽

Particle Swarm ◽

Particle Swarm Algorithm ◽

Self Tuning ◽

Multi Agent ◽

Swarm Algorithm ◽

Agent Particle

Download Full-text

Q-learning algorithm based multi-agent coordinated control method for microgrids

2015 9th International Conference on Power Electronics and ECCE Asia (ICPE-ECCE Asia) ◽

10.1109/icpe.2015.7167977 ◽

2015 ◽

Cited By ~ 4

Author(s):

Yuanyuan Xi ◽

Liuchen Chang ◽

Meiqin Mao ◽

Peng Jin ◽

Nikos Hatziargyriou ◽

...

Keyword(s):

Control Method ◽

Learning Algorithm ◽

Coordinated Control ◽

Q Learning ◽

Multi Agent

Download Full-text

Reinforcement Learning Based Hierarchical Multi-Agent Robotic Search Team in Uncertain Environment

Mehran University Research Journal of Engineering and Technology ◽

10.22581/muet1982.2103.17 ◽

2021 ◽

Vol 40 (3) ◽

pp. 645-662

Author(s):

Shahzaib Hamid ◽

Ali Nasir ◽

Yasir Saleem

Keyword(s):

Reinforcement Learning ◽

Multi Agent Systems ◽

Qualitative Comparison ◽

Q Learning ◽

Novel Approach ◽

Learning Agent ◽

Markov Decision ◽

Multi Agent ◽

Efficient Learning ◽

Prior Models

Field of robotics has been under the limelight because of recent advances in Artificial Intelligence (AI). Due to increased diversity in multi-agent systems, new models are being developed to handle complexity of such systems. However, most of these models do not address problems such as; uncertainty handling, efficient learning, agent coordination and fault detection. This paper presents a novel approach of implementing Reinforcement Learning (RL) on hierarchical robotic search teams. The proposed algorithm handles uncertainties in the system by implementing Q-learning and depicts enhanced efficiency as well as better time consumption compared to prior models. The reason for that is each agent can take action on its own thus there is less dependency on leader agent for RL policy. The performance of this algorithm is measured by introducing agents in an unknown environment with both Markov Decision Process (MDP) and RL policies at their disposal. Simulation-based comparison of the agent motion is presented using the results from of MDP and RL policies. Furthermore, qualitative comparison of the proposed model with prior models is also presented.

Download Full-text

An Approach for Fault Tolerance in Multi-Agent Systems using Learning Agents

International Journal of Intelligent Information Technologies ◽

10.4018/ijiit.2015070103 ◽

2015 ◽

Vol 11 (3) ◽

pp. 30-44

Author(s):

Mounira Bouzahzah ◽

Ramdane Maamri

Keyword(s):

Fault Tolerance ◽

Formal Model ◽

Fault Tolerant ◽

Learning Algorithm ◽

Multi Agent Systems ◽

Q Learning ◽

Learning Agents ◽

Agent Systems ◽

Learning Agent ◽

Multi Agent

Through this paper, the authors propose a new approach to get fault tolerant multi-agent systems using learning agents. Generally, the exceptions in the multi-agent system are divided into two main groups: private exceptions that are treated directly by the agents and global exceptions that combine all unexpected exceptions that need handlers to be solved. The proposed approach solves the problem of these global exceptions using learning agents. This work uses a formal model called hierarchical plans to model the activities of the system's agents in order to facilitate the exception detection and to model the communication with the learning agent. This latter uses a modified version of the Q Learning Algorithm in order to choose which handler can be used to solve an exceptions. The paper tries to give a new direction in the field of fault tolerance in multi-agent systems by using learning agents, the proposed solution makes it possible to adapt the handler used in case of failure within the context changes and treat repeated exceptions using learning agent experiences.

Download Full-text

Deep Q-Learning and Preference Based Multi-Agent System for Sustainable Agricultural Market

Sensors ◽

10.3390/s21165276 ◽

2021 ◽

Vol 21 (16) ◽

pp. 5276

Author(s):

María E. Pérez-Pons ◽

Ricardo S. Alonso ◽

Oscar García ◽

Goreti Marreiros ◽

Juan Manuel Corchado

Keyword(s):

Market Price ◽

Environmental Indicators ◽

User Preferences ◽

Future Market ◽

Price Setting ◽

Multi Agent System ◽

Agent System ◽

Q Learning ◽

Learning Agent ◽

Multi Agent

Yearly population growth will lead to a significant increase in agricultural production in the coming years. Twenty-first century agricultural producers will be facing the challenge of achieving food security and efficiency. This must be achieved while ensuring sustainable agricultural systems and overcoming the problems posed by climate change, depletion of water resources, and the potential for increased erosion and loss of productivity due to extreme weather conditions. Those environmental consequences will directly affect the price setting process. In view of the price oscillations and the lack of transparent information for buyers, a multi-agent system (MAS) is presented in this article. It supports the making of decisions in the purchase of sustainable agricultural products. The proposed MAS consists of a system that supports decision-making when choosing a supplier on the basis of certain preference-based parameters aimed at measuring the sustainability of a supplier and a deep Q-learning agent for agricultural future market price forecast. Therefore, different agri-environmental indicators (AEIs) have been considered, as well as the use of edge computing technologies to reduce costs of data transfer to the cloud. The presented MAS combines price setting optimizations and user preferences in regards to accessing, filtering, and integrating information. The agents filter and fuse information relevant to a user according to supplier attributes and a dynamic environment. The results presented in this paper allow a user to choose the supplier that best suits their preferences as well as to gain insight on agricultural future markets price oscillations through a deep Q-learning agent.

Download Full-text