A Reinforcement Learning-Based Reward Mechanism for Molecule Generation that Introduces Activity Information

Abstract In this paper, we expolore Multi-Agent Reinforcement Learning (MARL) methods for unmanned aerial vehicle (UAV) cluster. Considering that the current UAV cluster is still in the program control stage, the fully autonomous and intelligent cooperative combat has not been realised. In order to realise the autonomous planning of the UAV cluster according to the changing environment and cooperate with each other to complete the combat goal, we propose a new MARL framework. It adopts the policy of centralised training with decentralised execution, and uses Actor-Critic network to select the execution action and then to make the corresponding evaluation. The new algorithm makes three key improvements on the basis of Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm. The first is to improve learning framework; it makes the calculated Q value more accurate. The second is to add collision avoidance setting, which can increase the operational safety factor. And the third is to adjust reward mechanism; it can effectively improve the cluster’s cooperative ability. Then the improved MADDPG algorithm is tested by performing two conventional combat missions. The simulation results show that the learning efficiency is obviously improved, and the operational safety factor is further increased compared with the previous algorithm.

Download Full-text

A Multi-sensing Input and Multi-constraint Reward Mechanism Based Deep Reinforcement Learning Method for Self-driving Policy Learning

10.1007/978-3-030-89092-6_63 ◽

2021 ◽

pp. 691-701

Author(s):

Zhongli Wang ◽

Hao Wang ◽

Xin Cui ◽

Chaochao Zheng

Keyword(s):

Reinforcement Learning ◽

Policy Learning ◽

Learning Method ◽

Reward Mechanism

Download Full-text

Horizontal Auto-Scaling for Multi-Access Edge Computing Using Safe Reinforcement Learning

ACM Transactions on Embedded Computing Systems ◽

10.1145/3475991 ◽

2021 ◽

Vol 20 (6) ◽

pp. 1-33

Author(s):

Kaustabha Ray ◽

Ansuman Banerjee

Keyword(s):

Reinforcement Learning ◽

Large Scale ◽

Edge Computing ◽

Cloud Services ◽

Test Bed ◽

New Paradigm ◽

Markov Decision ◽

Multi Access ◽

Reward Mechanism ◽

Auto Scaling

Multi-Access Edge Computing (MEC) has emerged as a promising new paradigm allowing low latency access to services deployed on edge servers to avert network latencies often encountered in accessing cloud services. A key component of the MEC environment is an auto-scaling policy which is used to decide the overall management and scaling of container instances corresponding to individual services deployed on MEC servers to cater to traffic fluctuations. In this work, we propose a Safe Reinforcement Learning (RL)-based auto-scaling policy agent that can efficiently adapt to traffic variations to ensure adherence to service specific latency requirements. We model the MEC environment using a Markov Decision Process (MDP). We demonstrate how latency requirements can be formally expressed in Linear Temporal Logic (LTL). The LTL specification acts as a guide to the policy agent to automatically learn auto-scaling decisions that maximize the probability of satisfying the LTL formula. We introduce a quantitative reward mechanism based on the LTL formula to tailor service specific latency requirements. We prove that our reward mechanism ensures convergence of standard Safe-RL approaches. We present experimental results in practical scenarios on a test-bed setup with real-world benchmark applications to show the effectiveness of our approach in comparison to other state-of-the-art methods in literature. Furthermore, we perform extensive simulated experiments to demonstrate the effectiveness of our approach in large scale scenarios.

Download Full-text

Video Summarization via Label Distributions Dual-Reward

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2021/331 ◽

2021 ◽

Author(s):

Yongbiao Gao ◽

Ning Xu ◽

Xin Geng

Keyword(s):

Reinforcement Learning ◽

Video Summarization ◽

Experimental Results ◽

Average Score ◽

State Representation ◽

Benchmark Datasets ◽

Video Summaries ◽

Paper Label ◽

Reward Mechanism ◽

Score Distributions

Reinforcement learning maps from perceived state representation to actions, which is adopted to solve the video summarization problem. The reward is crucial for deal with the video summarization task via reinforcement learning, since the reward signal defines the goal of video summarization. However, existing reward mechanism in reinforcement learning cannot handle the ambiguity which appears frequently in video summarization, i.e., the diverse consciousness by different people on the same video. To solve this problem, in this paper label distributions are mapped from the CNN and LSTM-based state representation to capture the subjectiveness of video summaries. The dual-reward is designed by measuring the similarity between user score distributions and the generated label distributions. Not only the average score but also the the variance of the subjective opinions are considered in summary generation. Experimental results on several benchmark datasets show that our proposed method outperforms other approaches under various settings.

Download Full-text