UAV maneuvering decision -making algorithm based on Twin Delayed Deep Deterministic Policy Gradient Algorithm

Aiming at intelligent decision-making of UAV based on situation information in air combat, a novel maneuvering decision method based on deep reinforcement learning is proposed in this paper. The autonomous maneuvering model of UAV is established by Markov Decision Process. The Twin Delayed Deep Deterministic Policy Gradient(TD3) algorithm and the Deep Deterministic Policy Gradient (DDPG) algorithm in deep reinforcement learning are used to train the model, and the experimental results of the two algorithms are analyzed and compared. The simulation experiment results show that compared with the DDPG algorithm, the TD3 algorithm has stronger decision-making performance and faster convergence speed, and is more suitable forsolving combat problems. The algorithm proposed in this paper enables UAVs to autonomously make maneuvering decisions based on situation information such as position, speed, and relative azimuth, adjust their actions to approach and successfully strike the enemy, providing a new method for UAVs to make intelligent maneuvering decisions during air combat.

The Specification of Air-to-Air Combat Tactics Using UML Sequence Diagram

Air force air-to-air combat tactics are occurring at a high speed in three-dimensional space. The specification of the tactics requires dealing with a quite amount of information, which makes it a challenge to accurately describe the maneuvering procedure of the tactics. The specification of air-to-air tactics using natural languages is not suitable because of the intrinsic ambiguity of natural languages. Therefore, this paper proposes an approach of using UML Sequence Diagram to describe air-to-air combat tactics. Since the current Sequence Diagram notation is not sufficient to express all aspects of the tactics, we extend the syntax of the Sequence Diagram to accommodate the required features of air-to-air combat tactics. We evaluate the applicability of the extended Sequence Diagram to air-to-air combat tactics using a case example, that is the manned-unmanned teaming combat tactic. The result shows that Sequence Diagram specification is more advantageous than natural language specification in terms of readability, conciseness, and accuracy. However, the expressiveness of the Sequence Diagram is evaluated to be less powerful than natural language, requiring further study to address this issue.

Cooperative Attack-Defense Decision-Making of Multi-UAV Using Satisficing Decision-Enhanced Wolf Pack Search Algorithm

Unmanned Aerial Vehicles (UAVs) have shown their superiority for applications in complicated military missions. A cooperative attack-defense decision-making method based on satisficing decision-enhanced wolf pack search (SDEWPS) algorithm is developed for multi-UAV air combat in this paper. Firstly, the multi-UAV air combat mathematical model is provided and the attack-defense decision-making constraints are defined. Besides the traditional air combat situation, the capability of UAVs and target information including target type and target intention are all considered in this paper to establish the air combat superiority function. Then, the wolf pack search (WPS) algorithm is used to solve the attack decision problem. In order to improve efficiency, the satisficing decision theory is employed to enhance the WPS to obtain the satisficing solution rather than optimal solution. The simulation results show that the developed method can realize the cooperative attack decision-making.

Autonomous maneuver strategy of swarm air combat based on DDPG

Unmanned aerial vehicles (UAVs) have been found significantly important in the air combats, where intelligent and swarms of UAVs will be able to tackle with the tasks of high complexity and dynamics. The key to empower the UAVs with such capability is the autonomous maneuver decision making. In this paper, an autonomous maneuver strategy of UAV swarms in beyond visual range air combat based on reinforcement learning is proposed. First, based on the process of air combat and the constraints of the swarm, the motion model of UAV and the multi-to-one air combat model are established. Second, a two-stage maneuver strategy based on air combat principles is designed which include inter-vehicle collaboration and target-vehicle confrontation. Then, a swarm air combat algorithm based on deep deterministic policy gradient strategy (DDPG) is proposed for online strategy training. Finally, the effectiveness of the proposed algorithm is validated by multi-scene simulations. The results show that the algorithm is suitable for UAV swarms of different scales.