Air Combat Strategies Generation of CGF Based on MADDPG and Reward Shaping

In a one-on-one air combat game, the opponent’s maneuver strategy is usually not deterministic, which leads us to consider a variety of opponent’s strategies when designing our maneuver strategy. In this paper, an alternate freeze game framework based on deep reinforcement learning is proposed to generate the maneuver strategy in an air combat pursuit. The maneuver strategy agents for aircraft guidance of both sides are designed in a flight level with fixed velocity and the one-on-one air combat scenario. Middleware which connects the agents and air combat simulation software is developed to provide a reinforcement learning environment for agent training. A reward shaping approach is used, by which the training speed is increased, and the performance of the generated trajectory is improved. Agents are trained by alternate freeze games with a deep reinforcement algorithm to deal with nonstationarity. A league system is adopted to avoid the red queen effect in the game where both sides implement adaptive strategies. Simulation results show that the proposed approach can be applied to maneuver guidance in air combat, and typical angle fight tactics can be learnt by the deep reinforcement learning agents. For the training of an opponent with the adaptive strategy, the winning rate can reach more than 50%, and the losing rate can be reduced to less than 15%. In a competition with all opponents, the winning rate of the strategic agent selected by the league system is more than 44%, and the probability of not losing is about 75%.

Download Full-text

Maneuver Strategy Generation of UCAV for within Visual Range Air Combat Based on Multi-Agent Reinforcement Learning and Target Position Prediction

Applied Sciences ◽

10.3390/app10155198 ◽

2020 ◽

Vol 10 (15) ◽

pp. 5198

Author(s):

Weiren Kong ◽

Deyun Zhou ◽

Zhen Yang ◽

Kai Zhang ◽

Lina Zeng

Keyword(s):

Target Position ◽

Generation Algorithm ◽

Air Combat ◽

Reward Shaping ◽

Policy Gradient ◽

Multi Agent ◽

Position Prediction ◽

Unmanned Combat Air Vehicles ◽

The One ◽

Visual Range

With the development of unmanned combat air vehicles (UCAVs) and artificial intelligence (AI), within visual range (WVR) air combat confrontations utilizing intelligent UCAVs are expected to be widely used in future air combats. As controlling highly dynamic and uncertain WVR air combats from the ground stations of the UCAV is not feasible, it is necessary to develop an algorithm that can generate highly intelligent air combat strategies in order to enable UCAV to independently complete air combat missions. In this paper, a 1-vs.-1 WVR air combat strategy generation algorithm is proposed using the multi-agent deep deterministic policy gradient (MADDPG). A 1-vs.-1 WVR air combat is modeled as a two-player zero-sum Markov game (ZSMG). A method for predicting the position of the target is introduced into the model in order to enable the UCAV to predict the target’s actions and position. Moreover, to ensure that the UCAV is not limited by the constraints of the basic fighter maneuver (BFM) library, the action space is considered to be a continuous one. At the same time, a potential-based reward shaping method is proposed in order to improve the efficiency of the air combat strategy generation algorithm. Finally, the efficiency of the air combat strategy generation algorithm and the intelligence level of the resulting strategy is verified through simulation experiments. The results show that an air combat strategy using target position prediction is superior to the one that does not use target position prediction.

Download Full-text

Air combat training: Good Stick Index validation.

PsycEXTRA Dataset ◽

10.1037/e423562004-001 ◽

1979 ◽

Cited By ~ 1

Author(s):

Samuel B Moore

Keyword(s):

Combat Training ◽

Air Combat

Download Full-text

Visual cue requirements for target orientation assessment in air combat simulation.

PsycEXTRA Dataset ◽

10.1037/e437802004-001 ◽

1993 ◽

Cited By ~ 1

Author(s):

Harold D. Warner ◽

Gary L. Serfoss ◽

David C. Hubbard

Keyword(s):

Target Orientation ◽

Visual Cue ◽

Combat Simulation ◽

Air Combat

Download Full-text

Development and validation of a performance measurement system for air-to-air combat

PsycEXTRA Dataset ◽

10.1037/e573942012-002 ◽

1982 ◽

Author(s):

Anthony P. Ciavarelli ◽

Steven T. Breidenbach

Keyword(s):

Performance Measurement ◽

Measurement System ◽

Performance Measurement System ◽

Air Combat ◽

Development And Validation ◽

A Performance

Download Full-text

Janus – A New Approach to Air Combat Pilot Training

Transactions on Aerospace Research ◽

10.2478/tar-2019-0019 ◽

2019 ◽

Vol 2019 (4) ◽

pp. 7-22

Author(s):

Georges Bridel ◽

Zdobyslaw Goraj ◽

Lukasz Kiszkowiak ◽

Jean-Georges Brévot ◽

Jean-Pierre Devaux ◽

...

Keyword(s):

Low Cost ◽

Cost Effective ◽

High Energy ◽

Training System ◽

Embedded Sensors ◽

Pilot Training ◽

New Approach ◽

Combat Aircraft ◽

Air Combat ◽

The Cost

Abstract Advanced jet training still relies on old concepts and solutions that are no longer efficient when considering the current and forthcoming changes in air combat. The cost of those old solutions to develop and maintain combat pilot skills are important, adding even more constraints to the training limitations. The requirement of having a trainer aircraft able to perform also light combat aircraft operational mission is adding unnecessary complexity and cost without any real operational advantages to air combat mission training. Thanks to emerging technologies, the JANUS project will study the feasibility of a brand-new concept of agile manoeuvrable training aircraft and an integrated training system, able to provide a live, virtual and constructive environment. The JANUS concept is based on a lightweight, low-cost, high energy aircraft associated to a ground based Integrated Training System providing simulated and emulated signals, simulated and real opponents, combined with real-time feedback on pilot’s physiological characteristics: traditionally embedded sensors are replaced with emulated signals, simulated opponents are proposed to the pilot, enabling out of sight engagement. JANUS is also providing new cost effective and more realistic solutions for “Red air aircraft” missions, organised in so-called “Aggressor Squadrons”.

Download Full-text