scholarly journals Spellcaster Control Agent in StarCraft II Using Deep Reinforcement Learning

Electronics ◽  
2020 ◽  
Vol 9 (6) ◽  
pp. 996
Author(s):  
Wooseok Song ◽  
Woong Hyun Suh ◽  
Chang Wook Ahn
Keyword(s):  
Reinforcement Learning ◽  
Force Field ◽  
Real Time ◽  
Transfer Learning ◽  
Main Idea ◽  
Control Agent ◽  
Training Methods ◽  
Training Method ◽  
Combat Training

This paper proposes a DRL -based training method for spellcaster units in StarCraft II, one of the most representative Real-Time Strategy (RTS) games. During combat situations in StarCraft II, micro-controlling various combat units is crucial in order to win the game. Among many other combat units, the spellcaster unit is one of the most significant components that greatly influences the combat results. Despite the importance of the spellcaster units in combat, training methods to carefully control spellcasters have not been thoroughly considered in related studies due to the complexity. Therefore, we suggest a training method for spellcaster units in StarCraft II by using the A3C algorithm. The main idea is to train two Protoss spellcaster units under three newly designed minigames, each representing a unique spell usage scenario, to use ‘Force Field’ and ‘Psionic Storm’ effectively. As a result, the trained agents show winning rates of more than 85% in each scenario. We present a new training method for spellcaster units that releases the limitation of StarCraft II AI research. We expect that our training method can be used for training other advanced and tactical units by applying transfer learning in more complex minigame scenarios or full game maps.

scholarly journals Deep Transfer Learning Based Intrusion Detection System for Electric Vehicular Networks

Sensors ◽  
2021 ◽  
Vol 21 (14) ◽  
pp. 4736
Author(s):  
Sk. Tanzir Mehedi ◽  
Adnan Anwar ◽  
Ziaur Rahman ◽  
Kawsar Ahmed
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Intrusion Detection ◽  
Real Time ◽  
Transfer Learning ◽  
Security Requirements ◽  
Detection Accuracy ◽  
Area Network ◽  
Complex Data ◽  
Network Intrusion

The Controller Area Network (CAN) bus works as an important protocol in the real-time In-Vehicle Network (IVN) systems for its simple, suitable, and robust architecture. The risk of IVN devices has still been insecure and vulnerable due to the complex data-intensive architectures which greatly increase the accessibility to unauthorized networks and the possibility of various types of cyberattacks. Therefore, the detection of cyberattacks in IVN devices has become a growing interest. With the rapid development of IVNs and evolving threat types, the traditional machine learning-based IDS has to update to cope with the security requirements of the current environment. Nowadays, the progression of deep learning, deep transfer learning, and its impactful outcome in several areas has guided as an effective solution for network intrusion detection. This manuscript proposes a deep transfer learning-based IDS model for IVN along with improved performance in comparison to several other existing models. The unique contributions include effective attribute selection which is best suited to identify malicious CAN messages and accurately detect the normal and abnormal activities, designing a deep transfer learning-based LeNet model, and evaluating considering real-world data. To this end, an extensive experimental performance evaluation has been conducted. The architecture along with empirical analyses shows that the proposed IDS greatly improves the detection accuracy over the mainstream machine learning, deep learning, and benchmark deep transfer learning models and has demonstrated better performance for real-time IVN security.

scholarly journals Robust Quadrotor Control through Reinforcement Learning with Disturbance Compensation

2021 ◽  
Vol 11 (7) ◽  
pp. 3257
Author(s):  
Chen-Huan Pi ◽  
Wei-Yuan Ye ◽  
Stone Cheng
Keyword(s):  
Neural Network ◽  
Reinforcement Learning ◽  
Control Strategy ◽  
External Disturbance ◽  
Control Agent ◽  
Network Control ◽  
Outdoor Environment ◽  
Disturbance Compensation ◽  
Tracking Accuracy ◽  
Control Scheme

In this paper, a novel control strategy is presented for reinforcement learning with disturbance compensation to solve the problem of quadrotor positioning under external disturbance. The proposed control scheme applies a trained neural-network-based reinforcement learning agent to control the quadrotor, and its output is directly mapped to four actuators in an end-to-end manner. The proposed control scheme constructs a disturbance observer to estimate the external forces exerted on the three axes of the quadrotor, such as wind gusts in an outdoor environment. By introducing an interference compensator into the neural network control agent, the tracking accuracy and robustness were significantly increased in indoor and outdoor experiments. The experimental results indicate that the proposed control strategy is highly robust to external disturbances. In the experiments, compensation improved control accuracy and reduced positioning error by 75%. To the best of our knowledge, this study is the first to achieve quadrotor positioning control through low-level reinforcement learning by using a global positioning system in an outdoor environment.

scholarly journals Innovating Occupational Safety Training: A Scoping Review on Digital Games and Possible Applications in Agriculture

2021 ◽  
Vol 18 (4) ◽  
pp. 1868
Author(s):  
Lucia Vigoroso ◽  
Federica Caffaro ◽  
Margherita Micheletti Cremasco ◽  
Eugenio Cavallo
Keyword(s):  
Occupational Safety ◽  
Web Of Science ◽  
Digital Games ◽  
Safety Training ◽  
Training Methods ◽  
Training Method ◽  
Occupational Risks ◽  
Game Mechanics ◽  
Limited Application ◽  
Safety Knowledge

Digital games have been successfully applied in different working sectors as an occupational safety training method, but with a very limited application in agriculture. In agriculture and other productive sectors, unintentional injuries tend to occur with similar dynamics. A literature review was carried out to understand how occupational risks are addressed during game-based safety training in different productive sectors and how this can be transferred to agriculture. Literature about “serious game” and “gamification” as safety training methods was searched in WEB OF SCIENCE, SCOPUS, PUBMED and PsycINFO databases. In the forty-two publications retained, the computer was identified as the most adopted game support, whereas “points”, “levels”, “challenges” and “discovery” were the preferred game mechanics. Moreover, an association can be detected between the game mechanics and the elements developed in the game. Finally, during the game assessment, much positive feedback was collected and the games proved to be able to increase the operators’ skills and safety knowledge. In light of the results, insights are provided to develop an effective, satisfying and engaging safety game training for workers employed in agriculture. Games can be best used to learn and they are certain to improve over the next few years.

scholarly journals Real-Time Safety Optimization of Connected Vehicle Trajectories Using Reinforcement Learning

Sensors ◽  
2021 ◽  
Vol 21 (11) ◽  
pp. 3864
Author(s):  
Tarek Ghoul ◽  
Tarek Sayed
Keyword(s):  
Reinforcement Learning ◽  
Real Time ◽  
Low Cost ◽  
Safety Evaluation ◽  
Traffic Volume ◽  
Connected Vehicles ◽  
Connected Vehicle ◽  
Real World Data ◽  
Physical Constraints ◽  
Traffic Conflicts

Speed advisories are used on highways to inform vehicles of upcoming changes in traffic conditions and apply a variable speed limit to reduce traffic conflicts and delays. This study applies a similar concept to intersections with respect to connected vehicles to provide dynamic speed advisories in real-time that guide vehicles towards an optimum speed. Real-time safety evaluation models for signalized intersections that depend on dynamic traffic parameters such as traffic volume and shock wave characteristics were used for this purpose. The proposed algorithm incorporates a rule-based approach alongside a Deep Deterministic Policy Gradient reinforcement learning technique (DDPG) to assign ideal speeds for connected vehicles at intersections and improve safety. The system was tested on two intersections using real-world data and yielded an average reduction in traffic conflicts ranging from 9% to 23%. Further analysis was performed to show that the algorithm yields tangible results even at lower market penetration rates (MPR). The algorithm was tested on the same intersection with different traffic volume conditions as well as on another intersection with different physical constraints and characteristics. The proposed algorithm provides a low-cost approach that is not computationally intensive and works towards optimizing for safety by reducing rear-end traffic conflicts.

scholarly journals Intuitive real-time control strategy for high-density myoelectric hand prosthesis using deep and transfer learning

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Simon Tam ◽  
Mounir Boukadoum ◽  
Alexandre Campeau-Lecours ◽  
Benoit Gosselin
Keyword(s):  
Real Time ◽  
Transfer Learning ◽  
Control Strategy ◽  
High Density ◽  
Fine Tuning ◽  
Mode Switching ◽  
Correct Prediction ◽  
Learning Approach ◽  
Predictive Values ◽  
Hand Prosthesis

AbstractMyoelectric hand prostheses offer a way for upper-limb amputees to recover gesture and prehensile abilities to ease rehabilitation and daily life activities. However, studies with prosthesis users found that a lack of intuitiveness and ease-of-use in the human-machine control interface are among the main driving factors in the low user acceptance of these devices. This paper proposes a highly intuitive, responsive and reliable real-time myoelectric hand prosthesis control strategy with an emphasis on the demonstration and report of real-time evaluation metrics. The presented solution leverages surface high-density electromyography (HD-EMG) and a convolutional neural network (CNN) to adapt itself to each unique user and his/her specific voluntary muscle contraction patterns. Furthermore, a transfer learning approach is presented to drastically reduce the training time and allow for easy installation and calibration processes. The CNN-based gesture recognition system was evaluated in real-time with a group of 12 able-bodied users. A real-time test for 6 classes/grip modes resulted in mean and median positive predictive values (PPV) of 93.43% and 100%, respectively. Each gesture state is instantly accessible from any other state, with no mode switching required for increased responsiveness and natural seamless control. The system is able to output a correct prediction within less than 116 ms latency. 100% PPV has been attained in many trials and is realistically achievable consistently with user practice and/or employing a thresholded majority vote inference. Using transfer learning, these results are achievable after a sensor installation, data recording and network training/fine-tuning routine taking less than 10 min to complete, a reduction of 89.4% in the setup time of the traditional, non-transfer learning approach.

scholarly journals Real Time Path Planning of Robot using Deep Reinforcement Learning

2020 ◽  
Vol 53 (2) ◽  
pp. 15602-15607
Author(s):  
Jeevan Raajan ◽  
P V Srihari ◽  
Jayadev P Satya ◽  
B Bhikkaji ◽  
Ramkrishna Pasumarthy
Keyword(s):  
Reinforcement Learning ◽  
Path Planning ◽  
Real Time ◽  
Time Path

scholarly journals Deep Reinforcement Learning for End-to-End Local Motion Planning of Autonomous Aerial Robots in Unknown Outdoor Environments: Real-Time Flight Experiments

Sensors ◽  
2021 ◽  
Vol 21 (7) ◽  
pp. 2534
Author(s):  
Oualid Doukhi ◽  
Deok-Jin Lee
Keyword(s):  
Reinforcement Learning ◽  
Real Time ◽  
Autonomous Navigation ◽  
Control Technique ◽  
Local Motion ◽  
Aerial Robots ◽  
Novel Approach ◽  
Outdoor Environments ◽  
Aerial Vehicle ◽  
High Level

Autonomous navigation and collision avoidance missions represent a significant challenge for robotics systems as they generally operate in dynamic environments that require a high level of autonomy and flexible decision-making capabilities. This challenge becomes more applicable in micro aerial vehicles (MAVs) due to their limited size and computational power. This paper presents a novel approach for enabling a micro aerial vehicle system equipped with a laser range finder to autonomously navigate among obstacles and achieve a user-specified goal location in a GPS-denied environment, without the need for mapping or path planning. The proposed system uses an actor–critic-based reinforcement learning technique to train the aerial robot in a Gazebo simulator to perform a point-goal navigation task by directly mapping the noisy MAV’s state and laser scan measurements to continuous motion control. The obtained policy can perform collision-free flight in the real world while being trained entirely on a 3D simulator. Intensive simulations and real-time experiments were conducted and compared with a nonlinear model predictive control technique to show the generalization capabilities to new unseen environments, and robustness against localization noise. The obtained results demonstrate our system’s effectiveness in flying safely and reaching the desired points by planning smooth forward linear velocity and heading rates.

Sign in / Sign up

Export Citation Format

Share Document