scholarly journals Gesture commands for controlling high-level UAV behavior

2021 ◽  
Vol 3 (6) ◽  
Author(s):  
John Akagi ◽  
T. Devon Morris ◽  
Brady Moon ◽  
Xingguang Chen ◽  
Cameron K. Peterson
Keyword(s):  
Search Algorithm ◽  
Dynamic Environment ◽  
List Type ◽  
Hardware In The Loop ◽  
Monte Carlo Tree Search ◽  
Natural Interface ◽  
Constrained Environments ◽  
Novel Variant ◽  
High Level ◽  
Tree Search Algorithm

Abstract Directing groups of unmanned air vehicles (UAVs) is a task that typically requires the full attention of several operators. This can be prohibitive in situations where an operator must pay attention to their surroundings. In this paper we present a gesture device that assists operators in commanding UAVs in focus-constrained environments. The operator influences the UAVs’ behavior by using intuitive hand gesture movements. Gestures are captured using an accelerometer and gyroscope and then classified using a logistic regression model. Ten gestures were chosen to provide behaviors for a group of fixed-wing UAVs. These behaviors specified various searching, following, and tracking patterns that could be used in a dynamic environment. A novel variant of the Monte Carlo Tree Search algorithm was developed to autonomously plan the paths of the cooperating UAVs. These autonomy algorithms were executed when their corresponding gesture was recognized by the gesture device. The gesture device was trained to classify the ten gestures and accurately identified them 95% of the time. Each of the behaviors associated with the gestures was tested in hardware-in-the-loop simulations and the ability to dynamically switch between them was demonstrated. The results show that the system can be used as a natural interface to assist an operator in directing a fleet of UAVs. Article highlights A gesture device was created that enables operators to command a group of UAVs in focus-constrained environments. Each gesture triggers high-level commands that direct a UAV group to execute complex behaviors. Software simulations and hardware-in-the-loop testing shows the device is effective in directing UAV groups.

scholarly journals Enhanced Reinforcement Learning Method Combining One-Hot Encoding-Based Vectors for CNN-Based Alternative High-Level Decisions

2021 ◽  
Vol 11 (3) ◽  
pp. 1291
Author(s):  
Bonwoo Gu ◽  
Yunsick Sung
Keyword(s):  
Reinforcement Learning ◽  
Search Algorithm ◽  
Classification Criteria ◽  
Tree Search ◽  
Learning Method ◽  
Board Game ◽  
Ancient China ◽  
Monte Carlo Tree Search ◽  
High Level ◽  
Tree Search Algorithm

Gomoku is a two-player board game that originated in ancient China. There are various cases of developing Gomoku using artificial intelligence, such as a genetic algorithm and a tree search algorithm. Alpha-Gomoku, Gomoku AI built with Alpha-Go’s algorithm, defines all possible situations in the Gomoku board using Monte-Carlo tree search (MCTS), and minimizes the probability of learning other correct answers in the duplicated Gomoku board situation. However, in the tree search algorithm, the accuracy drops, because the classification criteria are manually set. In this paper, we propose an improved reinforcement learning-based high-level decision approach using convolutional neural networks (CNN). The proposed algorithm expresses each state as One-Hot Encoding based vectors and determines the state of the Gomoku board by combining the similar state of One-Hot Encoding based vectors. Thus, in a case where a stone that is determined by CNN has already been placed or cannot be placed, we suggest a method for selecting an alternative. We verify the proposed method of Gomoku AI in GuPyEngine, a Python-based 3D simulation platform.

scholarly journals Simple Regret Optimization in Online Planning for Markov Decision Processes

2014 ◽  
Vol 51 ◽  
pp. 165-205 ◽  
Author(s):  
Z. Feldman ◽  
C. Domshlak
Keyword(s):  
Markov Decision Processes ◽  
State Of The Art ◽  
Search Algorithm ◽  
Empirical Evaluation ◽  
Decision Processes ◽  
Monte Carlo Tree Search ◽  
Performance Loss ◽  
Online Planning ◽  
Markov Decision ◽  
High Level

We consider online planning in Markov decision processes (MDPs). In online planning, the agent focuses on its current state only, deliberates about the set of possible policies from that state onwards and, when interrupted, uses the outcome of that exploratory deliberation to choose what action to perform next. Formally, the performance of algorithms for online planning is assessed in terms of simple regret, the agent's expected performance loss when the chosen action, rather than an optimal one, is followed. To date, state-of-the-art algorithms for online planning in general MDPs are either best effort, or guarantee only polynomial-rate reduction of simple regret over time. Here we introduce a new Monte-Carlo tree search algorithm, BRUE, that guarantees exponential-rate and smooth reduction of simple regret. At a high level, BRUE is based on a simple yet non-standard state-space sampling scheme, MCTS2e, in which different parts of each sample are dedicated to different exploratory objectives. We further extend BRUE with a variant of ``learning by forgetting.'' The resulting parametrized algorithm, BRUE(alpha), exhibits even more attractive formal guarantees than BRUE. Our empirical evaluation shows that both BRUE and its generalization, BRUE(alpha), are also very effective in practice and compare favorably to the state-of-the-art.

Point-Based Monte Carto Online Planning in POMDPs

2013 ◽  
Vol 846-847 ◽  
pp. 1388-1391
Author(s):  
Bo Wu ◽  
Yan Peng Feng ◽  
Hong Yan Zheng
Keyword(s):  
Mean Squared Error ◽  
Search Algorithm ◽  
Search Tree ◽  
Real Time System ◽  
Monte Carlo Tree Search ◽  
Online Planning ◽  
Markov Decision ◽  
Partially Observable ◽  
Belief States ◽  
Tree Search Algorithm

The online planning and learning in partially observable Markov decision processes are often intractable because belief states space has two curses: dimensionality and history. In order to address this problem, this paper proposes a point-based Monte Carto online planning approach in POMDPs. This approach involves performing value backup at specific reachable belief points, rather than over the entire belief simplex, to speed up computation processes. Then Monte Carlo tree search algorithm is exploited to share the value of actions across each subtree of the search tree so as to minimise the mean squared error. The experimental results show that the proposed algorithm is effective in real-time system.

Sign in / Sign up

Export Citation Format

Share Document