In an adversarial multi-robot task, such as playing robot soccer, decisions for team and single-robot behaviour must be made quickly to take advantage of short-term fortuitous events. When no such opportunities exist, the team must execute sequences of coordinated team action that increases the likelihood of future opportunities. A hierarchical architecture, called STP, has been developed to control an autonomous team of robots operating in an adversarial environment. STP consists of skills for executing the low-level actions that make up robot behaviour, tactics for determining what skills to execute, and plays for coordinating synchronized activity among team members. The STP architecture combines each of these components to achieve autonomous team control. Moreover, the STP hierarchy allows for fast team response in adversarial environments while carrying out actions with longer goals. This article presents the STP architecture for controlling an autonomous robot team in a dynamic adversarial task that allows for coordinated team activity towards long-term goals, with the ability to respond rapidly to dynamic events. Secondly, the subcomponent of skills and tactics is presented as a generalized single-robot control hierarchy for hierarchical problem decomposition with flexible control policy implementation and reuse. Thirdly, the play techniques contribute as a generalized method for encoding and synchronizing team behaviour, providing multiple competing team responses, and for supporting effective strategy adaptation against opponent teams. STP has been fully implemented on a robot platform and thoroughly tested against a variety of unknown opponent teams in a number of RoboCup robot soccer competitions. These competition results are presented as a mechanism to analyse the performance of STP in a real setting.
Analysis of Methods for Incremental Policy Refinement by Kinesthetic Guidance