Project overview. The current study focuses on the nature of team coordination dynamics within all-human teams and Human-Autonomy Teams (HAT) in the context of the development of a fully-fledged synthetic agent that is a computational cognitive model for a three-agent Unmanned Aircraft System (UAS) ground crew. In this study, the relationship between team coordination dynamics and team performance within the HAT and all-human teams is considered. To serve as a teammate, the synthetic agent must be able to communicate and coordinate with its human teammates in a constructive and timely manner (Demir, McNeese, & Cooke, 2016). In this current research, there were three heterogeneous team members who communicated via a text-based communication system to photograph target waypoints. Each team member had a different role: (1) navigator – provides information regarding a flight plan with speed and altitude restrictions of each waypoint; (2) pilot – controls the UAS by adjusting its altitude and airspeed by negotiating with the photographer to take a good photo for the target waypoints; and (3) photographer – screens camera settings, and sends feedback to the other team members regarding the status of target’s photograph. At each target waypoint, this coordination sequence among the team members, called Information-Negotiation-Feedback (INF), is captured by a Kappa Score (Gorman, Amazeen, & Cooke, 2010) that describes the sequence and timing of the information coordination. Three conditions were created that manipulated the pilot role: (1) Synthetic – pilot was the synthetic agent, (2) Control – pilot was a randomly assigned participant, and (3) Experimenter – pilot was an experimenter who was highly experienced with the task and focused on pushing and pulling the information in a timely manner. Method. In this experiment, there were 30 teams (ten teams for each condition): control teams were composed of three participants randomly assigned to each role; synthetic and experimenter teams were composed of only two participants randomly assigned to the navigator and photographer roles. The experiment consisted of five missions (each 40 minutes) in which teams needed to take as many “good” photos as possible of ground targets while avoiding alarms and rule violations. Several measures were obtained from this research, including team performance scores (mission and target level), team process measures (situation awareness, process ratings, communication and coordination), and other measures (teamwork knowledge, workload, and demographics). The research reported here identifies how differences in team coordination, captured by Kappa, relate to performance of all human teams and HAT teams. In this paper, we focus on: (1) target level team performance scores calculated based on the time spent inside a target waypoint to get a good photo; and (2) two team coordination dynamics measures: stability and team communication determinism. Stability was inversely related to the largest Lyapunov Exponent which was estimated by Kappa, that is, the INF coordination sequence. Team communication determinism was estimated from communication data using Joint Recurrence Quantification Analysis (Marwan, Carmen, Thiel, & Kurths, 2007) and served as an index of flexible behavior. Results and discussion. In general, findings indicate that (1) synthetic teams were most stable, followed by experimenter teams, who were moderately stable, and control teams, who were least stable; and (2) extreme stability and instability corresponded to lower levels of performance; experimenter teams performed best, followed by control teams and, then synthetic teams. Thus, synthetic agents could be made more effective if interventions were developed to enhance the flexibility and adaptive nature of HATs (Demir, 2017).
Approaches for assessing communication in human-autonomy teams