How team cognition is conceptualized has evolved rapidly in the last decade with the emerging use of a systems approach, moving the focus from the cognition residing in the heads of individuals, to that distributed across the team. This is referred to as ‘distributed cognition’. Increasingly, network approaches are being explored in attempts to model team distributed cognition. The specific domain of interest in the present study is the sociotechnical system within a maritime control room. This comprises human, machine and software agents interacting to interpret sensor data in order to develop a timely and accurate picture of surrounding contacts at sea. To achieve the goal, information is shared or integrated across the maritime control room consoles. The aim of this study was to develop and apply a suite of workload, situation awareness and team performance measures, including network analysis techniques, to examine how the distributed cognition of a team might change as a function of console configuration and information integration within a control room, and how these changes, if any, impact overall team performance. Sixteen teams of six novices conducted two one-hour scenarios operating generic maritime control room positions. Each team completed a one-hour simulation in each of two console configuration layouts with the order counter-balanced (within-subject design). Half the teams conducted the two scenarios in a high integration condition, and half in a low integration condition (between-subjects). The human machine interface (HMI) designs for the high integration condition emerged from a series of task analyses and user-centered design workshops. The emergent cognitively –oriented HMI designs are based on the assumption that each console can freely share information with other consoles. To create an analogue of current, less-integrated, and more stove-piped systems, a low integration condition was created where not all information was shared across consoles, but instead was shared verbally by console operators. Contacts detected at sea were introduced into the simulation and the team’s task was to assess, report and derive a solution (location, course, and speed) for each detected contact. Individual situation awareness was measured through the Situation Present Assessment Method (SPAM) and individual workload through the Air Traffic Workload Indicator Task (ATWIT). Team interaction from the scenarios were video recorded and we applied the Event Analysis of Systemic Teamwork (EAST) approach to examine the task, social and information networks which emerged. Team performance was measured as the accuracy and timeliness of the solutions We found higher information integration lowered average team workload, and improved average team situation awareness and team performance (faster solutions and a more accurate tactical picture). We found no impact of console configuration on team performance or any other dependent measure. The EAST method uncovered patterns in the network analysis that are potentially explanatory for the team workload, situation awareness and performance findings as a function of the information integration manipulation. This experiment showed that there can be reductions in workload, and improvements to situation awareness and performance when information is shared between consoles in a considered design. This has implications for HMI design within a team setting. The set of diagnostic metrics developed were largely effective in examining teamwork and team performance. Acknowledgements. The authors would like to thank Justin Hill (Royal Australian Navy) for his subject matter expertise, Graeme Muller (elmTEK) for his software, technical and infrastructure support, David Munro-Ford (Total Technology Partners) for his simulation programming, Dr Aaron Roberts for his advice on general aspects of the experiment, and Professor Paul Salmon for his advice on EAST.
Comparing control room operators' and experts' assessment of team performance using structured task-specific observation protocols and scenario replay
