Measuring Situation Awareness in Training Systems: A Multivariate Approach
The Virtual Environment for Submarine Ship Handling Training (VESUB) project developed, demonstrated, and evaluated the training potential of a VE-based training system. Based on perceptual and cognitive task analyses, and input from subject matter experts, 15 performance variables were developed. These variables were combined and analyzed using multivariate techniques that provided a measure of situation awareness and general ship handling ability. These 2 new variables were then analyzed using an ANOVA with experience (novice or experienced) as the between subjects variable, and scenario session (pre-training or post-training) as the within subjects variable. The results indicated a significant improvement in situation awareness and general ship handling ability for both novice and experienced trainees. The VESUB training system provided the training needed to produce significant performance improvements for novice personnel who knew very little about ship handling, and also provided significant refresher training for experienced personnel. The use of multivariate analyses of specific embedded task measures provided an empirical link between situation awareness and human performance.