Developing and Evaluating an Augmented Reality Interface to Assist the Joint Tactical Air Controller by Applying Human Performance Models

2018 ◽  
Vol 62 (1) ◽  
pp. 686-690
Author(s):  
Chris Wickens ◽  
Gaia Dempsey ◽  
Andrew Pringle ◽  
Lucas Kazansky ◽  
Stefanie Hutka
Keyword(s):  
Augmented Reality ◽  
Cognitive Process ◽  
Figure Of Merit ◽  
Human Performance ◽  
Situational Awareness ◽  
Frame Of Reference ◽  
Relative Importance ◽  
Performance Models ◽  
Head Mounted Display ◽  
Cognitive Operations

We developed a 3D augmented reality head mounted display (DARSADS-SVS HMD) interface to support the Joint Tactical Air Controller (JTAC). The JTAC’s job is to integrate information about enemy attack units and nearby friendly forces and direct aircraft equipped with weapons to neutralize the enemy via close air support (CAS), while also safely routing air traffic. The JTAC’s numerous and often overlapping tasks involve maintaining detailed situational awareness (SA) of a large quantity of information, and making rapid decisions that carry life-or-death consequences. Thus, the JTAC role requires many different cognitive operations across different mission phases. Designing an effective human-factored system that supports maximum SA while minimizing cognitive load required us to harness computational cognitive models of SA-supporting visual scanning, display layout, 3D frame-of-reference transformations, clutter, legibility and working memory. We applied such models to different phases of the JTAC mission (e.g., airspace management, call-for-fire), establishing a Figure of Merit (FOM) for each given design by summing FOMs across models, thus creating a mechanism to evaluate designs based upon their balanced impact on competing cognitive drivers. Models were differentially weighted for each phase, according to the relative importance of the relevant cognitive process to the phase in question. In this research paper, we illustrate two such design comparisons.

Augmented Reality Procedure Assistance System for Operator Training and Simulation

2020 ◽  
Vol 64 (1) ◽  
pp. 1176-1180
Author(s):  
Eugene Hayden ◽  
Kang Wang ◽  
Chengjie Wu ◽  
Shi Cao
Keyword(s):  
Augmented Reality ◽  
Situation Awareness ◽  
Mental Workload ◽  
Field Of View ◽  
Task Completion ◽  
Operator Training ◽  
Head Mounted Display ◽  
Subject Design ◽  
Procedural Tasks ◽  
Speed Accuracy

This study explores the design, implementation, and evaluation of an Augmented Reality (AR) prototype that assists novice operators in performing procedural tasks in simulator environments. The prototype uses an optical see-through head-mounted display (OST HMD) in conjunction with a simulator display to supplement sequences of interactive visual and attention-guiding cues to the operator’s field of view. We used a 2x2 within-subject design to test two conditions: with/without AR-cues, each condition had a voice assistant and two procedural tasks (preflight and landing). An experiment examined twenty-six novice operators. The results demonstrated that augmented reality had benefits in terms of improved situation awareness and accuracy, however, it yielded longer task completion time by creating a speed-accuracy trade-off effect in favour of accuracy. No significant effect on mental workload is found. The results suggest that augmented reality systems have the potential to be used by a wider audience of operators.

Validating Human Performance Models: Building a House without Bricks

2020 ◽  
Vol 64 (1) ◽  
pp. 1511-1514
Author(s):  
Richard Steinberg ◽  
Raytheon Company ◽  
Alice Diggs ◽  
Raytheon Company ◽  
Jade Driggs
Keyword(s):  
Air Force ◽  
Human Performance ◽  
Control Program ◽  
Real Data ◽  
Performance Model ◽  
Force Model ◽  
Performance Models ◽  
Actual System ◽  
Support Design ◽  
Trade Offs

Verification and validation (V&V) for human performance models (HPMs) can be likened to building a house with no bricks, since it is difficult to obtain metrics to validate a model when the system is still in development. HPMs are effective for performing trade-offs between the human system designs factors including number of operators needed, the role of automated tasks versus operator tasks, and member task responsibilities required to operate a system. On a recent government contract, our team used a human performance model to provide additional analysis beyond traditional trade studies. Our team verified the contractually mandated staff size for using the system. This task demanded that the model have sufficient fidelity to provide information for high confidence staffing decisions. It required a method for verifying and validating the model and its results to ensure that it accurately reflected the real world. The situation caused a dilemma because there was no actual system to gather real data to use to validate the model. It is a challenge to validate human performance models, since they support design decisions prior to system. For example, crew models are typically inform the design, staffing needs, and the requirements for each operator’s user interface prior to development. This paper discusses a successful case study for how our team met the V&V challenges with the US Air Force model accreditation authority and successfully accredited our human performance model with enough fidelity for requirements testing on an Air Force Command and Control program.

scholarly journals Development of training method for vessel traffic service based on cognitive process

2021 ◽  
Author(s):  
Binbing Song ◽  
Hiroko Itoh ◽  
Yasumi Kawamura
Keyword(s):  
Cognitive Processes ◽  
Cognitive Process ◽  
Process Model ◽  
Situational Awareness ◽  
International Association ◽  
Tokyo Bay ◽  
Training Methods ◽  
Training Method ◽  
Oral Transmission ◽  
Maritime Traffic

AbstractVessel traffic service (VTS) is important to protect the safety of maritime traffic. Along with the expansion of monitoring area per VTS operator in Tokyo Bay, Japan, inexperienced operators must acquire the ability to quickly and accurately detect conditions that requires attention (CRAs) from a monitoring screen. In our previous study (Song B, Itoh H, Kawamura Y, Fukuto J (2018) Analysis of Cognitive Processes of Operators of Vessel Traffic Service. In: Proceedings of the 2018 International Association of Institutes of Navigation. IAIN 2018, pp 529–534, Song et al., J Jpn Inst Navig 140:48–54, 2019), we established a task analysis method based on the assumption that the cognitive process model consists of three stages: “situational awareness”, “situation judgment”, and “decision making”. A simulation experiment was conducted for VTS operators with different levels of ability and their cognitive processes were compared based on the observation of eye movements. The results showed that the inexperienced operators’ abilities to predict situation changes were lower. And it was considered that oral transmission of the knowledge is difficult, thus new training methods are needed to help the inexperienced operators to understand the prediction methods of experienced operators. In this study, based on the cognitive process of an experienced operator, we analyzed the prediction procedures of situation changes and developed an educational tool called vessel traffic routine (VTR). The training method learning VTR aims to quickly improve inexperienced VTS operators’ abilities to predict situation changes. A simulation verification experiment of the VTR effect was conducted for four inexperienced operators, who were divided into two groups with and without prior explanation of VTR. By evaluating the cognitive processes of inexperienced operators, it was confirmed that those given prior explanations of VTR were better at detecting CRAs.

scholarly journals Fooling the size–weight illusion—Using augmented reality to eliminate the effect of size on perceptions of heaviness and sensorimotor prediction

2021 ◽  
Author(s):  
Nina Rohrbach ◽  
Joachim Hermsdörfer ◽  
Lisa-Marie Huber ◽  
Annika Thierfelder ◽  
Gavin Buckingham
Keyword(s):  
Augmented Reality ◽  
Visual Cues ◽  
Equal Mass ◽  
Small Object ◽  
Verbal Reports ◽  
Fully Integrated ◽  
Head Mounted Display ◽  
Lift Off ◽  
Cognitive Knowledge ◽  
Weight Illusion

AbstractAugmented reality, whereby computer-generated images are overlaid onto the physical environment, is becoming significant part of the world of education and training. Little is known, however, about how these external images are treated by the sensorimotor system of the user – are they fully integrated into the external environmental cues, or largely ignored by low-level perceptual and motor processes? Here, we examined this question in the context of the size–weight illusion (SWI). Thirty-two participants repeatedly lifted and reported the heaviness of two cubes of unequal volume but equal mass in alternation. Half of the participants saw semi-transparent equally sized holographic cubes superimposed onto the physical cubes through a head-mounted display. Fingertip force rates were measured prior to lift-off to determine how the holograms influenced sensorimotor prediction, while verbal reports of heaviness after each lift indicated how the holographic size cues influenced the SWI. As expected, participants who lifted without augmented visual cues lifted the large object at a higher rate of force than the small object on early lifts and experienced a robust SWI across all trials. In contrast, participants who lifted the (apparently equal-sized) augmented cubes used similar force rates for each object. Furthermore, they experienced no SWI during the first lifts of the objects, with a SWI developing over repeated trials. These results indicate that holographic cues initially dominate physical cues and cognitive knowledge, but are dismissed when conflicting with cues from other senses.

Diminishing Reality: Potential Benefits and Risks

2021 ◽  
Vol 65 (1) ◽  
pp. 164-168
Author(s):  
I. Murph ◽  
M. McDonald ◽  
K. Richardson ◽  
M. Wilkinson ◽  
S. Robertson ◽  
...  
Keyword(s):  
Augmented Reality ◽  
Situational Awareness ◽  
Relevant Information ◽  
Attentional Focus ◽  
Primary Task ◽  
Training Methods ◽  
Cognitive Aids ◽  
Potential Benefits ◽  
Study Participants ◽  
Completion Times

Within distracting environments, it is difficult to maintain attentional focus on complex tasks. Cognitive aids can support attention by adding relevant information to the environment, such as via augmented reality (AR). However, there may be a benefit in removing elements from the environment, such as irrelevant alarms, displays, and conversations. De-emphasis of distracting elements is a type of AR called Diminished Reality (DR). Although de-emphasizing distraction may help focus on a primary task, it may also reduce situational awareness (SA) of other activities that may become relevant. In the current study, participants will assemble a medical ventilator during a simulated emergency while experiencing varying levels of DR. Participants will also be probed to assess secondary SA. We anticipate that participants will have better accuracy and completion times in the full DR conditions but their SA will suffer. Future applications include the design of future DR systems and improved training methods.

Understanding aviation operators’ variability in advanced systems

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Maria Papanikou ◽  
Utku Kale ◽  
András Nagy ◽  
Konstantinos Stamoulis
Keyword(s):  
Human Performance ◽  
Situational Awareness ◽  
Air Traffic ◽  
Content Type ◽  
Homogeneous Groups ◽  
Professional Groups ◽  
Performance Shaping Factors ◽  
Generational Changes ◽  
And Performance ◽  
Practical Implications

Purpose This study aims to identify variability in aviation operators in order to gain greater understanding of the changes in aviation professional groups. Research has commonly addressed human factors and automation in broad categories according to a group’s function (e.g., pilots, air traffic controllers [ATCOs], engineers). Accordingly, pilots and Air Traffic Controls (ATCOs) have been treated as homogeneous groups with a set of characteristics. Currently, critical themes of human performance in light of systems’ developments place the emphasis on quality training for improved situational awareness (SA), decision-making and cognitive load. Design/methodology/approach As key solutions centre on the increased understanding and preparedness of operators through quality training, the authors deploy an iterative mixed methodology to reveal generational changes of pilots and ATCOs. In total, 46 participants were included in the qualitative instrument and 70 in the quantitative one. Preceding their triangulation, the qualitative data were analysed using NVivo and the quantitative analysis was aided through descriptive statistics. Findings The results show that there is a generational gap between old and new generations of operators. Although positive views on advanced systems are being expressed, concerns about cognitive capabilities in the new systems, training and skills gaps, workload and role implications are presented. Practical implications The practical implications of this study extend to different profiles of operators that collaborate either directly or indirectly and that are critical to aviation safety. Specific implications are targeted on automation complacency, bias and managing information load, and training aspects where quality training can be aided by better understanding the occupational transitions under advanced systems. Originality/value In this paper, the authors aimed to understand the changing nature of the operators’ profession within the advanced technological context, and the perceptions and performance-shaping factors of pilots and ATCOs to define the generational changes.

Sign in / Sign up

Export Citation Format

Share Document