The Role of Visual and Auditory Communication in the Performance of a Joint Team Task

Objective This study examines visual, auditory, and the combination of both (bimodal) coupling modes in the performance of a two-person perceptual-motor task, in which one person provides the perceptual inputs and the other the motor inputs. Background Parking a plane or landing a helicopter on a mountain top requires one person to provide motor inputs while another person provides perceptual inputs. Perceptual inputs are communicated either visually, auditorily, or through both cues. Methods One participant drove a remote-controlled car around an obstacle and through a target, while another participant provided auditory, visual, or bimodal cues for steering and acceleration. Difficulty was manipulated using target size. Performance (trial time, path variability), cue rate, and spatial ability were measured. Results Visual coupling outperformed auditory coupling. Bimodal performance was best in the most difficult task condition but also high in the easiest condition. Cue rate predicted performance in all coupling modes. Drivers with lower spatial ability required a faster auditory cue rate, whereas drivers with higher ability performed best with a lower rate. Conclusion Visual cues result in better performance when only one coupling mode is available. As predicted by multiple resource theory, when both cues are available, performance depends more on auditory cueing. In particular, drivers must be able to transform auditory cues into spatial actions. Application Spotters should be trained to provide an appropriate cue rate to match the spatial ability of the driver or pilot. Auditory cues can enhance visual communication when the interpersonal task is visual with spatial outputs.

Gazing at the Gazers: An Investigation of Travel Advertisement Modality Interference

Tourists are often exposed to imagery of a destination through advertisement modalities such as texts and pictures before they visit the place. This research used an eye-tracking technique to examine the effect of spatial and modality configurations on tourist destination advertisement attention and travel intention. Drawing on multiple resource theory and based on two 4 (spatial displacement) × 3 (modality) experiments, results reveal that spatial location and modality have an impact on tourist attention with respect to their dwell time and fixation counts, which lead to greater propensity to visit a place. In addition, tourists paid more attention to advertisements that occupy congruent attentional resources than to those using interference resources.

Multiple Resource Loading and Auditory Preemption during a Continuous Signal Detection Task

Performance is typically reduced for dual-versus single-tasks, and potential interference between tasks can often be predicted by resource theories such as Multiple Resource Theory (MRT). MRT suggests that performance of two tasks drawing from the same set of cognitive resources is reduced in comparison to performance of two drawing on separate resource pools (e.g., visual-visual compared to visual-auditory). One drawback of MRT is the effect of auditory preemption on an ongoing visual task: whereas MRT expects no interference between auditory and visual tasks, auditory preemption theory posits a cost of attentional switching which may reduce performance. We investigated the effect of auditory preemption by administering an ongoing visual detection task and an interrupting auditory processing task. Results support the theory of auditory preemption as shown by a decline of response criterion ( C), but not perceptual sensitivity ( d’). MRT is also partially supported by an increase in perceived workload.

Effects of Non-Driving Related Task Modalities on Takeover Performance in Highly Automated Driving

Objective: Aim of the study was to evaluate the impact of different non-driving related tasks (NDR tasks) on takeover performance in highly automated driving. Background: During highly automated driving, it is allowed to engage in NDR tasks temporarily. However, drivers must be able to take over control when reaching a system limit. There is evidence that the type of NDR task has an impact on takeover performance, but little is known about the specific task characteristics that account for performance decrements. Method: Thirty participants drove in a simulator using a highly automated driving system. Each participant faced five critical takeover situations. Based on assumptions of Wickens’s multiple resource theory, stimulus and response modalities of a prototypical NDR task were systematically manipulated. Additionally, in one experimental group, the task was locked out simultaneously with the takeover request. Results: Task modalities had significant effects on several measures of takeover performance. A visual-manual texting task degraded performance the most, particularly when performed handheld. In contrast, takeover performance with an auditory-vocal task was comparable to a baseline without any task. Task lockout was associated with faster hands-on-wheel times but not altered brake response times. Conclusion: Results showed that NDR task modalities are relevant factors for takeover performance. An NDR task lockout was highly accepted by the drivers and showed moderate benefits for the first takeover reaction. Application: Knowledge about the impact of NDR task characteristics is an enabler for adaptive takeover concepts. In addition, it might help regulators to make decisions on allowed NDR tasks during automated driving.