Occupant Monitoring in Automated Vehicles: Classification of Situation Awareness Based on Head Movements While Cornering

We tested whether head-movements under automated driving can be used to classify a vehicle occupant as either situation-aware or unaware. While manually cornering, an active driver’s head tilt correlates with the road angle which serves as a visual reference, whereas an inactive passenger’s head follows the g-forces. Transferred to partial/conditional automation, the question arises whether aware occupant’s head-movements are comparable to drivers and if this can be used for classification. In a driving-simulator-study (n=43, within-subject design), four scenarios were used to generate or deteriorate situation awareness (manipulation checked). Recurrent neural networks were trained with the resulting head-movements. Inference statistics were used to extract the discriminating feature, ensuring explainability. A very accurate classification was achieved and the mean side rotation-rate was identified as the most differentiating factor. Aware occupants behave more like drivers. Therefore, head-movements can be used to classify situation awareness in experimental settings but also in real driving.

Download Full-text

Investigating the Impacts of Road Traffic Conditions and Driver’s Characteristics on Automated Vehicle Takeover Time and Quality Using a Driving Simulator

Journal of Advanced Transportation ◽

10.1155/2021/8859553 ◽

2021 ◽

Vol 2021 ◽

pp. 1-13

Author(s):

Jaehyun Jason So ◽

Sungho Park ◽

Jonghwa Kim ◽

Jejin Park ◽

Ilsoo Yun

Keyword(s):

Driving Simulator ◽

Road Traffic ◽

Vehicle Control ◽

Automated Vehicles ◽

Automated Driving ◽

Automated Vehicle ◽

The Road ◽

Traffic Conditions ◽

Vehicle System ◽

Driver’S Characteristics

This study investigates the impacts of road traffic conditions and driver’s characteristics on the takeover time in automated vehicles using a driving simulator. Automated vehicles are barely expected to maintain their fully automated driving capability at all times based on the current technologies, and the automated vehicle system transfers the vehicle control to a driver when the system can no longer be automatically operated. The takeover time is the duration from when the driver requested the vehicle control transition from the automated vehicle system to when the driver takes full control of the vehicle. This study assumes that the takeover time can vary according to the driver’s characteristics and the road traffic conditions; the assessment is undertaken with various participants having different characteristics in various traffic volume conditions and road geometry conditions. To this end, 25 km of the northbound road section between Osan Interchange and Dongtan Junction on Gyeongbu Expressway in Korea is modeled in the driving simulator; the experiment participants are asked to drive the vehicle and take a response following a certain triggering event in the virtual driving environment. The results showed that the level of service and road curvature do not affect the takeover time itself, but they significantly affect the stabilization time, that is, a duration for a driver to become stable and recover to a normal state. Furthermore, age affected the takeover time, indicating that aged drivers are likely to slowly respond to a certain takeover situation, compared to the younger drivers. With these findings, this study emphasizes the importance of having effective countermeasures and driver interface to monitor drivers in the automated vehicle system; therefore, an early and effective alarm system to alert drivers for the vehicle takeover can secure enough time for stable recovery to manual driving and ultimately to achieve safety during the takeover.

Download Full-text

Analysis of Visual Scanning Patterns Comparing Drivers of Simulated L2 and L0 Systems

Transportation Research Record Journal of the Transportation Research Board ◽

10.1177/0361198119852339 ◽

2019 ◽

Vol 2673 (10) ◽

pp. 755-761 ◽

Cited By ~ 1

Author(s):

Nathan Hatfield ◽

Yusuke Yamani ◽

Dakota B. Palmer ◽

Sarah Yahoodik ◽

Veronica Vasquez ◽

...

Keyword(s):

Eye Movements ◽

Driving Simulator ◽

Steering Wheel ◽

Automated Driving ◽

Young Drivers ◽

Hands On ◽

Visual Sampling ◽

The Mean ◽

Lateral Positioning ◽

Perception Training

Automated driving systems (ADS) partially or fully perform driving functions. Yet, the effects of ADS on drivers’ visual sampling patterns to the forward roadway remain underexplored. This study examined the eye movements of 24 young drivers during either manual (L0) or partially automated driving (L2) in a driving simulator. After completing a hazard anticipation training program, Road Awareness and Perception Training, drivers in both groups navigated a single simulated drive consisting of four environment types: highway, town, rural, and residential. Drivers of the simulated L2 system were instructed to keep their hands on the steering wheel and told that the system controls the speed and lateral positioning of the vehicle while avoiding potential threats on the forward roadway. The data indicate that the drivers produced fewer fixations during automated driving compared with manual driving. However, the breadth of horizontal and vertical eye movements and the mean fixation durations did not strongly support the null results between the two conditions. Existing hazard anticipation training programs may effectively protect drivers of partially automated systems from inattention to the forward roadway.

Download Full-text

Influence of automated driving on driver’s own localization: a driving simulator study

Journal of Intelligent and Connected Vehicles ◽

10.1108/jicv-08-2018-0006 ◽

2018 ◽

Vol 1 (3) ◽

pp. 99-106 ◽

Cited By ~ 2

Author(s):

Ryuichi Umeno ◽

Makoto Itoh ◽

Satoshi Kitazaki

Keyword(s):

Situation Awareness ◽

Driving Simulator ◽

Automated Driving ◽

Operation System ◽

Content Type ◽

Automatic Driving ◽

Driver Drowsiness ◽

Level 3 ◽

Fundamental Research ◽

Operation Group

Purpose Level 3 automated driving, which has been defined by the Society of Automotive Engineers, may cause driver drowsiness or lack of situation awareness, which can make it difficult for the driver to recognize where he/she is. Therefore, the purpose of this study was to conduct an experimental study with a driving simulator to investigate whether automated driving affects the driver’s own localization compared to manual driving. Design/methodology/approach Seventeen drivers were divided into the automated operation group and manual operation group. Drivers in each group were instructed to travel along the expressway and proceed to the specified destinations. The automated operation group was forced to select a course after receiving a Request to Intervene (RtI) from an automated driving system. Findings A driver who used the automated operation system tended to not take over the driving operation correctly when a lane change is immediately required after the RtI. Originality/value This is a fundamental research that examined how the automated driving operation affects the driver's own localization. The experimental results suggest that it is not enough to simply issue an RtI, and it is necessary to tell the driver what kind of circumstances he/she is in and what they should do next through the HMI. This conclusion can be taken into consideration for engineers who design automatic driving vehicles.

Download Full-text

Research on the classification for road traffic visibility based on the characteristics of driving behaviour – a driving simulator experiment

Journal of Intelligent and Connected Vehicles ◽

10.1108/jicv-10-2019-0011 ◽

2020 ◽

Vol 3 (1) ◽

pp. 30-36

Author(s):

Kun Wang ◽

Weihua Zhang ◽

Zhongxiang Feng ◽

Cheng Wang

Keyword(s):

Traffic Safety ◽

Driving Simulator ◽

Road Traffic ◽

Driving Behavior ◽

Control Measures ◽

Content Type ◽

The Road ◽

Road Traffic Safety ◽

Fine Classification

Purpose The purpose of this paper is to perform fine classification of road traffic visibility based on the characteristics of driving behavior under different visibility conditions. Design/methodology/approach A driving simulator experiment was conducted to collect data of speed and lane position. ANOVA was used to explore the difference in driving behavior under different visibility conditions. Findings The results show that only average speed is significantly different under different visibility conditions. With the visibility reducing, the average vehicle speed decreases. The road visibility conditions in a straight segment can be divided into five levels: less than 20, 20-30, 35-60, 60-140 and more than 140 m. The road visibility conditions in a curve segment can be also divided into four levels: less than 20, 20-30, 35-60 and more than 60 m. Originality/value A fine classification of road traffic visibility has been performed, and these classifications help to establish more accurate control measures to ensure road traffic safety under low-visibility conditions.

Download Full-text

Function-Specific Uncertainty Communication in Automated Driving

International Journal of Mobile Human Computer Interaction ◽

10.4018/ijmhci.2019040105 ◽

2019 ◽

Vol 11 (2) ◽

pp. 75-97

Author(s):

Alexander Kunze ◽

Stephen J. Summerskill ◽

Russell Marshall ◽

Ashleigh J. Filtness

Keyword(s):

Augmented Reality ◽

Situation Awareness ◽

Driving Simulator ◽

Automated Driving ◽

Driving Experience ◽

Visual Variables ◽

Simulator Study ◽

The Impact

Conveying the overall uncertainties of automated driving systems was shown to improve trust calibration and situation awareness, resulting in safer takeovers. However, the impact of presenting the uncertainties of multiple system functions has yet to be investigated. Further, existing research lacks recommendations for visualizing uncertainties in a driving context. The first study outlined in this publication investigated the implications of conveying function-specific uncertainties. The results of the driving simulator study indicate that the effects on takeover performance depends on driving experience, with less experienced drivers benefitting most. Interview responses revealed that workload increments are a major inhibitor of these benefits. Based on these findings, the second study explored the suitability of 11 visual variables for an augmented reality-based uncertainty display. The results show that particularly hue and animation-based variables are appropriate for conveying uncertainty changes. The findings inform the design of all displays that show content varying in urgency.

Download Full-text

The Effect of Augmented Reality Cues on Glance Behavior and Driver-Initiated Takeover on SAE Level 2 Automated-Driving

Proceedings of the Human Factors and Ergonomics Society Annual Meeting ◽

10.1177/1071181321651004 ◽

2021 ◽

Vol 65 (1) ◽

pp. 1342-1346

Author(s):

Nayara de Oliveira Faria ◽

Coleman Merenda ◽

Richard Greatbatch ◽

Kyle Tanous ◽

Chihiro Suga ◽

...

Keyword(s):

Augmented Reality ◽

Situation Awareness ◽

Driver Assistance System ◽

User Study ◽

Driving Simulator ◽

Autonomous Vehicle ◽

Driver Assistance ◽

Automated Driving ◽

The Relationship ◽

Level 2

In the present paper, we present a user study with an advanced-driver assistance system (ADAS) using augmented reality (AR) cues to highlight pedestrians and vehicles when approaching intersections of varying complexity. Our major goal is to understand the relationship between the presence and absence of AR, driver-initiated takeover rates and glance behavior when using a SAE Level 2 autonomous vehicle. Therefore, a user-study with eight participants on a medium-fidelity driving simulator was carried out. Overall, we found that AR cues can provide promising means to increase the system transparency, drivers’ situation awareness and trust in the system. Yet, we suggest that the dynamic glance allocation of attention during partially automated vehicles is still challenging for researchers as we still have much to understand and explore when AR cues become a distractor instead of an attention guider.

Download Full-text

Novel Auditory Displays in Highly Automated Vehicles: Sonification Improves Driver Situation Awareness, Perceived Workload, and Overall Experience

Proceedings of the Human Factors and Ergonomics Society Annual Meeting ◽

10.1177/1071181321651071 ◽

2021 ◽

Vol 65 (1) ◽

pp. 586-590

Author(s):

Chihab Nadri ◽

Sangjin Ko ◽

Colin Diggs ◽

Michael Winters ◽

V. K. Sreehari ◽

...

Keyword(s):

Situation Awareness ◽

Driving Simulator ◽

Visual Display ◽

Use Cases ◽

Automated Vehicles ◽

Automated Driving ◽

Auditory Displays ◽

User Perceptions ◽

Young Drivers ◽

Group Interviews

Highly automated driving systems are expected to require the design of new user-vehicle interactions. Sonification can be used to provide contextualized alarms and cues that can increase situation awareness and user experience. In this study, we examined user perceptions of potential use cases for level 4 automated vehicles in online focus group interviews (N=12). Also, in a driving simulator study, we evaluated (1) visual-only display; (2) non-speech with visual display; and (3) speech with visual display with 20 young drivers. Results indicated participants’ interest in the use cases and insight on desired functions in highly automated vehicles. Both audiovisual display conditions resulted in higher situation awareness for drivers than the visual-only condition. Some differences were found between the non-speech and speech conditions suggesting benefits of sonification for both driving and non-driving related auditory use cases. This study will provide guidance on sonification design for highly automated vehicles.

Download Full-text

Function-Specific Uncertainty Communication in Automated Driving

10.4018/978-1-6684-3694-3.ch049 ◽

2022 ◽

pp. 1002-1026

Author(s):

Alexander Kunze ◽

Stephen J. Summerskill ◽

Russell Marshall ◽

Ashleigh J. Filtness

Keyword(s):

Augmented Reality ◽

Situation Awareness ◽

Driving Simulator ◽

Automated Driving ◽

Driving Experience ◽

Visual Variables ◽

Simulator Study ◽

The Impact

Download Full-text

Peripheral Monitoring of Traffic in Conditionally Automated Driving

Proceedings of the Human Factors and Ergonomics Society Annual Meeting ◽

10.1177/1541931218621416 ◽

2018 ◽

Vol 62 (1) ◽

pp. 1828-1832 ◽

Cited By ~ 4

Author(s):

Jonas Radlmayr ◽

Karin Brüch ◽

Kathrin Schmidt ◽

Christine Solbeck ◽

Tristan Wehner

Keyword(s):

Situation Awareness ◽

Automated Vehicles ◽

Automated Driving ◽

Active System ◽

Subject Design ◽

Level 3 ◽

Driving Task

Conditionally automated vehicles (level 3) allow drivers to engage in visual, non-driving related tasks (NDRTs) while the automation is active. System limits require drivers to reengage in the dynamic driving task in take-over situations. If the NDRT is visually engaging, situation awareness (SA) necessary for a successful take-over can decrease. This study analyzed, if the SA of drivers increases while monitoring the surrounding traffic peripherally. A semi-transparent balloon game in the head-up display operationalized the engagement into a visual NDRT with the possibility of peripheral monitoring. In addition, participants without the possibility of monitoring due to simulated heavy fog (second group) were tested along with a third group that could monitor surroundings self-determined without a NDRT. The between-subject design included 57 participants. Results showed that self-determined monitoring leads to higher situation awareness compared to peripheral monitoring and no monitoring. This did not result in better take-over performances.

Download Full-text

Transitions Between Highly Automated and Longitudinally Assisted Driving: The Role of the Initiator in the Fight for Authority

Human Factors The Journal of the Human Factors and Ergonomics Society ◽

10.1177/0018720820946183 ◽

2020 ◽

pp. 001872082094618 ◽

Cited By ~ 1

Author(s):

Davide Maggi ◽

Richard Romano ◽

Oliver Carsten

Keyword(s):

Road Safety ◽

Driving Simulator ◽

Cruise Control ◽

Automated Driving ◽

Safety Issues ◽

The Road ◽

Operational Design ◽

Level 3 ◽

Driving Task

Objective A driving simulator study explored how drivers behaved depending on their initial role during transitions between highly automated driving (HAD) and longitudinally assisted driving (via adaptive cruise control). Background During HAD, drivers might issue a take-over request (TOR), initiating a transition of control that was not planned. Understanding how drivers behave in this situation and, ultimately, the implications on road safety is of paramount importance. Method Sixteen participants were recruited for this study and performed transitions of control between HAD and longitudinally assisted driving in a driving simulator. While comparing how drivers behaved depending on whether or not they were the initiators, different handover strategies were presented to analyze how drivers adapted to variations in the authority level they were granted at various stages of the transitions. Results Whenever they initiated the transition, drivers were more engaged with the driving task and less prone to follow the guidance of the proposed strategies. Moreover, initiating a transition and having the highest authority share during the handover made the drivers more engaged with the driving task and attentive toward the road. Conclusion Handover strategies that retained a larger authority share were more effective whenever the automation initiated the transition. Under driver-initiated transitions, reducing drivers’ authority was detrimental for both performance and comfort. Application As the operational design domain of automated vehicles (Society of Automotive Engineers [SAE] Level 3/4) expands, the drivers might very well fight boredom by taking over spontaneously, introducing safety issues so far not considered but nevertheless very important.

Download Full-text