Displays for Productive Non-Driving Related Tasks: Visual Behavior and Its Impact in Conditionally Automated Driving

(1) Background: Primary driving tasks are increasingly being handled by vehicle automation so that support for non-driving related tasks (NDRTs) is becoming more and more important. In SAE L3 automation, vehicles can require the driver-passenger to take over driving controls, though. Interfaces for NDRTs must therefore guarantee safe operation and should also support productive work. (2) Method: We conducted a within-subjects driving simulator study (N=53) comparing Heads-Up Displays (HUDs) and Auditory Speech Displays (ASDs) for productive NDRT engagement. In this article, we assess the NDRT displays’ effectiveness by evaluating eye-tracking measures and setting them into relation to workload measures, self-ratings, and NDRT/take-over performance. (3) Results: Our data highlights substantially higher gaze dispersion but more extensive glances on the road center in the auditory condition than the HUD condition during automated driving. We further observed potentially safety-critical glance deviations from the road during take-overs after a HUD was used. These differences are reflected in self-ratings, workload indicators and take-over reaction times, but not in driving performance. (4) Conclusion: NDRT interfaces can influence visual attention even beyond their usage during automated driving. In particular, the HUD has resulted in safety-critical glances during manual driving after take-overs. We found this impacted workload and productivity but not driving performance.

Investigating orthographic versus auditory cross-situational word learning with online and lab-based research

In recent years, cross-situational word learning (CSWL) paradigms have shown that novel words can be learned through implicit statistical learning. So far, CSWL studies using adult populations have focused on the presentation of spoken words (auditory information), however, words can also be learned through their written form (orthographic information). This study compares auditory and orthographic presentation of novel words with different degrees of phonological overlap using the CSWL paradigm. Additionally, we also present a lab-based and online-based approach to testing behavioural experiments. Due to the COVID-19 pandemic, lab testing was prematurely terminated, and testing was continued online using a newly created online testing protocol. Analyses first compared accuracy and response times across modalities, with our findings showing better and faster recognition performance for CSWL when novel words are presented through their written (orthographic condition) than through their spoken forms (auditory condition). As well, Bayesian modelling found that accuracy for the auditory condition was higher online compared to the lab-based experiment, whereas performance in the orthography condition was high in both experiments and generally outperformed the auditory condition. We discuss the implications of our findings for modality of presentation, as well as the benefits of our online testing protocol and its implementation for future research.

Multisensory action effects facilitate the performance of motor sequences

Research has shown that contingent, distinct action effects have a beneficial influence on motor sequence performance. Previous studies showed the beneficial influence of task-irrelevant action effects from one modality (auditory) on motor sequence performance, compared with no task-irrelevant action effects. The present study investigated the influence of task-irrelevant action effects on motor sequence performance from a multiple-modality perspective. We compared motor sequence performances of participants who received different task-irrelevant action effects in an auditory, visual, or audiovisual condition. In the auditory condition, key presses produced tones of a C-major scale that mapped to keys from left to right in ascending order. In the visual condition, key presses produced rectangles in different locations on the screen that mapped to keys from left to right in ascending order. In the audiovisual condition, both tone and rectangle effects were produced simultaneously by key presses. There were advantages for the audiovisual group in motor sequence initiation and execution. The results implied that, compared with unimodal action effects, action effects from multiple sensory modalities can prime an action faster and strengthen associations between successive actions, leading to faster motor sequence performance.

Infant Movement Response to Auditory Rhythm

Rhythmic entrainment occurs when an auditory rhythm drives an internal movement oscillator, thus providing a continuous time reference that improves temporal and spatial movement parameters. Entrainment processes and outcomes are well known for adults, but research is lacking for infants who might benefit from diagnosis and treatment of irregular rhythms within biological, sensorimotor, cognitive, and social domains. The present study used a combination of inertial measurement units and custom-made software to determine the amount, tempo, and regularity of movement in 28 infants aged 6-10 months while they were exposed to silence, an irregular rhythmic cue, or a regular rhythmic cue with tempo changes. We also assessed changes in the infants’ movement parameters following a one-week rhythm training protocol. While results revealed no significant effect of auditory condition on amount or tempo of movement, infant movement was significantly more regular when infants were exposed to 120 bpm (beats per minute) than to an irregular rhythmic cue or a 10% faster rhythmic cue (132 bpm). Infants showed no notable changes in movement amount, tempo, or regularity following one week of training involving auditory and physical rhythm. Overall, infants seem to engage in spontaneous movements with or without auditory rhythm but may not show tempo sensitivity through their movements. Increased movement regularity suggests that 120 bpm may be a preferred tempo for infants, at which they are more likely to demonstrate well-timed movements that may reflect interval entrainment. Infants’ auditory-motor systems appear not to respond to a 1-week rhythm training protocol.

The Influence of Noise Emitted by Vehicles on Pedestrian Crossing Decision-Making: A Study in a Virtual Environment

When crossing a road, pedestrians must detect traffic, combine data coming from different perceptual modalities, evaluate the time envelope for safely cross the street, and monitor the position of oncoming vehicles to perform corrective actions if needed. This study analyzed the influence of noise emitted by vehicles, or its absence, on pedestrians’ crossing decision-making. Experiments were performed in a virtual environment using two road scenarios. Participants were presented with stimuli of approaching vehicles that varied regarding speed, movement patterns, and auditory condition: one concerning the approaching of an electric vehicle, another regarding the approaching of a gasoline combustion vehicle, and, finally, a condition regarding the absence of auditory cues. Participants were tasked with indicating the moment when they decided to cross the street. The results show that, despite the noise variations caused by the type of vehicle and its speed pattern, the participants’ decision to cross was mostly based on vehicle distance. When a vehicle approaches the crosswalk from a short distance and with no occlusion to the pedestrian’s visibility, the sound does not seem to influence the pedestrians’ crossing decision-making.

Replay of Stimulus-specific Temporal Patterns during Associative Memory Formation

Forming a memory often entails the association of recent experience with present events. This recent experience is usually an information-rich and dynamic representation of the world around us. We here show that associating a static cue with a previously shown dynamic stimulus yields a detectable, dynamic representation of this stimulus. We further implicate this representation in the decrease of low-frequency power (∼4–30 Hz) in the ongoing EEG, which is a well-known correlate of successful memory formation. The reappearance of content-specific patterns in desynchronizing brain oscillations was observed in two sensory domains, that is, in a visual condition and in an auditory condition. Together with previous results, these data suggest a mechanism that generalizes across domains and processes, in which the decrease in oscillatory power allows for the dynamic representation of information in ongoing brain oscillations.

Pengembangan Terapi Cermin Pada Latihan Bersepeda Berbasis Virtual Reality Untuk Meningkatkan Gerak Motorik Ekstremitas Atas Pasca Stroke

<p>Latihan bersepeda berbasis virtual reality (VR-Bicycling) bagi penderita pasca stroke merupakan teknologi yang dikembangkan untuk meningkatkan gerak motorik otot. Penerapan VR-Bicycling umumnya untuk melatih gerak motorik ekstremitas bawah daripada gerak motorik ekstremitas atas. Penggunaan terapi cermin (Mirror Therapy-MT) merupakan bentuk terapi gerak motorik ekstremitas atas dengan mempengaruhi propriosepsi otak yang berfikir bahwa eksterimitas atas yang mengalami hemiparesis dapat bergerak baik. Dalam penelitian ini dikembangan modul VR-Bicycling untuk melatih gerak motorik ekstremitas atas berbasis MT dengan mengkombinasikan umpan balik visual dan audio pada lingkungan virtual (Virtual Environment-VE). Umpan balik visual untuk mengetahui waktu tunda dan perbedaan spasial berupa ketepatan lengan virtual dalam menangkap apel atau menghindari bom sepanjang lintasan bersepeda. Sedangkan umpan balik audio menggunakan metronome untuk melatih operabilitas dan kontrol lengan virtual. Kinect digunakan sebagai motion capture gerakan lengan nyata. Pengujian melibatkan sepuluh penderita pasca stroke untuk melatih gerak motorik ekstremitas atas yang mengalami hemiparesis dalam mengontrol gerakan lengan virtual pada VR-Bicycling. Evaluasi sistem dilakukan dengan meminta penderita pasca stroke menjawab daftar pertanyaan yang mengandung faktor operabilitas, konsentrasi, kontrol gerakan, waktu tunda, dan perbedaan spasial dari terapi ekstremitas atas pada VR-Bicycling menggunakan konsep terapi cermin. Dari hasil evaluasi diketahui bahwa waktu tunda dan perbedaan spasial pada terapi ekstremitas atas menggunakan VR-Bicycling dengan MT merupakan dua variabel penting untuk mengontrol arah dan ketepatan gerakan lengan virtual pada VE.</p><p> </p><p class="Judul2"><strong><em>Abstract</em></strong></p><p class="Abstract"><em>A virtual reality bicycling (VR-Bicycling) was encouraged to address motor control and fitness deficits of person post-stroke. The applied of VR-Bicycling for therapy of person post-stroke has focused primarily on improving movement and use of the lower extremity than upper extremity. Mirror therapy (MT) is an instrument that enables a person with an amputation to view a reflection of an intact limb in the visual plane of the missing limb by tricking the brain into believing the missing limb is actually moving. The use of virtual reality (VR) over MT is that VR allows more complete immersion into the illusion, whereas MT requires tight focus and concentration to truly see the illusion as real. In this research, we had developed a VR-Bicycling module based MT for training upper extremity of person post-stroke while doing simulated bicycling. Person post-stroke cycled with guidance of visual cueing (represented as apples and bombs) and auditory condition (provided by metronome) in the virtual environment (VE). We tested ten person post-stroke as they cycled on a stationary bicycle while moving their virtual limb to hit apples or avoid bombs in VE. A VR-Bicycling module measures the motion data of upper extremity by Kinect and evaluate immersion and motor cognition by using factor analysis with three factors consisting of controllability, concentration, sense of delay, and spatial difference. The evaluation result indicates that temporal delay and spatial fluctuation have strong relationship to control virtual limb movement correctly.</em></p>

Sound Emission Level in Spinning Classes and the Influence in the Health of Teachers

Noise presents itself today as the most frequent physical agent in the workplace and its harmfulness is already proven by the literature. Among its main consequences are noise-induced hearing loss that can be caused by any type of excessive sound emission, including those used in Spinning classes. The objective of the present study was to evaluate the sound pressure levels in Spinning classes in Brasília-DF academies, as well as to identify the teacher’s perception of their auditory condition. The Bruel & Kjaer 2250 sound pressure gauge was used to ensure the reliability of measurements. 43 teachers participated and the sound pressure levels were measured at 20 academies. The results indicate that 100% of the academies are working with values above the limits allowed by the current legislation (85dBA). Thus, the higher the weekly workload, the higher the maximum sound pressure level and the equivalent to which the teachers are exposed. Concerning the auditory perception of the teachers, (69.8%) they stated that they did not feel bothered by the loud sound. It is suggested that academies, teachers and students be guided to prevent possible hearing damage and to try to control the noise level in the classroom.

Effects of Visual and Auditory Presentation Styles and Musical Elements on Working Memory as Measured by Monosyllabic Sequential Digit Recall

Information is often paired with music to facilitate memory and learning. However, there is a lack of basic research investigating how visual and auditory presentation styles and musical elements might facilitate recall. The purpose of this study is to isolate and determine the effects of visual and auditory presentation styles and musical elements on working memory as measured by sequential monosyllabic digit recall performance. Recall was tested on 60 undergraduate university students during six different conditions: (a) Visual + Auditory Chant, (b) Visual + Auditory Melody, (c) Visual + Auditory Speech, (d) Auditory Chant, (e) Auditory Melody, and (f) Auditory Speech. There was a significant interaction between presentation style and musical element conditions. There were significant differences between auditory and visual + auditory conditions in the melody and speech conditions but not in the chant condition. In all cases, the auditory condition had more accurate recall than the visual + auditory condition, with recall differences largest during the speech condition. There was no significant difference between chant and melody but significant differences between chant and speech and melody and speech in the visual + auditory condition. In the auditory condition, recall accuracy was lower for speech than for melody or chant. There was no significant difference between chant and melody, chant and speech, or melody and speech in the visual + auditory condition. Congruent with existing research, the addition of visual input likely overloaded working memory resulting in worse recall. Implications for clinical practice, limitations, and suggestions for future research are provided.

Replay of Stimulus Specific Temporal Patterns during Associative Memory Formation

Forming a memory often entails the association of recent experience with present events. This recent experience is usually an information rich and dynamic representation of the world around us. We here show that associating a static cue with a previously shown dynamic stimulus, yields a detectable, dynamic representation of this stimulus in working memory. We further implicate this representation in the decrease of low-frequency power (∼4-30 Hz) in the ongoing electroencephalogram (EEG), which is a well-known correlate of successful memory formation. The maintenance of content specific patterns in desynchronizing brain oscillations was observed in two sensory domains, i.e. in a visual and in an auditory condition. Together with previous results, these data suggest a mechanism that generalizes across domains and processes, in which the decrease in oscillatory power allows for the dynamic representation of information in ongoing brain oscillations.