scholarly journals Multimodal-Multisensory Experiments: Design and Implementation

2020 ◽  
Author(s):  
Moein Razavi ◽  
Takashi Yamauchi ◽  
Vahid Janfaza ◽  
Anton Leontyev ◽  
Shanle Longmire-Monford ◽  
...  
Keyword(s):  
Virtual Reality ◽  
Response Time ◽  
Galvanic Skin Response ◽  
Low Cost ◽  
Body Movement ◽  
Human Mind ◽  
Body Motion ◽  
Mouse Cursor ◽  
Skin Response ◽  
Electroencephalogram Eeg

AbstractThe human mind is multimodal. Yet most behavioral studies rely on century-old measures of behavior - task accuracy and latency (response time). Multimodal and multisensory analysis of human behavior creates a better understanding of how the mind works. The problem is that designing and implementing these experiments is technically complex and costly. This paper introduces versatile and economical means of developing multimodal-multisensory human experiments. We provide an experimental design framework that automatically integrates and synchronizes measures including electroencephalogram (EEG), galvanic skin response (GSR), eye-tracking, virtual reality (VR), body movement, mouse/cursor motion and response time. Unlike proprietary systems (e.g., iMotions), our system is free and open-source; it integrates PsychoPy, Unity and Lab Streaming Layer (LSL). The system embeds LSL inside PsychoPy/Unity for the synchronization of multiple sensory signals - gaze motion, electroencephalogram (EEG), galvanic skin response (GSR), mouse/cursor movement, and body motion - with low-cost consumer-grade devices in a simple behavioral task designed by PsychoPy and a virtual reality environment designed by Unity. This tutorial shows a step-by-step process by which a complex multimodal-multisensory experiment can be designed and implemented in a few hours. When conducting the experiment, all of the data synchronization and recoding of the data to disk will be done automatically.

scholarly journals Multimodal-Multisensory Experiments

2020 ◽  
Author(s):  
Moein Razavi ◽  
Takashi Yamauchi ◽  
Vahid Janfaza ◽  
Anton Leontyev ◽  
Shanle Longmire-Monford ◽  
...  
Keyword(s):  
Virtual Reality ◽  
Response Time ◽  
Galvanic Skin Response ◽  
Low Cost ◽  
Body Movement ◽  
Human Mind ◽  
Body Motion ◽  
Mouse Cursor ◽  
Skin Response ◽  
Electroencephalogram Eeg

The human mind is multimodal. Yet most behavioral studies rely on century-old measures of behavior—task accuracy and latency (response time). Multimodal and multisensory analysis of human behavior creates a better understanding of how the mind works. The problem is that designing and implementing these experiments is technically complex and costly. This paper introduces versatile and economical means of developing multimodal-multisensory human experiments. We provide an experimental design framework that automatically integrates and synchronizes measures including electroencephalogram (EEG), galvanic skin response (GSR), eye-tracking, virtual reality (VR), body movement, mouse/cursor motion and response time. Unlike proprietary systems (e.g., iMotions), our system is free and open-source; it integrates PsychoPy, Unity and Lab Streaming Layer (LSL). The system embeds LSL inside PsychoPy/Unity for the synchronization of multiple sensory signals—gaze motion, electroencephalogram (EEG), galvanic skin response (GSR), mouse/cursor movement, and body motion—with low-cost consumer-grade devices in a simple behavioral task designed by PsychoPy and a virtual reality environment designed by Unity. This tutorial shows a step-by-step process by which a complex multimodal-multisensory experiment can be designed and implemented in a few hours. When conducting the experiment, all of the data synchronization and recoding of the data to disk will be done automatically.

scholarly journals Opensync: An Opensource Platform for Synchronizing Multiple Measures in Neuroscience Experiments

2021 ◽  
Author(s):  
Moein Razavi Ghods ◽  
Vahid Janfaza ◽  
Takashi Yamauchi ◽  
Anton Leontyev ◽  
Shanle Longmire-Monford ◽  
...  
Keyword(s):  
Human Behavior ◽  
Human Mind ◽  
Body Motion ◽  
User Input ◽  
Multiple Measures ◽  
Related Information ◽  
Behavioral Studies ◽  
Skin Response ◽  
Input Response ◽  
Electroencephalogram Eeg

Background: The human mind is multimodal. Yet most behavioral studies rely on century-old measures such as task accuracy and latency. To create a better understanding of human behavior and brain functionality, we should introduce other measures and analyze behavior from various aspects. However, it is technically complex and costly to design and implement the experiments that record multiple measures. To address this issue, a platform that allows synchronizing multiple measures from human behavior is needed. Method: This paper introduces an opensource platform named OpenSync, which can be used to synchronize multiple measures in neuroscience experiments. This platform helps to automatically integrate, synchronize and record physiological measures (e.g., electroencephalogram (EEG), galvanic skin response (GSR), eye-tracking, body motion, etc.), user input response (e.g., from mouse, keyboard, joystick, etc.), and task-related information (stimulus markers). In this paper, we explain the structure and details of OpenSync, provide two case studies in PsychoPy and Unity. Comparison with existing tools: Unlike proprietary systems (e.g., iMotions), OpenSync is free and it can be used inside any opensource experiment design software (e.g., PsychoPy, OpenSesame, Unity, etc., https://pypi.org/project/OpenSync/ and https://github.com/moeinrazavi/OpenSync_Unity). Results: Our experimental results show that the OpenSync platform is able to synchronize multiple measures with microsecond resolution.

scholarly journals Wearables and the Quantified Self: Systematic Benchmarking of Physiological Sensors

Sensors ◽  
2019 ◽  
Vol 19 (20) ◽  
pp. 4448 ◽  
Author(s):  
Günther Sagl ◽  
Bernd Resch ◽  
Andreas Petutschnig ◽  
Kalliopi Kyriakou ◽  
Michael Liedlgruber ◽  
...  
Keyword(s):  
Heart Rate ◽  
Galvanic Skin Response ◽  
Low Cost ◽  
Wearable Sensors ◽  
Sensor Data ◽  
Quantified Self ◽  
High Quality ◽  
Laboratory Equipment ◽  
Skin Response ◽  
Physiological Sensors

Wearable sensors are increasingly used in research, as well as for personal and private purposes. A variety of scientific studies are based on physiological measurements from such rather low-cost wearables. That said, how accurate are such measurements compared to measurements from well-calibrated, high-quality laboratory equipment used in psychological and medical research? The answer to this question, undoubtedly impacts the reliability of a study’s results. In this paper, we demonstrate an approach to quantify the accuracy of low-cost wearables in comparison to high-quality laboratory sensors. We therefore developed a benchmark framework for physiological sensors that covers the entire workflow from sensor data acquisition to the computation and interpretation of diverse correlation and similarity metrics. We evaluated this framework based on a study with 18 participants. Each participant was equipped with one high-quality laboratory sensor and two wearables. These three sensors simultaneously measured the physiological parameters such as heart rate and galvanic skin response, while the participant was cycling on an ergometer following a predefined routine. The results of our benchmarking show that cardiovascular parameters (heart rate, inter-beat interval, heart rate variability) yield very high correlations and similarities. Measurement of galvanic skin response, which is a more delicate undertaking, resulted in lower, but still reasonable correlations and similarities. We conclude that the benchmarked wearables provide physiological measurements such as heart rate and inter-beat interval with an accuracy close to that of the professional high-end sensor, but the accuracy varies more for other parameters, such as galvanic skin response.

scholarly journals Effects of Virtual Reality-Based Relaxation Techniques on Psychological, Physiological, and Biochemical Stress Indicators

Healthcare ◽  
2021 ◽  
Vol 9 (12) ◽  
pp. 1729
Author(s):  
Eglė Mazgelytė ◽  
Virginija Rekienė ◽  
Edita Dereškevičiūtė ◽  
Tomas Petrėnas ◽  
Jurgita Songailienė ◽  
...  
Keyword(s):  
Virtual Reality ◽  
Stress Reduction ◽  
Galvanic Skin Response ◽  
Random Sequence ◽  
Relaxation Techniques ◽  
Relaxation Methods ◽  
Stress Indicators ◽  
Management Intervention ◽  
Skin Response ◽  
Physiological And Biochemical

Various relaxation techniques could benefit from merging with virtual reality (VR) technologies, as these technologies are easily applicable, involving, and user-friendly. To date, it is unclear which relaxation technique using biofeedback combined with VR technology is the most effective. The study aimed to compare the effectiveness of brief VR-based biofeedback-assisted relaxation techniques including electroencephalographic biofeedback, mindfulness-based biofeedback, galvanic skin response biofeedback, and respiratory biofeedback. Forty-three healthy volunteers (age 34.7 ± 7.2 years), comprising 28 (65%) women and 15 (35%) men, were enrolled in the study. All the participants were exposed to four distinct relaxation sessions according to a computer-generated random sequence. The efficacy of relaxation methods was evaluated by examining psychological, physiological, and biochemical stress indicators. All VR-based relaxation techniques reduced salivary steroid hormone (i.e., cortisol, cortisone, and total glucocorticoid) levels and increased galvanic skin response values. Similarly, all interventions led to a significantly reduced subjectively perceived psychological strain level. Three out of the four interventions (i.e., electroencephalographic, respiratory, and galvanic skin response-based biofeedback relaxation sessions) resulted in a decreased self-reported fatigue level. We suggest that newly developed VR-based relaxations techniques are potential tools for stress reduction and might be particularly suitable for individuals who are not capable of adhering to a strict and time-consuming stress management intervention schedule.

Galvanic Skin Response As a Measure of Engagement During Play in Virtual Reality

2020 ◽  
Author(s):  
Roni Barak Ventura ◽  
Maurizio Porfiri
Keyword(s):  
Virtual Reality ◽  
Galvanic Skin Response ◽  
User Engagement ◽  
Experimental Setting ◽  
Information Theoretic ◽  
Emotional Processes ◽  
Skin Response ◽  
Series Of Experiments ◽  
Under Performing ◽  
Physiological Correlate

Abstract Competition is a common design strategy used to enhance user engagement and participation. However, it remains unclear how winning or losing might influence player’s engagement. In a recent study, we used behavioral metrics to quantify player engagement during competitive gameplay in virtual reality. To control for players’ status of winning or losing, we programmed a virtual opponent to either under-perform, over-perform, or tie with them. We conducted a series of experiments and found that players’ engagement was higher when they were losing, compared to when they played alone. Nevertheless, players’ engagement did not change during competition with an under-performing or equally-performing opponent. By applying the information-theoretic notion of transfer entropy, we unveiled a causal relationships between relative performance and engagement, whereby players monitored the scores and adapted their behavior accordingly. However, behavioral metrics are not detached from volition and may be influenced by confounding factors. This limitation is addressed by the use of physiological metrics, which offer largely unbiased measurements of cognitive and emotional processes. In the present study, we sought to strengthen our prior findings by measuring engagement with a physiological correlate. We conducted experiments in the same experimental setting and we measured players’ galvanic skin response during competition. We discovered that players’ skin conductance was higher when they were outperformed by their opponent, indicating that they were experiencing less flow. The results inform the design of technology-mediated applications toward sustaining user engagement and participation.

Using BCI and EEG to process and analyze driver’s brain activity signals during VR simulation

2021 ◽  
Vol 60 (4) ◽  
pp. 137-153
Author(s):  
Mirosław Nader ◽  
Ilona Jacyna-Gołda ◽  
Stanisław Nader ◽  
Karol Nehring
Keyword(s):  
Virtual Reality ◽  
Brain Activity ◽  
Age Groups ◽  
Functional Model ◽  
Flow Simulation ◽  
Human Mind ◽  
Pilot Studies ◽  
Computer Interfaces ◽  
External Stimuli ◽  
Electroencephalogram Eeg

The use of popular brain–computer interfaces (BCI) to analyze signals and the behavior of brain activity is a very current problem that is often undertaken in various aspects by many researchers. This comparison turns out to be particularly useful when studying the flows of information and signals in the human-machine-environment system, especially in the field of transportation sciences. This article presents the results of a pilot study of driver behavior with the use of a proprietary simulator based on Virtual Reality technology. The study uses the technology of studying signals emitted by the human mind and its specific zones in response to given environmental factors. A solution based on virtual reality with the limitation of external stimuli emitted by the real world was proposed, and computational analysis of the obtained data was performed. The research focused on traffic situations and how they affect the subject. The test was attended by representatives of various age groups, both with and without a driving license. This study presents an original functional model of a research stand in VR technology that we designed and built. Testing in VR conditions allows to limit the influence of undesirable external stimuli that may distort the results of readings. At the same time, it increases the range of road events that can be simulated without generating any risk for the participant. In the presented studies, the BCI was used to assess the driver's behavior, which allows for the activity of selected brain waves of the examined person to be registered. Electroencephalogram (EEG) was used to study the activity of brain and its response to stimuli coming from the Virtual Reality created environment. Electrical activity detection is possible thanks to the use of electrodes placed on the skin in selected areas of the skull. The structure of the proprietary test-stand for signal and information flow simulation tests, which allows for the selection of measured signals and the method of parameter recording, is presented. An important part of this study is the presentation of the results of pilot studies obtained in the course of real research on the behavior of a car driver.

scholarly journals Cybersickness Testing Of Gender And Experience Factors Using Virtual Reality

2021 ◽  
Vol 2 (2) ◽  
pp. 63-69
Author(s):  
Ancella Hendrika ◽  
Clara Theresia ◽  
Thedy Yogasara
Keyword(s):  
Virtual Reality ◽  
Galvanic Skin Response ◽  
Measuring Instruments ◽  
Simulator Sickness ◽  
Sports Industry ◽  
Measurement Results ◽  
Skin Response ◽  
Simulator Sickness Questionnaire

One of the technologies that people are starting to get interested in is virtual reality (VR). VR is widely used as a means of entertainment, even more so at this time, the e-sports industry is developing rapidly. However, the use of VR can cause cybersickness, a disease arising from sensory and perceptual mismatches between the visual and vestibular systems. The emergence of cybersickness can be related to gender and experience using VR. There have been studies on cybersickness, but the results obtained had not come to the same conclusion. This research aims to identify the effect of gender and experience using VR, predict the timing of cybersickness by using physiological measurements, and provide recommendations that can minimize cybersickness in activities using VR. The measuring instruments used are the galvanic skin response (GSR) and a simulator sickness questionnaire (SSQ). In this study, the influence test is conducted by using ANOVA and Kruskal-Wallis to determine whether gender and experience using VR affect the potential for cybersickness. Based on the GSR measurement results, it found that gender and experiences of using VR do not affect a person's potential for cybersickness. From the result of SSQ measurement, gender does not affect the cybersickness, but the experience of using VR affected a person's potential for cybersickness. Qualitatively, cybersickness symptoms appear in the 15-20 minutes after the VR game has set in. Therefore, it is recommended to limit the usage of VR to less than 15 to 20 minutes per session. Keywords: cybersickness, galvanic skin response (GSR), simulator sickness questionnaire (SSQ), virtual reality

scholarly journals Pulse-Galvanic Skin Response Analysis with Multiple Sensor Device Design

2021 ◽  
pp. 109-112
Author(s):  
Mehmet Ali Dincer ◽  
Kubra Evren Sahin ◽  
Savas Sahin
Keyword(s):  
Embedded System ◽  
Galvanic Skin Response ◽  
Low Cost ◽  
Response Analysis ◽  
Recording Device ◽  
Multiple Sensors ◽  
Non Invasive ◽  
Skin Response ◽  
Multiple Sensor ◽  
Descriptive Statistical Analysis

In this study, the development of a low-cost electronic card-based medical device measuring and recording patient data was described via non-invasive methods. Both the descriptive statistical analysis and the regression model was performed from the pulse and galvanic skin response (GSR) from the volunteer' data. It is important to measure and record different data simultaneously with multiple sensors from the patient during the treatment, medical operation and care periods of the patients. The data measured from the designed device was evaluated for the patient's position, GSR, the respiration rate, the blood oxygen content, and the heart rate. The designed measurement and recording device were implemented with an embedded system-based microcontroller card. The designed device might provide for monitoring and recording data with led display, serial port, microSD card or internet of things.

Sign in / Sign up

Export Citation Format

Share Document