Immersive telepresence on the operation of unmanned vehicles

Unmanned ground vehicles are usually deployed in situations, where it is too dangerous or not feasible to have an operator onboard. One challenge faced when such vehicles are teleoperated is maintaining a high situational awareness, due to aspects such as limitation of cameras, characteristics of network transmission, and the lack of other sensory information, such as sounds and vibrations. Situation awareness refers to the understanding of the information, events, and actions that will impact the execution and the objectives of the tasks at the current and near future of the operation of the vehicle. This work investigates how the simultaneous use of immersive telepresence and mixed reality could impact the situation awareness of the operator and the navigation performance. A user study was performed to compare our proposed approach with a traditional unmanned vehicle control station. Quantitative data obtained from the vehicle’s behavior and the situation awareness global assessment technique were used to analyze such impacts. Results provide evidence that our approach is relevant when the task requires a detailed observation of the surroundings, leading to higher situation awareness and navigation performance.

Download Full-text

Catch My Drift: Elevating Situation Awareness for Highly Automated Driving with an Explanatory Windshield Display User Interface

Multimodal Technologies and Interaction ◽

10.3390/mti2040071 ◽

2018 ◽

Vol 2 (4) ◽

pp. 71 ◽

Cited By ~ 1

Author(s):

Patrick Lindemann ◽

Tae-Young Lee ◽

Gerhard Rigoll

Keyword(s):

User Interface ◽

Situation Awareness ◽

Urban Areas ◽

User Study ◽

Mixed Reality ◽

Driving Simulation ◽

Medium Effect ◽

Automated Driving ◽

Good Medium ◽

Highly Automated Driving

Broad access to automated cars (ACs) that can reliably and unconditionally drive in all environments is still some years away. Urban areas pose a particular challenge to ACs, since even perfectly reliable systems may be forced to execute sudden reactive driving maneuvers in hard-to-predict hazardous situations. This may negatively surprise the driver, possibly causing discomfort, anxiety or loss of trust, which might be a risk for the acceptance of the technology in general. To counter this, we suggest an explanatory windshield display interface with augmented reality (AR) elements to support driver situation awareness (SA). It provides the driver with information about the car’s perceptive capabilities and driving decisions. We created a prototype in a human-centered approach and implemented the interface in a mixed-reality driving simulation. We conducted a user study to assess its influence on driver SA. We collected objective SA scores and self-ratings, both of which yielded a significant improvement with our interface in good (medium effect) and in bad (large effect) visibility conditions. We conclude that explanatory AR interfaces could be a viable measure against unwarranted driver discomfort and loss of trust in critical urban situations by elevating SA.

Download Full-text

Is More Information Better? How Dismounted Soldiers Use Video Feed From Unmanned Vehicles

Journal of Cognitive Engineering and Decision Making ◽

10.1177/1555343412445054 ◽

2012 ◽

Vol 7 (1) ◽

pp. 26-48 ◽

Cited By ~ 6

Author(s):

Ronny Ophir-Arbelle ◽

Tal Oron-Gilad ◽

Avinoam Borowsky ◽

Yisrael Parmet

Keyword(s):

Situation Awareness ◽

Urban Areas ◽

Urban Environments ◽

Unmanned Vehicles ◽

Sources Of Information ◽

Ground Vehicles ◽

Multiple Sources ◽

Mental Demand ◽

Video Feed ◽

Additional Support

Operational tactics in urban areas are often aided by information from unmanned aerial vehicles (UAVs). A major challenge for dismounted soldiers, particularly in urban environments, is to understand the conflict area in general and particularly from the UAV feed. The UAV feed is usually used to enhance soldiers’ situation awareness abilities but less for identifying specific elements. A possible way to further enhance soldiers’ abilities is to provide them with multiple sources of information (e.g., aerial and ground views). This study examined the benefits of presenting video feed from UAVs and unmanned ground vehicles (UGVs) in a combined interface, relative to presenting aerial feed alone. Thirty former infantry soldiers with no experience in operating unmanned vehicles participated. Objective performance, subjective evaluations, and eye-tracking patterns were examined in two scenarios. In Scenario 1, performance scores in both identification and orientation tasks were superior in the combined configuration. In Scenario 2, performance scores in the identification tasks were improved, and the addition of the UGV feed did not harm performance in the orientation task. Eye movement scanning patterns reinforced that both UAV and UGV feeds were used for the mission. The combined configuration generated consistent benefits with regard to the identification tasks, perceived mental demand, and reduction of false reports without having any apparent cost on participants. Ground views may provide additional support to dismounted soldiers.

Download Full-text

Effects of User Interfaces on Take-Over Performance: A Review of the Empirical Evidence

Information ◽

10.3390/info12040162 ◽

2021 ◽

Vol 12 (4) ◽

pp. 162

Author(s):

Soyeon Kim ◽

René van Egmond ◽

Riender Happee

Keyword(s):

User Interface ◽

Situation Awareness ◽

User Interfaces ◽

Situational Awareness ◽

Selection Criterion ◽

Auditory Modality ◽

Automated Driving ◽

Automated Vehicle ◽

Positive Effects ◽

High Urgency

In automated driving, the user interface plays an essential role in guiding transitions between automated and manual driving. This literature review identified 25 studies that explicitly studied the effectiveness of user interfaces in automated driving. Our main selection criterion was how the user interface (UI) affected take-over performance in higher automation levels allowing drivers to take their eyes off the road (SAE3 and SAE4). We categorized user interface (UI) factors from an automated vehicle-related information perspective. Short take-over times are consistently associated with take-over requests (TORs) initiated by the auditory modality with high urgency levels. On the other hand, take-over requests directly displayed on non-driving-related task devices and augmented reality do not affect take-over time. Additional explanations of take-over situation, surrounding and vehicle information while driving, and take-over guiding information were found to improve situational awareness. Hence, we conclude that advanced user interfaces can enhance the safety and acceptance of automated driving. Most studies showed positive effects of advanced UI, but a number of studies showed no significant benefits, and a few studies showed negative effects of advanced UI, which may be associated with information overload. The occurrence of positive and negative results of similar UI concepts in different studies highlights the need for systematic UI testing across driving conditions and driver characteristics. Our findings propose future UI studies of automated vehicle focusing on trust calibration and enhancing situation awareness in various scenarios.

Download Full-text

Assessing a Decision Support Tool for SOC Analysts

Digital Threats: Research and Practice ◽

10.1145/3430753 ◽

2021 ◽

Author(s):

Jassim Happa ◽

Ioannis Agrafiotis ◽

Martin Helmhout ◽

Thomas Bashford-Rogers ◽

Michael Goldsmith ◽

...

Keyword(s):

Decision Support ◽

Mixed Method ◽

User Study ◽

Situational Awareness ◽

Decision Support Tool ◽

Lessons Learned ◽

Visualization Tool ◽

Support Tool ◽

Response Strategies ◽

Method Approach

In recent years, many tools have been developed to understand attacks that make use of visualization, but few examples aims to predict real-world consequences. We have developed a visualization tool that aims to improve decision support during attacks. Our tool visualizes propagation of risks from IDS and AV-alert data by relating sensor alerts to Business Process (BP) tasks and machine assets: an important capability gap present in many Security Operation Centres (SOCs) today. In this paper we present a user study in which we evaluate the tool's usability and ability to deliver situational awareness to the analyst. Ten analysts from seven SOCs performed carefully designed tasks related to understanding risks and prioritising recovery decisions. The study was conducted in laboratory conditions, with simulated attacks, and used a mixed-method approach to collect data from questionnaires, eyetracking and voice-recorded interviews. The findings suggest that providing analysts with situational awareness relating to business priorities can help them prioritise response strategies. Finally, we provide an in-depth discussion on the wider questions related to user studies in similar conditions as well as lessons learned from our user study and developing a visualization tool of this type.

Download Full-text

Multimodal Mixed Reality Impact on a Hand Guiding Task with a Holographic Cobot

Multimodal Technologies and Interaction ◽

10.3390/mti4040078 ◽

2020 ◽

Vol 4 (4) ◽

pp. 78

Author(s):

Andoni Rivera Pinto ◽

Johan Kildal ◽

Elena Lazkano

Keyword(s):

Augmented Reality ◽

Industrial Production ◽

User Study ◽

Haptic Feedback ◽

Mixed Reality ◽

Robot Arm ◽

Pick And Place ◽

Place Task ◽

Guidance Technique ◽

The Impact

In the context of industrial production, a worker that wants to program a robot using the hand-guidance technique needs that the robot is available to be programmed and not in operation. This means that production with that robot is stopped during that time. A way around this constraint is to perform the same manual guidance steps on a holographic representation of the digital twin of the robot, using augmented reality technologies. However, this presents the limitation of a lack of tangibility of the visual holograms that the user tries to grab. We present an interface in which some of the tangibility is provided through ultrasound-based mid-air haptics actuation. We report a user study that evaluates the impact that the presence of such haptic feedback may have on a pick-and-place task of the wrist of a holographic robot arm which we found to be beneficial.

Download Full-text

Design on Hierarchical Testing System for Unmanned Ground Vehicles

Advanced Materials Research ◽

10.4028/www.scientific.net/amr.346.817 ◽

2011 ◽

Vol 346 ◽

pp. 817-822 ◽

Cited By ~ 1

Author(s):

Xin Li ◽

Guang Ming Xiong ◽

Yang Sun ◽

Shao Bin Wu ◽

Jian Wei Gong ◽

...

Keyword(s):

Evaluation System ◽

Test System ◽

Unmanned Vehicles ◽

Intelligent Vehicles ◽

Test Method ◽

Unmanned Ground Vehicles ◽

Ground Vehicles ◽

Test Environment ◽

Test And Evaluation ◽

Comprehensive Test

The test system for technical abilities of unmanned vehicles is gradually developed from the single test to comprehensive test. The pre-established test and evaluation system can promote the development of unmanned ground vehicles. The 2009 Future Challenge: Intelligent Vehicles and Beyond (FC’09) pushed China's unmanned vehicles out of laboratories. This paper proposed to design a more scientific and comprehensive test system for future competitions to better guide and regulate the development of China's unmanned vehicles. According to the design idea of stage by stage and level by level, the hierarchical test content from simple to advanced, from local to overall is designed. Then the hierarchic test environment is established according to the levels of test content. The test method based on multi-platform and multi-sensor is put forward to ensure the accuracy of test results. The testing criterion framework is set up to regulate future unmanned vehicle contests and to assess the unmanned vehicles scientifically and accurately.

Download Full-text

Modelling Situation Awareness Information and System Requirements for the Mission using Goal-Oriented Task Analysis Approach

Software Design and Development ◽

10.4018/978-1-4666-4301-7.ch023 ◽

2014 ◽

pp. 460-478

Author(s):

Cyril Onwubiko

Keyword(s):

Situation Awareness ◽

Task Analysis ◽

Situational Awareness ◽

Computer Network ◽

Valuable Insight ◽

Test Scenario ◽

Agent Oriented Software Engineering ◽

Awareness Information ◽

System Requirements ◽

Security Test

This chapter describes work on modelling situational awareness information and system requirements for the mission. Developing this model based on Goal-Oriented Task Analysis representation of the mission using an Agent Oriented Software Engineering methodology advances current information requirement models because it provides valuable insight on how to effectively achieve the mission’s requirements (information, systems, networks, and IT infrastructure), and offers enhanced situational awareness within the Computer Network Defence environment. Further, the modelling approach using Secure Tropos is described, and model validation using a security test scenario is discussed.

Download Full-text

Citizen-to-Citizen Resource Sharing in Emergency Response

Crowdsourcing ◽

10.4018/978-1-5225-8362-2.ch029 ◽

2019 ◽

pp. 578-605

Author(s):

Soon Ae Chun ◽

Jaideep S. Vaidya ◽

Vijayalakshmi Atluri ◽

Basit Shafiq ◽

Nabil R. Adam

Keyword(s):

Citizen Participation ◽

Emergency Response ◽

Situation Awareness ◽

Resource Sharing ◽

Large Scale ◽

Situational Awareness ◽

Government Agencies ◽

Mobile Crowdsourcing ◽

Disaster Situation ◽

The Government

During large-scale manmade or natural disasters, such as Superstorm Sandy and Hurricanes Harvey and Irma, collaborations among government agencies, NGOs, and businesses need to be coordinated to provide necessary resources to respond to emergency events. However, resources from citizens themselves are underutilized, such as their equipment or expertise. The citizen participation via social media enhanced the situational awareness, but the response management is still mainly handled by the government or government-sanctioned partners. By harnessing the power of citizen crowdsourcing, government agencies can create enhanced disaster situation awareness and facilitate effective utilization of resources provided by citizen volunteers, resulting in more effective disaster responses. This chapter presents a public engagement in emergency response (PEER) framework that provides an online and mobile crowdsourcing platform for incident reporting and citizens' resource volunteering as well as an intelligent recommender system to match-make citizen resources with emergency tasks.

Download Full-text