scholarly journals Robot training in virtual environments using Reinforcement Learning techniques

2020 ◽  
Author(s):  
Natália Souza Soares ◽  
João Marcelo Xavier Natário Teixeira ◽  
Veronica Teichrieb
Keyword(s):  
Reinforcement Learning ◽  
Virtual Environments ◽  
Virtual Environment ◽  
Real World ◽  
Real Data ◽  
Indoor Navigation ◽  
Virtual Training ◽  
Learning Techniques ◽  
Policy Optimization ◽  
Environment Parameters

In this work, we propose a framework to train a robot in a virtual environment using Reinforcement Learning (RL) techniques and thus facilitating the use of this type of approach in robotics. With our integrated solution for virtual training, it is possible to programmatically change the environment parameters, making it easy to implement domain randomization techniques on-the-fly. We conducted experiments with a TurtleBot 2i in an indoor navigation task with static obstacle avoidance using an RL algorithm called Proximal Policy Optimization (PPO). Our results show that even though the training did not use any real data, the trained model was able to generalize to different virtual environments and real-world scenes.

The Transfer of Spatial Knowledge in Virtual Environment Training

1998 ◽  
Vol 7 (2) ◽  
pp. 129-143 ◽  
Author(s):  
David Waller ◽  
Earl Hunt ◽  
David Knapp
Keyword(s):  
Gender Differences ◽  
Virtual Environments ◽  
Virtual Environment ◽  
Real World ◽  
Spatial Knowledge ◽  
Training Effectiveness ◽  
Spatial Representations ◽  
Training Environment ◽  
Virtual Training ◽  
Real World Situation

Many training applications of virtual environments (VEs) require people to be able to transfer spatial knowledge acquired in a VE to a real-world situation. Using the concept of fidelity, we examine the variables that mediate the transfer of spatial knowledge and discuss the form and development of spatial representations in VE training. We report the results of an experiment in which groups were trained in six different environments (no training, real world, map, VE desktop, VE immersive, and VE long immersive) and then were asked to apply route and configurational knowledge in a real-world maze environment. Short periods of VE training were no more effective than map training; however with sufficient exposure to the virtual training environment, VE training eventually surpassed real-world training. Robust gender differences in training effectiveness of VEs were also found.

Detection of Unexpected Motion While Driving:From Psychophysics to Real World via Virtual Environments

1996 ◽  
Vol 5 (2) ◽  
pp. 163-172 ◽  
Author(s):  
Andrew Liu ◽  
Alex P. Pentland
Keyword(s):  
Virtual Environments ◽  
Virtual Environment ◽  
Real World ◽  
Driving Simulator ◽  
Optokinetic Nystagmus ◽  
Visual Fields ◽  
Lower Field ◽  
Eye Fixations

This paper describes a set of experiments investigating the interaction between the location of eye fixations and the detection of unexpected motion while driving. Both psychophysical and real-world observations indicate that there are differences between the upper and lower visual fields with respect to driving. We began with psychophysical experiments to test whether the detection of unexpected motion Is inherently different in the upper and lower visual fields. No difference was found. However, when texture was added to the driving surface, a large difference was found, possibly due to optokinetic nystagmus stimulated by the texture. These results were confirmed in a driving simulator, and their implications for head-up displays (HUDs) explored. We found that the same upper/lower field asymmetry could be found with digital HUDs but not with analog HUDs. These experiments illustrate how virtual environment technology can connect knowledge from psychophysical experimentation to more realistic situations.

scholarly journals Vision-Based Multirotor Following Using Synthetic Learning Techniques

Sensors ◽  
2019 ◽  
Vol 19 (21) ◽  
pp. 4794
Author(s):  
Alejandro Rodriguez-Ramos ◽  
Adrian Alvarez-Fernandez ◽  
Hriday Bavle ◽  
Pascual Campoy ◽  
Jonathan P. How
Keyword(s):  
Reinforcement Learning ◽  
Object Detection ◽  
Learning Strategies ◽  
Motion Control ◽  
Domain Adaptation ◽  
Control Strategies ◽  
Synthetic Data ◽  
Real Data ◽  
Stable Convergence ◽  
Learning Techniques

Deep- and reinforcement-learning techniques have increasingly required large sets of real data to achieve stable convergence and generalization, in the context of image-recognition, object-detection or motion-control strategies. On this subject, the research community lacks robust approaches to overcome unavailable real-world extensive data by means of realistic synthetic-information and domain-adaptation techniques. In this work, synthetic-learning strategies have been used for the vision-based autonomous following of a noncooperative multirotor. The complete maneuver was learned with synthetic images and high-dimensional low-level continuous robot states, with deep- and reinforcement-learning techniques for object detection and motion control, respectively. A novel motion-control strategy for object following is introduced where the camera gimbal movement is coupled with the multirotor motion during the multirotor following. Results confirm that our present framework can be used to deploy a vision-based task in real flight using synthetic data. It was extensively validated in both simulated and real-flight scenarios, providing proper results (following a multirotor up to 1.3 m/s in simulation and 0.3 m/s in real flights).

Providing Situation Awareness Assistance to Users of Large-Scale, Dynamic, Complex Virtual Environments

1993 ◽  
Vol 2 (4) ◽  
pp. 297-313 ◽  
Author(s):  
Martin R. Stytz ◽  
Elizabeth Block ◽  
Brian Soltz
Keyword(s):  
Virtual Environments ◽  
Virtual Environment ◽  
Situation Awareness ◽  
Real World ◽  
Large Scale ◽  
Virtual Space ◽  
Prototype System ◽  
First Person ◽  
Focus Attention ◽  
Unobtrusive Observation

As virtual environments grow in complexity, size, and scope users will be increasingly challenged in assessing the situation in them. This will occur because of the difficulty in determining where to focus attention and in assimilating and assessing the information as it floods in. One technique for providing this type of assistance is to provide the user with a first-person, immersive, synthetic environment observation post, an observatory, that permits unobtrusive observation of the environment without interfering with the activity in the environment. However, for large, complex synthetic environments this type of support is not sufficient because the mere portrayal of raw, unanalyzed data about the objects in the virtual space can overwhelm the user with information. To address this problem, which exists in both real and virtual environments, we are investigating the forms of situation awareness assistance needed by users of large-scale virtual environments and the ways in which a virtual environment can be used to improve situation awareness of real-world environments. A technique that we have developed is to allow a user to place analysis modules throughout the virtual environment. Each module provides summary information concerning the importance of the activity in its portion of the virtual environment to the user. Our prototype system, called the Sentinel, is embedded within a virtual environment observatory and provides situation awareness assistance for users within a large virtual environment.

Influence of Physical Characteristics of Routes on Distance Cognition in Virtual Environments

2005 ◽  
Vol 32 (5) ◽  
pp. 777-785 ◽  
Author(s):  
Ebru Cubukcu ◽  
Jack L Nasar
Keyword(s):  
Virtual Reality ◽  
Virtual Environments ◽  
Virtual Environment ◽  
Real World ◽  
Three Dimensional ◽  
Spatial Behavior ◽  
Self Report ◽  
Parking Garage ◽  
The Real ◽  
Predicting Behavior

Discrepanices between perceived and actual distance may affect people's spatial behavior. In a previous study Nasar, using self report of behavior, found that segmentation (measured through the number of buildings) along the route affected choice of parking garage and path from the parking garage to a destination. We recreated that same environment in a three-dimensional virtual environment and conducted a test to see whether the same factors emerged under these more controlled conditions and to see whether spatial behavior in the virtual environment accurately reflected behavior in the real environment. The results confirmed similar patterns of response in the virtual and real environments. This supports the use of virtual reality as a tool for predicting behavior in the real world and confirms increases in segmentation as related to increases in perceived distance.

scholarly journals Transfer of Calibration in Virtual Reality to both Real and Virtual Environments

2019 ◽  
Vol 63 (1) ◽  
pp. 1943-1947 ◽  
Author(s):  
Hannah M. Solini ◽  
Ayush Bhargava ◽  
Christopher C. Pagano
Keyword(s):  
Virtual Reality ◽  
Virtual Environments ◽  
Virtual Environment ◽  
Real World ◽  
Optic Flow ◽  
Calibration Transfer ◽  
The Real ◽  
Before And After ◽  
Virtual Display ◽  
World Environment

It is often questioned whether task performance attained in a virtual environment can be transferred appropriately and accurately to the same task in the real world. With advancements in virtual reality (VR) technology, recent research has focused on individuals’ abilities to transfer calibration achieved in a virtual environment to a real-world environment. Little research, however, has shown whether transfer of calibration from a virtual environment to the real world is similar to transfer of calibration from a virtual environment to another virtual environment. As such, the present study investigated differences in calibration transfer to real-world and virtual environments. In either a real-world or virtual environment, participants completed blind walking estimates before and after experiencing perturbed virtual optic flow via a head-mounted virtual display (HMD). Results showed that individuals calibrated to perturbed virtual optic flow and that this calibration carried over to both real-world and virtual environments in a like manner.

Learned Behavior

2016 ◽  
pp. 94-123 ◽  
Author(s):  
Jacquelyne Forgette ◽  
Michael Katchabaw
Keyword(s):  
Reinforcement Learning ◽  
Virtual Environments ◽  
Virtual Environment ◽  
Virtual World ◽  
Learning System ◽  
Experimental Results ◽  
Virtual Characters ◽  
Learned Behavior ◽  
The World ◽  
Over Time

A key challenge in programming virtual environments is to produce virtual characters that are autonomous and capable of action selections that appear believable. In this chapter, motivations are used as a basis for learning using reinforcements. With motives driving the decisions of characters, their actions will appear less structured and repetitious, and more human in nature. This will also allow developers to easily create virtual characters with specific motivations, based mostly on their narrative purposes or roles in the virtual world. With minimum and maximum desirable motive values, the characters use reinforcement learning to drive action selection to maximize their rewards across all motives. Experimental results show that a character can learn to satisfy as many as four motives, even with significantly delayed rewards, and motive changes that are caused by other characters in the world. While the actions tested are simple in nature, they show the potential of a more complicated motivation driven reinforcement learning system. The developer need only define a character's motivations, and the character will learn to act realistically over time in the virtual environment.

scholarly journals Navigation, Wayfinding, and Place Experience within a Virtual City

2000 ◽  
Vol 9 (5) ◽  
pp. 435-447 ◽  
Author(s):  
Craig D. Murray ◽  
John M. Bowers ◽  
Adrian J. West ◽  
Steve Pettifer ◽  
Simon Gibson
Keyword(s):  
Qualitative Study ◽  
Virtual Environments ◽  
Virtual Environment ◽  
Virtual Worlds ◽  
Real World ◽  
Virtual World ◽  
Place Experience ◽  
Virtual City

We report a qualitative study of navigation, wayfinding, and place experience within a virtual city. “Cityscape” is a virtual environment (VE), partially algorithmically generated and intended to be redolent of the aggregate forms of real cities. In the present study, we observed and interviewed participants during and following exploration of a desktop implementation of Cityscape. A number of emergent themes were identified and are presented and discussed. Observing the interaction with the virtual city suggested a continuous relationship between real and virtual worlds. Participants were seen to attribute real-world properties and expectations to the contents of the virtual world. The implications of these themes for the construction of virtual environments modeled on real-world forms are considered.

Virtual Environments Psychotherapy: A Case Study of Fear of Flying Disorder

1997 ◽  
Vol 6 (1) ◽  
pp. 127-132 ◽  
Author(s):  
Max M. North ◽  
Sarah M. North ◽  
Joseph R. Coble
Keyword(s):  
Virtual Environments ◽  
Virtual Environment ◽  
Real World ◽  
Psychological Disorders ◽  
Anxiety Symptoms ◽  
Fear Of Flying ◽  
The Real ◽  
Display Technology ◽  
The Subject

Current computer and display technology allows the creation of virtual environment scenes that can be utilized for treating a variety of psychological disorders. This case study demonstrates the effectiveness of virtual environment desensitization (VED) in the treatment of a subject who suffered from fear of flying, a disorder that affects a large number of people. The subject, accompanied by a virtual therapist, was placed in the cockpit of a virtual helicopter and flown over a simulated city for five sessions. The VED treatment resulted in both a significant reduction of anxiety symptoms and the ability to face the phobic situations in the real world.

Sign in / Sign up

Export Citation Format

Share Document