scholarly journals Virtual-Taobao: Virtualizing Real-World Online Retail Environment for Reinforcement Learning

2019 ◽  
Vol 33 ◽  
pp. 4902-4909 ◽  
Author(s):  
Jing-Cheng Shi ◽  
Yang Yu ◽  
Qing Da ◽  
Shi-Yong Chen ◽  
An-Xiang Zeng
Keyword(s):  
Reinforcement Learning ◽  
Real World ◽  
Physical Environment ◽  
Physical World ◽  
Online Retail ◽  
The Real ◽  
Policy Model ◽  
Physical Environments ◽  
Sampling Cost ◽  
Norm Constraint

Applying reinforcement learning in physical-world tasks is extremely challenging. It is commonly infeasible to sample a large number of trials, as required by current reinforcement learning methods, in a physical environment. This paper reports our project on using reinforcement learning for better commodity search in Taobao, one of the largest online retail platforms and meanwhile a physical environment with a high sampling cost. Instead of training reinforcement learning in Taobao directly, we present our environment-building approach: we build Virtual-Taobao, a simulator learned from historical customer behavior data, and then we train policies in Virtual-Taobao with no physical sampling costs. To improve the simulation precision, we propose GAN-SD (GAN for Simulating Distributions) for customer feature generation with better matched distribution; we propose MAIL (Multiagent Adversarial Imitation Learning) for generating better generalizable customer actions. To further avoid overfitting the imperfection of the simulator, we propose ANC (Action Norm Constraint) strategy to regularize the policy model. In experiments, Virtual-Taobao is trained from hundreds of millions of real Taobao customers’ records. Compared with the real Taobao, Virtual-Taobao faithfully recovers important properties of the real environment. We further show that the policies trained purely in Virtual-Taobao, which has zero physical sampling cost, can have significantly superior real-world performance to the traditional supervised approaches, through online A/B tests. We hope this work may shed some light on applying reinforcement learning in complex physical environments.

EXPLORING HEALTHCARE ARCHITECTURE THROUGH THE MEDIUM OF FILM: Motives and Techniques

2014 ◽  
Vol 8 (1) ◽  
pp. 29
Author(s):  
Stephen Verderber
Keyword(s):  
Real World ◽  
The Real ◽  
Healthcare Settings ◽  
Interdisciplinary Field ◽  
Body Of Knowledge ◽  
Behavior Setting ◽  
Physical Environments ◽  
Scant Attention ◽  
History Of

The interdisciplinary field of person-environment relations has, from its origins, addressed the transactional relationship between human behavior and the built environment. This body of knowledge has been based upon qualitative and quantitative assessment of phenomena in the “real world.” This knowledge base has been instrumental in advancing the quality of real, physical environments globally at various scales of inquiry and with myriad user/client constituencies. By contrast, scant attention has been devoted to using simulation as a means to examine and represent person-environment transactions and how what is learned can be applied. The present discussion posits that press-competency theory, with related aspects drawn from functionalist-evolutionary theory, can together function to help us learn of how the medium of film can yield further insights to person-environment (P-E) transactions in the real world. Sampling, combined with extemporary behavior setting analysis, provide the basis for this analysis of healthcare settings as expressed throughout the history of cinema. This method can be of significant aid in examining P-E transactions across diverse historical periods, building types and places, healthcare and otherwise, otherwise logistically, geographically, or temporally unattainable in real time and space.

scholarly journals How to train your robot with deep reinforcement learning: lessons we have learned

2021 ◽  
pp. 027836492098785
Author(s):  
Julian Ibarz ◽  
Jie Tan ◽  
Chelsea Finn ◽  
Mrinal Kalakrishnan ◽  
Peter Pastor ◽  
...  
Keyword(s):  
Machine Learning ◽  
Reinforcement Learning ◽  
Case Studies ◽  
Real World ◽  
Review Article ◽  
The Real ◽  
Complex Skills ◽  
Real World Learning ◽  
Level Sensor ◽  
Embodied Agent

Deep reinforcement learning (RL) has emerged as a promising approach for autonomously acquiring complex behaviors from low-level sensor observations. Although a large portion of deep RL research has focused on applications in video games and simulated control, which does not connect with the constraints of learning in real environments, deep RL has also demonstrated promise in enabling physical robots to learn complex skills in the real world. At the same time, real-world robotics provides an appealing domain for evaluating such algorithms, as it connects directly to how humans learn: as an embodied agent in the real world. Learning to perceive and move in the real world presents numerous challenges, some of which are easier to address than others, and some of which are often not considered in RL research that focuses only on simulated domains. In this review article, we present a number of case studies involving robotic deep RL. Building off of these case studies, we discuss commonly perceived challenges in deep RL and how they have been addressed in these works. We also provide an overview of other outstanding challenges, many of which are unique to the real-world robotics setting and are not often the focus of mainstream RL research. Our goal is to provide a resource both for roboticists and machine learning researchers who are interested in furthering the progress of deep RL in the real world.

scholarly journals Real-world size of objects serves as an axis of object space

2021 ◽  
Author(s):  
Taicheng Huang ◽  
Yiying Song ◽  
Jia Liu
Keyword(s):  
Real World ◽  
Temporal Cortex ◽  
Principal Component ◽  
Physical World ◽  
Task Demands ◽  
Deep Convolutional Neural Networks ◽  
Object Space ◽  
The Real ◽  
Ventral Temporal Cortex ◽  
Dimensional Object

Our mind can represent various objects from the physical world metaphorically into an abstract and complex high-dimensional object space, with a finite number of orthogonal axes encoding critical object features. Previous fMRI studies have shown that the middle fusiform sulcus in the ventral temporal cortex separates the real-world small-size map from the large-size map. Here we asked whether the feature of objects' real-world size constructed an axis of object space with deep convolutional neural networks (DCNNs) based on three criteria of sensitivity, independence and necessity that are impractical to be examined altogether with traditional approaches. A principal component analysis on features extracted by the DCNNs showed that objects' real-world size was encoded by an independent component, and the removal of this component significantly impaired DCNN's performance in recognizing objects. By manipulating stimuli, we found that the shape and texture of objects, rather than retina size, co-occurrence and task demands, accounted for the representation of the real-world size in the DCNNs. A follow-up fMRI experiment on humans further demonstrated that the shape, but not the texture, was used to infer the real-world size of objects in humans. In short, with both computational modeling and empirical human experiments, our study provided the first evidence supporting the feature of objects' real-world size as an axis of object space, and devised a novel paradigm for future exploring the structure of object space.

scholarly journals Making Science Fiction Real: Neoliberalism, Real-Life and Esports in Eve Online

2018 ◽  
Vol 10 (3) ◽  
Author(s):  
Mark Richard Johnson ◽  
Robert Mejia
Keyword(s):  
Science Fiction ◽  
Virtual Worlds ◽  
Real World ◽  
Free Market ◽  
Real Life ◽  
Physical World ◽  
The Real ◽  
Contemporary World ◽  
Neoliberal Capitalism ◽  
Political Economic

In this paper, we argue that EVE Online is a fruitful site for exploring how the representational and political-economic elements of science fiction intersect to exert a sociocultural and political-economic force on the shape and nature of the future-present. EVE has been oft heralded for its economic and sociocultural complexity, and for employing a free market ethos and ethics in its game world. However, we by contrast seek not to consider how EVE reflects our contemporary world, but rather how our contemporary neoliberal milieu reflects EVE. We explore how EVE works to make its world of neoliberal markets and borderline anarcho-capitalism manifest through the political economic and sociocultural assemblages mobilized beyond the game. We explore the deep intertwining of  behaviors of players both within and outside of the game, demonstrating that EVE promotes neoliberal  activity in its players, encourages these behaviors outside the game, and that players who have found success in the real world of neoliberal capitalism are those best-positioned for success in the time-demanding and resource-demanding world of EVE. This thereby sets up a reciprocal ideological determination between the real and virtual worlds of EVE players, whereby each reinforces the other. We lastly consider the “Alliance Tournament” event, which romanticizes conflict and competition, and argue that it serves as a crucial site for deploying a further set of similar rhetorical resources. The paper therefore offers an understanding of the sociocultural and political-economic pressure exerted on the “physical” world by the intersection of EVE’s representational and material elements, and what these show us about the real-world ideological power of science fictional worlds.

scholarly journals Augmented Reality Research and Applications in Education

2021 ◽  
Author(s):  
Ezgi Pelin Yildiz
Keyword(s):  
Virtual Reality ◽  
Augmented Reality ◽  
Real Time ◽  
Virtual Environment ◽  
Real World ◽  
The Real ◽  
Virtual Objects ◽  
Physical Environments ◽  
Real Objects ◽  
As If

Augmented reality is defined as the technology in which virtual objects are blended with the real world and also interact with each other. Although augmented reality applications are used in many areas, the most important of these areas is the field of education. AR technology allows the combination of real objects and virtual information in order to increase students’ interaction with physical environments and facilitate their learning. Developing technology enables students to learn complex topics in a fun and easy way through virtual reality devices. Students interact with objects in the virtual environment and can learn more about it. For example; by organizing digital tours to a museum or zoo in a completely different country, lessons can be taught in the company of a teacher as if they were there at that moment. In the light of all these, this study is a compilation study. In this context, augmented reality technologies were introduced and attention was drawn to their use in different fields of education with their examples. As a suggestion at the end of the study, it was emphasized that the prepared sections should be carefully read by the educators and put into practice in their lessons. In addition it was also pointed out that it should be preferred in order to communicate effectively with students by interacting in real time, especially during the pandemic process.

Using Multi-Modal Data to Examine Equity in Activity-Monitor Gaming Within Real-World Communities

2020 ◽  
pp. 118-142
Author(s):  
Mary K. Stewart ◽  
Danielle E. Hagood ◽  
Cynthia Carter Ching
Keyword(s):  
Physical Activity ◽  
Real World ◽  
Physical World ◽  
Future Research ◽  
Game Play ◽  
The Real ◽  
Modal Data ◽  
World Energy ◽  
Middle School Youth ◽  
Diverse Groups

It is rare for research on augmented-reality games to examine equity and access as grounded in features of the actual neighborhoods where game play takes place, and in the affordances of communities and their built environments for gamified ambulatory physical activity in the real world. This chapter studies two diverse groups of middle-school youth, situated in urban and suburban areas, who wore activity monitors as they went through daily activities and played an online game that synced with their monitors. The game drew data from the wearable devices so that the more youth engaged in step-countable physical activity in the real world, the more game-world energy they earned. This chapter analyzes the actual communities where our participants' activity and game play was situated. The chapter lays out the multi-modal data sources in that analysis and provides some potential models that can be employed by others in related work. Finally, the chapter closes by articulating some directions and concerns for future research in a gamified physical world.

Virtual Ethics

2010 ◽  
pp. 102-108 ◽  
Author(s):  
John Nordlinger
Keyword(s):  
Real World ◽  
Virtual World ◽  
Thought Experiments ◽  
Physical World ◽  
The Real

Many of the opportunities in the virtual world are not available in the physical world, others open our eyes to real world opportunities we couldn’t imagine and teach us vocabulary and skills applicable to the real world. This chapter explores some of the connections between virtual decisions and real consequences, as envisioned in thought experiments of early philosophers from both eastern and western traditions.

scholarly journals Parkour, The Affective Appropriation of Urban Space, and the Real/Virtual Dialectic

2012 ◽  
Vol 11 (3) ◽  
pp. 229-253 ◽  
Author(s):  
Jeffrey L. Kidder
Keyword(s):  
Information And Communication Technologies ◽  
Urban Space ◽  
Virtual Worlds ◽  
Real World ◽  
Communication Technologies ◽  
Physical World ◽  
The Internet ◽  
The Real ◽  
Information And Communication ◽  
The City

Parkour is a new sport based on athletically and artistically overcoming urban obstacles. In this paper, I argue that the real world practices of parkour are dialectically intertwined with the virtual worlds made possible by information and communication technologies. My analysis of parkour underscores how globalized ideas and images available through the Internet and other media can be put into practice within specific locales. Practitioners of parkour, therefore, engage their immediate, physical world at the same time that they draw upon an imagination enabled by their on–screen lives. As such, urban researchers need to consider the ways that virtual worlds can change and enhance how individuals understand and utilize the material spaces of the city.

scholarly journals TagHunt: Uma plataforma Combinando a Internet das Coisas com Scavenger Hunt Games

2015 ◽  
Author(s):  
Kiev Gama ◽  
Rafael Wanderley ◽  
Daniel Maranhão ◽  
Vinicius Garcia
Keyword(s):  
Internet Of Things ◽  
Physical Environment ◽  
Physical World ◽  
The Internet ◽  
Qr Code ◽  
Ordinary Objects ◽  
Physical Environments ◽  
Heterogeneous Objects ◽  
The Internet Of Things

The “Internet of Things” (IoT) brings the notion of heterogeneous objects using ubiquitous technologies to interact among them and with the physical environment through technologies such as Bluetooth, ZigBee, GPRS, NFC, QR code, among others. Based on the possibility of linking ordinary objects from the physical world to the Internet, this paper proposes and details a platform called TagHunt, for creating and playing scavenger hunt games. This platform leverages on smartphones’ capability to interact with ordinary objects using IoT-based technologies such as NFC and QR Code, stimulating the player to interact with physical environments looking for “clues” in the game.

scholarly journals Rule Extraction by Structural Learning with an Immediate Critic

1999 ◽  
Vol 3 (5) ◽  
pp. 341-347 ◽  
Author(s):  
Masumi Ishikawa ◽  
Keyword(s):  
Neural Networks ◽  
Reinforcement Learning ◽  
Supervised Learning ◽  
Real World ◽  
Back Propagation ◽  
Rule Extraction ◽  
Distributed Representation ◽  
Structural Learning ◽  
The Real ◽  
Training Samples

Studies on rule extraction using neural networks have exclusively adopted supervised learning, in which correct outputs are always given as training samples. The real world, however, does not always provide correct answers. We advocate the use of learning with an immediate critic, which is simple reinforcement learning. It uses an immediate binary reinforcement signal indicating whether or not an output is correct. This, of course, makes learning more difficult and time-consuming than supervised learning. Learning with an immediate critic alone, however, is not powerful enough in extracting rules from data because distributed representation emerges just as in back propagation learning. We propose to combine learning with an immediate critic and structural learning with forgetting (SLF) - structural learning with an immediate critic and forgetting (SLCF). A procedure of rule extraction from data by SLCF is similar to that by SLF. Applications of the proposed method to rule extraction from lenses data demonstrate its effectiveness.

Sign in / Sign up

Export Citation Format

Share Document