Appreciative Learning for Immersive Reward System in Education Game Development

Game replayability is very important in serious game to maximize the understanding for the learning content. The replayability is the result from the gameplay experience. Games have the advantage of providing a fun experience, and immersion is a vital element in game design to produce the experience. However, the design of immersion in games is often not well conceptualized so that it does not produce the expected experience. This study uses Appreciative Learning based reward system, which focuses on positive things such as peak achievements, opportunities, exploration of potential and optimism for the future. The reward activity consists of four stages, namely Discovery, Dream, Design and Destiny. Reward personalization is done by regulating reward behavior using artificial intelligence which runs in all four stages. Appreciative Learning will be used to design immersive experiences consisting of sensory, imaginary and challenge-based immersion, which are the three main elements of immersive games. Intelligent agent behavior is modeled using the Finite State Machine. This study produces an immersive reward design that is applied to the concept of Appreciative Learning in designing a serious game.

Social Presence as a Moderator of the Effect of Agent Behavior on Emotional Experience in Social Interactions in Virtual Reality

Background: Exposure therapy involves exposure to feared stimuli and is considered to be the gold-standard treatment for anxiety disorders. While its application in Virtual Reality (VR) has been very successful for phobic disorders, the effects of exposure to virtual social stimuli in Social Anxiety Disorder are heterogeneous. This difference has been linked to demands on realism and presence, particularly social presence, as a pre-requisite in evoking emotional experiences in virtual social interactions. So far, however, the influence of social presence on emotional experience in social interactions with virtual agents remains unknown.Objective: We investigated the relationship between realism and social presence and the moderating effect of social presence on the relationship between agent behavior and experienced emotions in virtual social interaction.Methods: Healthy participants (N = 51) faced virtual agents showing supportive and dismissive behaviors in two virtual environments (short interactions and oral presentations). At first, participants performed five blocks of short one-on-one interactions with virtual agents (two male and two female agents per block). Secondly, participants gave five presentations in front of an audience of 16 agents. In each scenario, agent behavior was a within subjects factor, resulting in one block of neutral, two blocks of negative, and two blocks of positive agent behavior. Ratings of agent behavior (valence and realism), experience (valence and arousal), and presence (physical and social) were collected after every block. Moderator effects were investigated using mixed linear models with random intercepts. Correlations were analyzed via repeated measures correlations.Results: Ratings of valence of agent behaviors showed reliable relationships with experienced valence and less reliable relationships with experienced arousal. These relationships were moderated by social presence in the presentation scenario. Results for the interaction scenario were weaker but potentially promising for experimental studies. Variations in social presence and realism over time were correlated but social presence proved a more reliable moderator.Conclusion: Our findings emphasize the role of social presence for emotional experience in response to specific agent behaviors in virtual social interactions. While these findings should be replicated with experimental designs and in clinical samples, variability in social presence might account for heterogeneity in efficacy of virtual exposure to treat social anxiety disorder.

Evaluating Strategic Structures in Multi-Agent Inverse Reinforcement Learning

A core question in multi-agent systems is understanding the motivations for an agent's actions based on their behavior. Inverse reinforcement learning provides a framework for extracting utility functions from observed agent behavior, casting the problem as finding domain parameters which induce such a behavior from rational decision makers. We show how to efficiently and scalably extend inverse reinforcement learning to multi-agent settings, by reducing the multi-agent problem to N single-agent problems while still satisfying rationality conditions such as strong rationality. However, we observe that rewards learned naively tend to lack insightful structure, which causes them to produce undesirable behavior when optimized in games with different players from those encountered during training. We further investigate conditions under which rewards or utility functions can be precisely identified, on problem domains such as normal-form and Markov games, as well as auctions, where we show we can learn reward functions that properly generalize to new settings.

Market sentiment and heterogeneous agents in an evolutive financial model

We study a financial market populated by heterogeneous agents, whose decisions are driven by “animal spirits”. Each agent may have either correct, optimistic or pessimistic beliefs about the fundamental value, which can change from time to time based on an evolutionary mechanism. The evolutionary selection of beliefs depends on a weighted evaluation of the general market sentiment perceived by the agents and on a profitability measure of the existent strategies. As the relevance given to the sentiment index increases, a herding phenomenon in agent behavior may occur and animal spirits can drive the market toward polarized economic regimes, which coexist and are characterized by persistent high or low levels of optimism and pessimism. This conduct is detectable from agents polarized shares and beliefs, which in turn influence the price level. Such polarized regimes can consist in stable steady states or can be characterized by endogenous dynamics, generating persistent alternating waves of optimism and pessimism, as well as return distributions displaying the typical features of financial time series, such as fat tails, excess volatility and multifractality. Moreover, we show that if the sentiment has no or low relevance on belief selection, those stylized facts are abated or are missing from the simulated time series.

Advances in the agent-based modeling of economic and social behavior

In this review we discuss advances in the agent-based modeling of economic and social systems. We show the state of the art of the heuristic design of agents and how behavioral economics and laboratory experiments have improved the modeling of agent behavior. We further discuss how economic networks and social systems can be modeled and we discuss novel methodology and data sources. Lastly, we present an overview of estimation techniques to calibrate and validate agent-based models and show avenues for future research.