scholarly journals Promoting the Emergence of Behavior Norms in a Principal–Agent Problem—An Agent-Based Modeling Approach Using Reinforcement Learning

2021 ◽  
Vol 11 (18) ◽  
pp. 8368
Author(s):  
Saeed Harati ◽  
Liliana Perez ◽  
Roberto Molowny-Horas
Keyword(s):  
Reinforcement Learning ◽  
Social Systems ◽  
Learning Algorithms ◽  
Agent Based Modeling ◽  
Principal Agent ◽  
Temporal Difference ◽  
Agent Based ◽  
Principal Agent Problem ◽  
The Individual ◽  
Better Than

One of the complexities of social systems is the emergence of behavior norms that are costly for individuals. Study of such complexities is of interest in diverse fields ranging from marketing to sustainability. In this study we built a conceptual Agent-Based Model to simulate interactions between a group of agents and a governing agent, where the governing agent encourages other agents to perform, in exchange for recognition, an action that is beneficial for the governing agent but costly for the individual agents. We equipped the governing agent with six Temporal Difference Reinforcement Learning algorithms to find sequences of decisions that successfully encourage the group of agents to perform the desired action. Our results show that if the individual agents’ perceived cost of the action is low, then the desired action can become a trend in the society without the use of learning algorithms by the governing agent. If the perceived cost to individual agents is high, then the desired output may become rare in the space of all possible outcomes but can be found by appropriate algorithms. We found that Double Learning algorithms perform better than other algorithms we used. Through comparison with a baseline, we showed that our algorithms made a substantial difference in the rewards that can be obtained in the simulations.

ANALYZING THE AGENT-BASED MODEL AND ITS IMPLICATIONS

2003 ◽  
Vol 06 (03) ◽  
pp. 331-347 ◽  
Author(s):  
YUTAKA I. LEON SUEMATSU ◽  
KEIKI TAKADAMA ◽  
NORBERTO E. NAWA ◽  
KATSUNORI SHIMOHARA ◽  
OSAMU KATAI
Keyword(s):  
Social Sciences ◽  
Internal Structure ◽  
Design Methodology ◽  
Social Systems ◽  
Agent Based Model ◽  
Social Phenomena ◽  
Agent Based ◽  
Individual Level ◽  
The Social ◽  
The Individual

Agent-based models (ABMs) have been attracting the attention of researchers in the social sciences, becoming a prominent paradigm in the study of complex social systems. Although a great number of models have been proposed for studying a variety of social phenomena, no general agent design methodology is available. Moreover, it is difficult to validate the accuracy of these models. For this reason, we believe that some guidelines for ABMs design must be devised; therefore, this paper is a first attempt to analyze the levels of ABMs, identify and classify several aspects that should be considered when designing ABMs. Through our analysis, the following implications have been found: (1) there are two levels in designing ABMs: the individual level, related to the design of the agents' internal structure, and the collective level, which concerns the design of the agent society or macro-dynamics of the model; and (2) the mechanisms of these levels strongly affect the outcomes of the models.

TRADER SPECIES WITH DIFFERENT DECISION STRATEGIES AND PRICE DYNAMICS IN FINANCIAL MARKETS: AN AGENT-BASED MODELING PERSPECTIVE

2010 ◽  
Vol 09 (02) ◽  
pp. 327-344 ◽  
Author(s):  
WEI ZHANG ◽  
GEN LI ◽  
XIONG XIONG ◽  
YONG JIE ZHANG
Keyword(s):  
Financial Markets ◽  
Asset Price ◽  
Agent Based Modeling ◽  
Price Dynamics ◽  
Trading Behavior ◽  
Agent Based ◽  
Price Process ◽  
Trading Decisions ◽  
The Individual ◽  
Analytical Approaches

Investors with different trading strategies can be viewed as different "species" in financial markets. Since the asset price is ultimately determined by the individual trading decisions, the combination and evolution of different trader species in financial market ecology will have great impact to the price dynamics. Considering the limitations and shortcomings of traditional analytical approaches in financial economics in dealing with this issue, an agent-based computational model is introduced in this paper. With the co-existence of 3-type trader species that make different decisions based on their own beliefs and constrains, it is found that although rational speculation destabilizes the price process with the presence of positive feedback strategy, as suggested in the literature, introducing extra noise trading behavior to the market will make the price process back to a more stationary situation, meaning that the market will be healthier if more diversified trader species co-exist in the markets.

Agent based modeling of the coevolution of hostility and pacifism

2015 ◽  
Vol 26 (09) ◽  
pp. 1550098 ◽  
Author(s):  
Fermin Dalmagro ◽  
Juan Jimenez
Keyword(s):  
Social Systems ◽  
Stable State ◽  
Agent Based Modeling ◽  
Armed Conflicts ◽  
Agent Based ◽  
Public Data ◽  
Prisoners Dilemma ◽  
Quantitative Characteristics ◽  
Global Peace ◽  
Emergent Behaviors

We propose a model based on a population of agents whose states represent either hostile or peaceful behavior. Randomly selected pairs of agents interact according to a variation of the Prisoners Dilemma game, and the probabilities that the agents behave aggressively or not are constantly updated by the model so that the agents that remain in the game are those with the highest fitness. We show that the population of agents oscillate between generalized conflict and global peace, without either reaching a stable state. We then use this model to explain some of the emergent behaviors in collective conflicts, by comparing the simulated results with empirical data obtained from social systems. In particular, using public data reports we show how the model precisely reproduces interesting quantitative characteristics of diverse types of armed conflicts, public protests, riots and strikes.

scholarly journals Iterative SARSA: The Modified SARSA Algorithm for Finding the Optimal Path

2020 ◽  
Vol 8 (6) ◽  
pp. 4333-4338
Keyword(s):  
Reinforcement Learning ◽  
Comparative Analysis ◽  
Mobile Robots ◽  
Learning Algorithm ◽  
Learning Algorithms ◽  
Optimal Path ◽  
Autonomous Mobile Robots ◽  
Current Standard ◽  
Q Learning ◽  
Better Than

This paper presents a thorough comparative analysis of various reinforcement learning algorithms used by autonomous mobile robots for optimal path finding and, we propose a new algorithm called Iterative SARSA for the same. The main objective of the paper is to differentiate between the Q-learning and SARSA, and modify the latter. These algorithms use either the on-policy or off-policy methods of reinforcement learning. For the on-policy method, we have used the SARSA algorithm and for the off-policy method, the Q-learning algorithm has been used. These algorithms also have an impacting effect on finding the shortest path possible for the robot. Based on the results obtained, we have concluded how our algorithm is better than the current standard reinforcement learning algorithms

scholarly journals Using deep reinforcement learning to speed up collective cell migration

2019 ◽  
Vol 20 (S18) ◽  
Author(s):  
Hanxu Hou ◽  
Tian Gan ◽  
Yaodong Yang ◽  
Xianglei Zhu ◽  
Sen Liu ◽  
...  
Keyword(s):  
Cell Migration ◽  
Reinforcement Learning ◽  
Real Life ◽  
Time Lapse ◽  
Research Process ◽  
Agent Based Modeling ◽  
Collective Cell Migration ◽  
Modeling Framework ◽  
Agent Based ◽  
Number Of Cells

Abstract Background Collective cell migration is a significant and complex phenomenon that affects many basic biological processes. The coordination between leader cell and follower cell affects the rate of collective cell migration. However, there are still very few papers on the impacts of the stimulus signal released by the leader on the follower. Tracking cell movement using 3D time-lapse microscopy images provides an unprecedented opportunity to systematically study and analyze collective cell migration. Results Recently, deep reinforcement learning algorithms have become very popular. In our paper, we also use this method to train the number of cells and control signals. By experimenting with single-follower cell and multi-follower cells, it is concluded that the number of stimulation signals is proportional to the rate of collective movement of the cells. Such research provides a more diverse approach and approach to studying biological problems. Conclusion Traditional research methods are always based on real-life scenarios, but as the number of cells grows exponentially, the research process is too time consuming. Agent-based modeling is a robust framework that approximates cells to isotropic, elastic, and sticky objects. In this paper, an agent-based modeling framework is used to establish a simulation platform for simulating collective cell migration. The goal of the platform is to build a biomimetic environment to demonstrate the importance of stimuli between the leading and following cells.

scholarly journals A hierarchical multi-resolution agent-based modeling and simulation framework for household electricity demand profile

SIMULATION ◽  
2020 ◽  
Vol 96 (8) ◽  
pp. 655-678 ◽  
Author(s):  
Imran Mahmood ◽  
Quair-tul-ain ◽  
Hasan Arshad Nasir ◽  
Fahad Javed ◽  
José A Aguado
Keyword(s):  
Modeling And Simulation ◽  
Energy Demand ◽  
Large Scale ◽  
Electricity Consumption ◽  
Agent Based Modeling ◽  
Energy Planning ◽  
Modeling Framework ◽  
Agent Based ◽  
The Individual ◽  
Domestic Electricity

Analyzing demand behavior of end consumers is pivotal in long term energy planning. Various models exist for simulating household load profiles to cater different purposes. A macroscopic viewpoint necessitates modeling of a large-scale population at an aggregate level, whereas a microscopic perspective requires measuring loads at a granular level, pertinent to the individual devices of a household. Both aspects have lucrative benefits, instigating the need to combine them into a modeling framework which allows model scalability and flexibility, and to analyze domestic electricity consumption at different resolutions. In this applied research, we propose a multi-resolution agent-based modeling and simulation (ABMS) framework for estimating domestic electricity consumption. Our proposed framework simulates per minute electricity consumption by combining large neighborhoods, the behavior of household individuals, their interactions with the electrical appliances, their sociological habits and the effects of exogenous conditions such as weather and seasons. In comparison with the existing energy models, our framework uniquely provides a hierarchical, multi-scale, multi-resolution implementation using a multi-layer architecture. This allows the modelers flexibility in order to model large-scale neighborhoods at one end, without any loss of expressiveness in modeling microscopic details of individuals’ activities at house level, and energy consumption at the appliance level, at the other end. The validity of our framework is demonstrated using a case study of 264 houses. A validated ABMS framework will support: (a) Effective energy planning; (b) Estimation of the future energy demand; (c) and the analysis of the complex dynamic behavior of the consumers.

Multi-Agent Simulation in Organizations

2011 ◽  
pp. 2728-2733
Author(s):  
Nikola Vlahovic ◽  
Vlatko Ceric
Keyword(s):  
Social Systems ◽  
Real Life ◽  
Agent Based Modeling ◽  
Multi Agent Systems ◽  
Social Constructs ◽  
Social Phenomena ◽  
Agent Based ◽  
Agent Systems ◽  
Agent Simulation ◽  
Multi Agent

Most economic and business systems are complex, dynamic, and nondeterministic systems. Different modeling techniques have been used for representing real life economic and business organizations either on a macro level (such as national economics) or micro level (such as business processes within a firm or strategies within an industry). Even though general computer simulation was used for modeling various systems (Zeigler, 1976) since the 1970s the limitation of computer resources did not allow for in-depth simulation of dynamic social phenomena. The dynamics of social systems and impact of the behavior of individual entities in social constructs were modeled using mathematical modeling or system dynamics. With the growing interest in multi agent systems that led to its standardization in the 1990s, multi agent systems were proposed for the use of modeling social systems (Gilbert & Conte, 1995). Multi agent simulation was able to provide a high level disintegration of the models and proper treatment of inhomogeneity and individualism of the agents, thus allowing for simulation of cooperation and competition. A number of simulation models were developed in the research of biological and ecological systems, such as models for testing the behavior and communication between social insects (bees and ants). Artificial systems for testing hypothesis about social order and norms, as well as ancient societies (Kohler, Gumerman, & Reynolds, 2005) were also simulated. Since then, agent-based modeling and simulation (ABMS) established itself as an attractive modeling technique (Klugl, 2001; Moss & Davidsson, 2001). Numerous software toolkits were released, such as Swarm, Repast, MASON and SeSAm. These toolkits make agent-based modeling easy enough to be attractive to practitioners from a variety of subject areas dealing with social interactions. They make agent-based modeling accessible to a large number of analysts with less programming experience.

scholarly journals Cybernetics Approach Using Agent-Based Modeling in the Process of Evacuating Educational Institutions in Case of Disasters

2021 ◽  
Vol 13 (18) ◽  
pp. 10277
Author(s):  
Ștefan Ionescu ◽  
Ionuț Nica ◽  
Nora Chiriță
Keyword(s):  
Exhaust System ◽  
Emergency Evacuation ◽  
Agent Based Modeling ◽  
Educational Institutions ◽  
Teaching Staff ◽  
Agent Based ◽  
System A ◽  
The People ◽  
The Individual ◽  
High Degree

In the context of an emergency, evacuating people from a location in the shortest possible time is essential, as is the high degree of safety that people should expect when evacuating. Lately, in Romania there have been more and more fire events generated by different causes. This article will use agent-based modeling to simulate an emergency evacuation model in NetLogo. The model has been used to perform and analyze various scenarios. With the help of NetLogo, we managed to perform 400 simulations with the evacuation of 180 people (students, teachers, and non-teaching staff) based on which we developed several recommendations to streamline the evacuation process in order to reduce the possibility of death. The present research will help to identify the evacuation times from a school, but it will also highlight certain aspects that may occur during the evacuation. The model that was used in this research took into account the individual particularities of the people taking part in the evacuation, emphasizing the effects that form in a crowd of people when evacuating; effects such as the funnel effect, which is caused by the formation of bottlenecks around narrow areas. All these things are part of the analysis of the measurement of entropy of the exhaust system, a problem that has captured all of the specialists’ attention. Finally, solutions have been proposed to improve evacuation time in case of disasters.

Sign in / Sign up

Export Citation Format

Share Document