Sensitivity analysis of agent-based models: a new protocol

Agent-based models (ABMs) are increasingly used in the management sciences. Though useful, ABMs are often critiqued: it is hard to discern why they produce the results they do and whether other assumptions would yield similar results. To help researchers address such critiques, we propose a systematic approach to conducting sensitivity analyses of ABMs. Our approach deals with a feature that can complicate sensitivity analyses: most ABMs include important non-parametric elements, while most sensitivity analysis methods are designed for parametric elements only. The approach moves from charting out the elements of an ABM through identifying the goal of the sensitivity analysis to specifying a method for the analysis. We focus on four common goals of sensitivity analysis: determining whether results are robust, which elements have the greatest impact on outcomes, how elements interact to shape outcomes, and which direction outcomes move when elements change. For the first three goals, we suggest a combination of randomized finite change indices calculation through a factorial design. For direction of change, we propose a modification of individual conditional expectation (ICE) plots to account for the stochastic nature of the ABM response. We illustrate our approach using the Garbage Can Model, a classic ABM that examines how organizations make decisions.

Uncertainty Quantification, Model Calibration and Sensitivity

Better understanding of the behaviour of agent-based models, aimed at embedding them in the broader, model-based line of scientific enquiry, requires a comprehensive framework for analysing their results. Seeing models as tools for experimenting in silico, this chapter discusses the basic tenets and techniques of uncertainty quantification and experimental design, both of which can help shed light on the workings of complex systems embedded in computational models. In particular, we look at: relationships between model inputs and outputs, various types of experimental design, methods of analysis of simulation results, assessment of model uncertainty and sensitivity, which helps identify the parts of the model that matter in the experiments, as well as statistical tools for calibrating models to the available data. We focus on the role of emulators, or meta-models – high-level statistical models approximating the behaviour of the agent-based models under study – and in particular, on Gaussian processes (GPs). The theoretical discussion is illustrated by applications to the Routes and Rumours model of migrant route formation introduced before.

The Boundaries of Cognition and Decision Making

This chapter outlines the role that individual-level empirical evidence gathered from psychological experiments and surveys can play in informing agent-based models, and the model-based approach more broadly. To begin with, we provide an overview of the way that this empirical evidence can be used to inform agent-based models. Additionally, we provide three detailed exemplars that outline the development and implementation of experiments conducted to inform an agent-based model of asylum migration, as well as how such data can be used. There is also an extended discussion of important considerations and potential limitations when conducting laboratory or online experiments and surveys, followed by a brief introduction to exciting new developments in experimental methodology, such as gamification and virtual reality, that have the potential to address some of these limitations and open the door to promising and potentially very fruitful new avenues of research.

Blockchain-Based Wine Supply Chain for the Industry Advancement

The wine sector is one of the most ‘amazing’ and significant agri-food sectors worldwide since ancient times, considering revenue or employment as well as health aspects. This article aims to describe the impact of the implementation of blockchain technology (BCT) in the wine supply chain. After the literature review, the study is based on Agent Based Models (ABMs) and carried out by the GAMA program. Then, the model and simulation of BCT wine supply chain is designed. Finally, the paper compares traditional and BCT-based supply chains, and the advantages of the last one are evident. Blockchain is a useful tool to ensure a traceability system and to protect the production from any type of fraud and contamination.

Modelling Complexity and Uncertainty in Fisheries Stock Assessment

<p>Stock assessment models are used to determine the population size of fish stocks. Although stock assessment models are complex, they still make simplifying assumptions. Generally, they treat each species separately, include little, if any, spatial structure, and may not adequately quantify uncertainty. These assumptions can introduce bias and can lead to incorrect inferences. This thesis is about more realistic models and their inference. This realism may be incorporated by explicitly modelling complex processes, or by admitting our uncertainty and modelling it correctly. We develop an agent-based model that can describe fish populations as a collection of individuals which differ in their growth, maturation, migration, and mortality. The aim of this model is to better capture the richness in natural processes that determine fish abundance and subsequent population response to anthropogenic removals. However, this detail comes at considerable computational cost. A single model run can take many hours, making inference using standard methods impractical. We apply this model to New Zealand snapper (Pagurus auratus) in northern New Zealand. Next, we developed an age-structured state-space model. We suggest that this sophisticated model has the potential to better represent uncertainty in stock assessment. However, it pushes the boundaries of the current practical limits of computing and we admit that its practical application remains limited until the MCMC mixing issues that we encountered can be resolved. The processes that underpin agent-based models are complex and we may need to seek new sources of data to inform these types of models. To make a start here we derive a state-space model to estimate the path taken by individual fish from the day they are tagged to the day of their recapture. The model uses environmental information collected using pop-up satellite archival tags. We use tag recorded depth and oceanographic temperature to estimate the location at any given time. We apply this model to Antarctic toothfish (Dissostichus mawsoni) in the Ross Sea. Finally, to reduce the computational burden of agent-based models we use Bayesian emulation. This approach replaces the simulation model with an approximating algorithm called an emulator. The emulator is calibrated using relatively few runs of the original model. A good emulator provides a close approximation to the original model and has significant speed gains. Thus, inferences become tractable. We have made the first steps towards developing a tractable approach to fisheries modelling in complex settings through the creation of realistic models, and their emulation. With further development, Bayesian emulation could result in the increased ability to consider and evaluate innovative methods in fisheries modelling. Future avenues for application and exploration range from spatial and multi species models, to ecosystem-based models and beyond.</p>

Modelling Complexity and Uncertainty in Fisheries Stock Assessment

<p>Stock assessment models are used to determine the population size of fish stocks. Although stock assessment models are complex, they still make simplifying assumptions. Generally, they treat each species separately, include little, if any, spatial structure, and may not adequately quantify uncertainty. These assumptions can introduce bias and can lead to incorrect inferences. This thesis is about more realistic models and their inference. This realism may be incorporated by explicitly modelling complex processes, or by admitting our uncertainty and modelling it correctly. We develop an agent-based model that can describe fish populations as a collection of individuals which differ in their growth, maturation, migration, and mortality. The aim of this model is to better capture the richness in natural processes that determine fish abundance and subsequent population response to anthropogenic removals. However, this detail comes at considerable computational cost. A single model run can take many hours, making inference using standard methods impractical. We apply this model to New Zealand snapper (Pagurus auratus) in northern New Zealand. Next, we developed an age-structured state-space model. We suggest that this sophisticated model has the potential to better represent uncertainty in stock assessment. However, it pushes the boundaries of the current practical limits of computing and we admit that its practical application remains limited until the MCMC mixing issues that we encountered can be resolved. The processes that underpin agent-based models are complex and we may need to seek new sources of data to inform these types of models. To make a start here we derive a state-space model to estimate the path taken by individual fish from the day they are tagged to the day of their recapture. The model uses environmental information collected using pop-up satellite archival tags. We use tag recorded depth and oceanographic temperature to estimate the location at any given time. We apply this model to Antarctic toothfish (Dissostichus mawsoni) in the Ross Sea. Finally, to reduce the computational burden of agent-based models we use Bayesian emulation. This approach replaces the simulation model with an approximating algorithm called an emulator. The emulator is calibrated using relatively few runs of the original model. A good emulator provides a close approximation to the original model and has significant speed gains. Thus, inferences become tractable. We have made the first steps towards developing a tractable approach to fisheries modelling in complex settings through the creation of realistic models, and their emulation. With further development, Bayesian emulation could result in the increased ability to consider and evaluate innovative methods in fisheries modelling. Future avenues for application and exploration range from spatial and multi species models, to ecosystem-based models and beyond.</p>