scholarly journals Evaluation of Integrated Assessment Model hindcast experiments: A case study of the GCAM 3.0 land use module

2017 ◽  
Author(s):  
Abigail C. Snyder ◽  
Robert P. Link ◽  
Katherine V. Calvin

Abstract. Hindcasting experiments (conducting a model forecast for a time period in which observational data is available) are rarely undertaken in the Integrated Assessment Model (IAM) community. When they are undertaken, the results are often evaluated using global aggregates or otherwise highly aggregated skill scores that mask deficiencies. We select a set of deviation based measures that can be applied at different spatial scales (regional versus global) to make evaluating the large number of variable-region combinations in IAMs more tractable. We also identify performance benchmarks for these measures, based on the statistics of the observational dataset, that allow a model to be evaluated in absolute terms rather than relative to the performance of other models at similar tasks. This is key in the integrated assessment community, where there often are not multiple models conducting hindcast experiments to allow for model intercomparison. The performance benchmarks serve a second purpose, providing information about the reasons a model may perform poorly on a given measure and therefore identifying opportunities for improvement. As a case study, the measures are applied to the results of a past hindcast experiment focusing on land allocation in the Global Change Assessment Model (GCAM) version 3.0. We find quantitative evidence that global aggregates alone are not sufficient for evaluating IAMs, such as GCAM, that require global supply to equal global demand at each time period. Additionally, the deviation measures examined in this work successfully identity parametric and structural changes that may improve land allocation decisions in GCAM. Future work will involve implementing the suggested improvements to the GCAM land allocation system identified by the measures in this work, using the measures to quantify performance improvement due to these changes, and, ideally, applying these measures to other sectors of GCAM and other land allocation models.

2017 ◽  
Vol 10 (12) ◽  
pp. 4307-4319 ◽  
Author(s):  
Abigail C. Snyder ◽  
Robert P. Link ◽  
Katherine V. Calvin

Abstract. Hindcasting experiments (conducting a model forecast for a time period in which observational data are available) are being undertaken increasingly often by the integrated assessment model (IAM) community, across many scales of models. When they are undertaken, the results are often evaluated using global aggregates or otherwise highly aggregated skill scores that mask deficiencies. We select a set of deviation-based measures that can be applied on different spatial scales (regional versus global) to make evaluating the large number of variable–region combinations in IAMs more tractable. We also identify performance benchmarks for these measures, based on the statistics of the observational dataset, that allow a model to be evaluated in absolute terms rather than relative to the performance of other models at similar tasks. An ideal evaluation method for hindcast experiments in IAMs would feature both absolute measures for evaluation of a single experiment for a single model and relative measures to compare the results of multiple experiments for a single model or the same experiment repeated across multiple models, such as in community intercomparison studies. The performance benchmarks highlight the use of this scheme for model evaluation in absolute terms, providing information about the reasons a model may perform poorly on a given measure and therefore identifying opportunities for improvement. To demonstrate the use of and types of results possible with the evaluation method, the measures are applied to the results of a past hindcast experiment focusing on land allocation in the Global Change Assessment Model (GCAM) version 3.0. The question of how to more holistically evaluate models as complex as IAMs is an area for future research. We find quantitative evidence that global aggregates alone are not sufficient for evaluating IAMs that require global supply to equal global demand at each time period, such as GCAM. The results of this work indicate it is unlikely that a single evaluation measure for all variables in an IAM exists, and therefore sector-by-sector evaluation may be necessary.


2020 ◽  
Author(s):  
Stefan C. Dekker ◽  
Maria J. Santos ◽  
Hanneke Van 'tVeen ◽  
Detlef P. van Vuuren

<p>The variabilities in both time and space of the flows between the components of the water-food-energy are dependent on many driving factors. In this study we use global scenarios from the Integrated Assessment Model IMAGE to analyse future changes in flows in the water, food and energy nexus. With Sankey diagrams we show how flows between energy and food production will likely increase by 60% and water consumption by 20% in 2050 by using a reference scenario. The inclusion of climate action policies, combined with dietary changes, increased yield efficiency and food waste reduction leads to similar resources uses of water and land, and much lower greenhouse gas emissions compared to 2010.</p><p>We found that based on data, spatial scales are an important but complicating factor in nexus analysis. This is because different resources have their own physical and spatial scale characteristics within the nexus. To examine the effect of scaling on future nexus development, we analyse how local decisions and local resource availability of the use of biomass as energy source impacts other resources. Biomass use potentially impacts forest systems and might compete with land for food and water resources within the nexus. The use of biomass and more specifically charcoal will likely further increase mainly due to urbanization in developing countries. We have examined how different shared socio economic pathway (SSP) scenarios result in (i) future demand for biomass for energy and compare those to measured (with remote sensing) and modelled net primary productivity values of forested systems, (ii) estimate the amount of land needed for biomass production that might compete with food production, and (iii) estimate the water amount needed to produce biomass to meet the different biomass demands. We found that current productivity of non-protected forests is globally higher than the demand, but regionally it closely meets the demand for tropical areas in Central America and Africa. This while tropical areas in South America and Indonesia show decreasing biomass demands for energy for the SSP1-SSP3 scenarios. From this analysis we clearly see differences at regional scales in the competition between the resources land and water are found. </p><p>We conclude that a nexus framework analysis which estimates flows between the different components across scales is fundamental to understand system sustainability. Such approach benefits from combining global scenarios of Integrated Assessment models with local conditions to understand the sustainability in the nexus in time and space.</p>


2007 ◽  
Vol 85 (3) ◽  
pp. 757-773 ◽  
Author(s):  
I.M. Caldwell ◽  
V.W. Maclaren ◽  
J.M. Chen ◽  
W.M. Ju ◽  
S. Zhou ◽  
...  

2021 ◽  
Vol 167 (3-4) ◽  
Author(s):  
Camilla C. N. de Oliveira ◽  
Gerd Angelkorte ◽  
Pedro R. R. Rochedo ◽  
Alexandre Szklo

Sign in / Sign up

Export Citation Format

Share Document