Combining multiple statistical methods to evaluate the performance of process-based vegetation models across three forest stands

AbstractProcess-based vegetation models are crucial tools to better understand biosphere-atmosphere exchanges and ecophysiological responses to climate change. In this contribution the performance of two global dynamic vegetation models, i.e. CARAIB and ISBACC, and one stand-scale forest model, i.e. 4C, was compared to long-term observed net ecosystem carbon exchange (NEE) time series from eddy covariance monitoring stations at three old-grown European beech (Fagus sylvatica L.) forest stands. Residual analysis, wavelet analysis and singular spectrum analysis were used beside conventional scalar statistical measures to assess model performance with the aim of defining future targets for model improvement. We found that the most important errors for all three models occurred at the edges of the observed NEE distribution and the model errors were correlated with environmental variables on a daily scale. These observations point to possible projection issues under more extreme future climate conditions. Recurrent patterns in the residuals over the course of the year were linked to the approach to simulate phenology and physiological evolution during leaf development and senescence. Substantial model errors occurred on the multi-annual time scale, possibly caused by the lack of inclusion of management actions and disturbances. Other crucial processes defined were the forest structure and the vertical light partitioning through the canopy. Further, model errors were shown not to be transmitted from one time scale to another. We proved that models should be evaluated across multiple sites, preferably using multiple evaluation methods, to identify processes that request reconsideration.

Download Full-text

The sub-annual calibration of hydrological models considering climatic intra-annual variations

10.5194/hess-2017-396 ◽

2017 ◽

Cited By ~ 1

Author(s):

Binru Zhao ◽

Huichao Dai ◽

Dawei Han ◽

Guiwen Rong

Keyword(s):

Time Scale ◽

Temporal Dynamics ◽

Model Performance ◽

Optimal Time ◽

Temperature Pattern ◽

Climate Conditions ◽

Fcm Algorithm ◽

Calibration Scheme ◽

Annual Variations ◽

Improve Model

Abstract. Changing climate leads to change of temporal dynamics of hydrological systems by affecting the catchment conditions. Considering climatic variations when calibrating a hydrological model can improve model performance, which allows parameter sets to vary according to sub-periods with different climate conditions. This study has explored climatic intra-annual variations by using two classification approaches to recognize the sub-periods with similar climatic patterns, Calendar-Based Grouping (CBG) method and Fuzzy C-Means (FCM) algorithm. The model performances of the sub-annual calibration schemes based on these two approaches are compared using the conceptual model IHACRES. The effect of time scales on sub-annual calibration schemes was also studied. Results indicate that the sub-annual calibration scheme based on CBG method performs better than that based on Rainfall-dominated FCM algorithm, since the CBG method has a better performance in recognizing temperature pattern, and the main source of catchment change is from the change of vegetation, which is mainly affected by temperature in the study site. The optimal time scale is dependent on the sub-annual calibration scheme, with bimonthly for CBG method and Temperature-dominated FCM algorithm and seasonal for Rainfall-dominated FCM algorithm. Overall, when using sub-annual calibration schemes, the selection of the partitioning method and time scale is very important to model performances

Download Full-text

Transferability of monthly water balance models under changing climate conditions in an arid catchment

10.5194/egusphere-egu2020-8552 ◽

2020 ◽

Author(s):

Zana Topalovic ◽

Andrijana Todorovic ◽

Jasna Plavsic

Keyword(s):

Water Balance ◽

Model Performance ◽

Subjective Assessment ◽

Water Balance Components ◽

Climate Conditions ◽

Changing Climate ◽

Statistical Measures ◽

Water Balance Models ◽

Model Components ◽

Monthly Water Balance

Assessment of climate change impact on water resources is often based on hydrologic projections developed using monthly water balance models (MWBMs) forced by climate projections. These models are calibrated against historical data but are expected to provide accurate flow simulations under changing climate conditions. However, an evaluation of these models&#8217; performance is needed to explore their applicability under changing climate conditions, assess uncertainties and eventually indicate model components that should be improved. This should be done in a comprehensive evaluation framework specifically tailored to evaluate applicability of MWBMs in changing climatic conditions.&#160;In this study, we evaluated performance of four MWBMs (abcd, Budyko, GR2M and WASMOD) used for hydrologic simulations in the arid Wimmera River catchment in Australia. This catchment is selected as a challenge for model application because it was affected by the Millennium drought, characterised by a decrease in precipitation and a dramatic drop in runoff. The model evaluation within the proposed framework starts with dividing the complete record period into five non-overlapping sub-periods, calibration and cross-validation (i.e., transfers) of the models. The Kling-Gupta efficiency coefficient is used for the calibration in each sub-period. Consistency in model performance, parameter estimates and simulated water balance components across the sub-periods is analysed. Model performance is quantified with statistical performance measures and errors in hydrological signatures. Because the relatively short monthly hydrologic series can lead to biased numerical performance indicators, the framework also includes subjective assessment of model performance and transferability.&#160;&#160;The results show that model transfer between climatically contrasted sub-periods affect all statistical measures of model performance and some hydrologic signatures: standard deviation of flows, high flow percentile and percentage of zero flows. While some signatures are reproduced well in all transfers (baseflow index, lag 1 and lag 12 autocorrelations), suggesting their low informativeness about MWBM performance, many signatures are consistently poorly reproduced, even in the calibrations (seasonal distribution, most flow percentiles, streamflow elasticity). This means that good model performance in terms of statistical measures does not imply good performance in terms of hydrologic signatures, probably because the models are not conditioned to reproduce them. Generally, the greatest drop in performance of all the models is obtained in transfers to the driest period, although abcd and Budyko slightly outperformed GR2M and WASMOD. Subjective assessment of model performance largely corresponds to the numerical indicators.&#160;Simulated water balance components, especially soil and groundwater storages and baseflow, significantly vary across the simulation periods. These results suggest that the model components and the parameters that control them are sensitive to the calibration period. Therefore, improved model conceptualisations (particularly partitioning of fast and slow runoff components) and enhanced calibration strategies that put more emphasis on parameters related to slow runoff are needed. More robust MWBM structures or calibration strategies should advance transferability of MWBMs, which is a prerequisite for effective water resources management under changing climate conditions.

Download Full-text

Evaluation of the performance of CORDEX regional climate models in simulating present climate conditions of Tanzania

Journal of Southern Hemisphere Earth System Science ◽

10.1071/es16005 ◽

2016 ◽

Vol 66 (1) ◽

pp. 32

Author(s):

Philbert Luhunga ◽

Joel Botai ◽

Frederick Kahimba

Keyword(s):

Air Temperature ◽

General Circulation ◽

Climate Models ◽

Regional Climate ◽

Model Performance ◽

Regional Climate Models ◽

General Circulation Models ◽

Variable Model ◽

Climate Conditions ◽

Statistical Measures

Regional climate models (RCMs) are widely used in regional assessment of climate change impacts. However, the reliability of individual models needs to be assessed before using their output for impact assessment. In this study, we evaluate the performance of RCMs from the Coordinated Regional Climate Downscaling Experiment program (CORDEX) to simulate minimum air temperature (TN), maximum air temperature (TX) and rainfall over Tanzania. Output from four RCMs driven by boundary conditions from three General Circulation Models (GCMs) and ERA-Interim data are evaluated against observed data from 22 weather stations. The evaluation is based on determining how well the RCMs reproduce climatological trends, interannual, and annual cycles of TN, TX and rainfall. Statistical measures of model performance that include the bias, root mean square error, correlation and trend analysis are used. It is found that RCMs capture the annual cycle of TN, TX and rainfall well, however underestimate and overestimate the amount of rainfall in March–April–May (MAM) and October–November–December (OND) seasons respectively. Most RCMs reproduce interannual variations of TN, TX and rainfall. The source of uncertainties can be analysed when the same RCM is driven by different GCMs and different RCMs driven by same GCM simulate TN, TX and rainfall differently. It is found that the biases and errors from the RCMs and driving GCMs contribute roughly equally. Overall, the evaluation finds reasonable (although variable) model skill in representing mean climate, interannual variability and temperature trends, suggesting the potential use of CORDEX RCMs in simulating TN, TX and rainfall over Tanzania.

Download Full-text

Distribution of the timber quality attribute ‘knot surface’ in logs of Fagus sylvatica L. from pure and mixed forest stands

European Journal of Forest Research ◽

10.1007/s10342-021-01378-9 ◽

2021 ◽

Author(s):

Kirsten Höwler ◽

Torsten Vor ◽

Peter Schall ◽

Peter Annighöfer ◽

Dominik Seidel ◽

...

Keyword(s):

Fagus Sylvatica ◽

Mixed Forest ◽

European Beech ◽

Forest Stands ◽

Species Identity ◽

Mixed Stands ◽

Mixed Forests ◽

Climate Conditions ◽

Timber Quality ◽

Positive Effects

AbstractResearch on mixed forests has mostly focused on tree growth and productivity, or resistance and resilience in changing climate conditions, but only rarely on the effects of tree species mixing on timber quality. In particular, it is still unclear whether the numerous positive effects of mixed forests on productivity and stability come at the expense of timber quality. In this study, we used photographs of sawn boards from 90 European beech (Fagus sylvatica L.) trees of mixed and pure forest stands to analyze internal timber quality through the quality indicator knot surface that was quantitatively assessed using the software Datinf® Measure. We observed a decrease in knot surface with increasing distance from the pith as well as smaller values in the lower log sections. Regarding the influence of neighborhood species identity, we found only minor effects meaning that timber qualities in mixed stands of beech and Norway spruce (Picea abies (L.) H. Karst.) tended to be slightly worse compared to pure beech stands.

Download Full-text

Assessment of a physical-biogeochemical coupled model system for operational service in the Baltic Sea

Ocean Science ◽

10.5194/os-8-683-2012 ◽

2012 ◽

Vol 8 (4) ◽

pp. 683-701 ◽

Cited By ~ 20

Author(s):

Z. Wan ◽

J. She ◽

M. Maar ◽

L. Jonasson ◽

J. Baasch-Larsen

Keyword(s):

Baltic Sea ◽

Inorganic Nitrogen ◽

Biogeochemical Model ◽

Observation Data ◽

Model Assessment ◽

Model Errors ◽

The Baltic Sea ◽

Statistical Measures ◽

The Baltic ◽

Operational Service

Abstract. Thanks to the abundant observation data, we are able to deploy the traditional point-to-point comparison and statistical measures in combination with a comprehensive model validation scheme to assess the skills of the biogeochemical model ERGOM in providing an operational service for the Baltic Sea. The model assessment concludes that the operational products can resolve the main observed seasonal features for phytoplankton biomass, dissolved inorganic nitrogen, dissolved inorganic phosphorus and dissolved oxygen in euphotic layers as well as their vertical profiles. This assessment reflects that the model errors of the operational system at the current stage are mainly caused by insufficient light penetration, excessive organic particle export downward, insufficient regional adaptation and some from improper initialization. This study highlights the importance of applying multiple schemes in order to assess model skills rigidly and identify main causes for major model errors.

Download Full-text

Space-time variability of hydrological drought and wetness in Iran using NCEP/NCAR and GPCC datasets

Hydrology and Earth System Sciences ◽

10.5194/hess-14-1919-2010 ◽

2010 ◽

Vol 14 (10) ◽

pp. 1919-1930 ◽

Cited By ~ 21

Author(s):

T. Raziei ◽

I. Bordi ◽

L. S. Pereira ◽

A. Sutera

Keyword(s):

Time Series ◽

Singular Spectrum Analysis ◽

Standardized Precipitation Index ◽

Principal Component ◽

Reanalysis Data ◽

Space Time ◽

Hydrological Drought ◽

Time Variability ◽

Climate Conditions ◽

Good Agreement

Abstract. Space-time variability of hydrological drought and wetness over Iran is investigated using the National Centers for Environmental Prediction/National Center for Atmospheric Research (NCEP/NCAR) reanalysis and the Global Precipitation Climatology Centre (GPCC) dataset for the common period 1948–2007. The aim is to complement previous studies on the detection of long-term trends in drought/wetness time series and on the applicability of reanalysis data for drought monitoring in Iran. Climate conditions of the area are assessed through the Standardized Precipitation Index (SPI) on 24-month time scale, while Principal Component Analysis (PCA) and Varimax rotation are used for investigating drought/wetness variability, and drought regionalization, respectively. Singular Spectrum Analysis (SSA) is applied to the time series of interest to extract the leading nonlinear components and compare them with linear fittings. Differences in drought and wetness area coverage resulting from the two datasets are discussed also in relation to the change occurred in recent years. NCEP/NCAR and GPCC are in good agreement in identifying four sub-regions as principal spatial modes of drought variability. However, the climate variability in each area is not univocally represented by the two datasets: a good agreement is found for south-eastern and north-western regions, while noticeable discrepancies occur for central and Caspian sea regions. A comparison with NCEP Reanalysis II for the period 1979–2007, seems to exclude that the discrepancies are merely due to the introduction of satellite data into the reanalysis assimilation scheme.

Download Full-text

Investigation and Optimization of the C-ANN Structure in Predicting the Compressive Strength of Foamed Concrete

Materials ◽

10.3390/ma13051072 ◽

2020 ◽

Vol 13 (5) ◽

pp. 1072 ◽

Cited By ~ 19

Author(s):

Dong Van Dao ◽

Hai-Bang Ly ◽

Huong-Lan Thi Vu ◽

Tien-Thinh Le ◽

Binh Thai Pham

Keyword(s):

Compressive Strength ◽

Civil Engineering ◽

Mean Squared Error ◽

Model Performance ◽

Absolute Error ◽

Machine Learning Algorithms ◽

Coefficient Of Determination ◽

Dry Density ◽

Statistical Measures ◽

Foamed Concrete

Development of Foamed Concrete (FC) and incessant increases in fabrication technology have paved the way for many promising civil engineering applications. Nevertheless, the design of FC requires a large number of experiments to determine the appropriate Compressive Strength (CS). Employment of machine learning algorithms to take advantage of the existing experiments database has been attempted, but model performance can still be improved. In this study, the performance of an Artificial Neural Network (ANN) was fully analyzed to predict the 28 days CS of FC. Monte Carlo simulations (MCS) were used to statistically analyze the convergence of the modeled results under the effect of random sampling strategies and the network structures selected. Various statistical measures such as Coefficient of Determination (R2), Mean Absolute Error (MAE), and Root Mean Squared Error (RMSE) were used for validation of model performance. The results show that ANN is a highly efficient predictor of the CS of FC, achieving a maximum R2 value of 0.976 on the training part and an R2 of 0.972 on the testing part, using the optimized C-ANN-[3–4–5–1] structure, which compares with previous published studies. In addition, a sensitivity analysis using Partial Dependence Plots (PDP) over 1000 MCS was also performed to interpret the relationship between the input parameters and 28 days CS of FC. Dry density was found as the variable with the highest impact to predict the CS of FC. The results presented could facilitate and enhance the use of C-ANN in other civil engineering-related problems.

Download Full-text

Ages and transit times as important diagnostics of model performance for predicting carbon dynamics in terrestrial vegetation models

10.5194/bg-2017-308-rc1 ◽

2017 ◽

Author(s):

Anonymous

Keyword(s):

Model Performance ◽

Carbon Dynamics ◽

Terrestrial Vegetation ◽

Transit Times ◽

Vegetation Models

Download Full-text

Small-scale processes with large-scale impacts: Investigating canopy structure controls on energy fluxes to the forest snowpack

10.5194/egusphere-egu2020-19027 ◽

2020 ◽

Author(s):

Giulia Mazzotti ◽

Richard Essery ◽

Johanna Malle ◽

Clare Webster ◽

Tobias Jonas

Keyword(s):

High Resolution ◽

Land Surface ◽

Large Scale ◽

Meteorological Data ◽

Canopy Structure ◽

Model Performance ◽

Turbulent Fluxes ◽

Temporal Variations ◽

Small Scale ◽

Forest Stands

Forest canopies strongly affect snowpack energetics during wintertime. In discontinuous forest stands, spatio-temporal variations in radiative and turbulent fluxes create complex snow distribution and melt patterns, with further impacts on the hydrological regimes and on the land surface properties of seasonally snow-covered forested environments.As increasingly detailed canopy structure datasets are becoming available, canopy-induced energy exchange processes can be explicitly represented in high-resolution snow models. We applied the modelling framework FSM2 to obtain spatially distributed simulations of the forest snowpack in subalpine and boreal forest stands at high spatial (2m) and temporal (10min) resolution. Modelled sub-canopy radiative and turbulent fluxes were compared to detailed meteorological data of incoming irradiances, air and snow surface temperatures. These were acquired with novel observational systems, including 1) a motorized cable car setup recording spatially and temporally resolved data along a transect and 2) a handheld setup designed to capture temporal snapshots of 2D spatial distributions across forest discontinuities.The combination of high-resolution modelling and multi-dimensional datasets allowed us to assess model performance at the level of individual energy balance components, under various meteorological conditions and across canopy density gradients. We showed which canopy representation strategies within FSM2 best succeeded in reproducing snowpack energy transfer dynamics in discontinuous forests, and derived implications for implementing forest snow processes in coarser-resolution models.

Download Full-text

Effect of inaccurate specification of time-correlated model error in an Ensemble Smoother

10.5194/egusphere-egu2020-559 ◽

2020 ◽

Author(s):

Haonan Ren ◽

Peter Jan Van Leeuwen ◽

Javier Amezcua

Keyword(s):

Data Assimilation ◽

Time Scale ◽

Correlation Time ◽

Time Window ◽

Model Error ◽

Multiple Time ◽

Model Errors ◽

Time Step ◽

Single Observation ◽

Kalman Smoother

Data assimilation has been often performed under the perfect model assumption known as the strong-constraint setting. There is an increasing number of researches accounting for the model errors, the weak-constrain setting, but often with different degrees of approximation or simplification without knowing their impact on the data assimilation results. We investigate what effect inaccurate model errors, in particular, the an inaccurate time correlation, can have on data assimilation results, with a Kalman Smoother and the Ensemble Kalman Smoother. We choose a linear auto-regressive model for the experiment. We assume the true state of the system has the correct and fixed correlation time-scale&#160;&#969;r in the model errors, and the prior or the background generated by the model contains the model error with the fixed, guessed time-scale &#969;g which differs from the correct one and is also used in the data assimilation process. There are 10 variables in the system and we separate the simulation period into multiple time-windows. And we use a fairly large ensemble size (up to 200 ensemble members) to improve the accuracy of the data assimilation results. In order to evaluate the performance of the EnKS with auto-correlated model errors, we calculate the ratio of root-mean-square error over the spread of all ensemble members. The results with a single observation at the end of the simulation time-window show that, using an underestimated correlation time-scale leads to overestimated spread of the ensemble, and with an overestimated time-scale, the results show underestimation in the ensemble spread. However, with very dense observation frequency, observing every time-step for instance, the results are completely opposite to the results with a single observation. In order to understand the results, we derive the expression for the true posterior state covariance and the posterior covariance using the incorrect decorrelation time-scale. We do this for a Kalman Smoother to avoid the sampling uncertainties. The results are richer than expected and highly dependent on the observation frequency. From the analytical solution of the analysis, we find that the RMSE is a function of both &#969;r&#160;and &#969;g, and the spread or the variance only depends on &#969;g. We also find that the analyzed variance is not always a monotonically increasing function of &#969;g, and it also depends on the observation frequency. In general, the results show the effect of the correlated model error and the incorrect correlation time-scale on data assimilation result, which is also affected by the observation frequency.

Download Full-text