scholarly journals Mapping model behaviour using Self-Organizing Maps

2009 ◽  
Vol 13 (3) ◽  
pp. 395-409 ◽  
Author(s):  
M. Herbst ◽  
H. V. Gupta ◽  
M. C. Casper
Keyword(s):  
Time Series ◽  
Monte Carlo ◽  
Model Performance ◽  
Model Parameters ◽  
Self Organizing Map ◽  
Watershed Model ◽  
Monte Carlo Data ◽  
Statistical Measures ◽  
Trade Offs ◽  
Self Organizing

Abstract. Hydrological model evaluation and identification essentially involves extracting and processing information from model time series. However, the type of information extracted by statistical measures has only very limited meaning because it does not relate to the hydrological context of the data. To overcome this inadequacy we exploit the diagnostic evaluation concept of Signature Indices, in which model performance is measured using theoretically relevant characteristics of system behaviour. In our study, a Self-Organizing Map (SOM) is used to process the Signatures extracted from Monte-Carlo simulations generated by the distributed conceptual watershed model NASIM. The SOM creates a hydrologically interpretable mapping of overall model behaviour, which immediately reveals deficits and trade-offs in the ability of the model to represent the different functional behaviours of the watershed. Further, it facilitates interpretation of the hydrological functions of the model parameters and provides preliminary information regarding their sensitivities. Most notably, we use this mapping to identify the set of model realizations (among the Monte-Carlo data) that most closely approximate the observed discharge time series in terms of the hydrologically relevant characteristics, and to confine the parameter space accordingly. Our results suggest that Signature Index based SOMs could potentially serve as tools for decision makers inasmuch as model realizations with specific Signature properties can be selected according to the purpose of the model application. Moreover, given that the approach helps to represent and analyze multi-dimensional distributions, it could be used to form the basis of an optimization framework that uses SOMs to characterize the model performance response surface. As such it provides a powerful and useful way to conduct model identification and model uncertainty analyses.

scholarly journals Mapping model behaviour using Self-Organizing Maps

2008 ◽  
Vol 5 (6) ◽  
pp. 3517-3555 ◽  
Author(s):  
M. Herbst ◽  
H. V. Gupta ◽  
M. C. Casper
Keyword(s):  
Time Series ◽  
Monte Carlo ◽  
Model Performance ◽  
Model Parameters ◽  
Self Organizing Map ◽  
Watershed Model ◽  
Monte Carlo Data ◽  
Statistical Measures ◽  
Trade Offs ◽  
Self Organizing

Abstract. Hydrological model evaluation and identification essentially depends on the extraction of information from model time series and its processing. However, the type of information extracted by statistical measures has only very limited meaning because it does not relate to the hydrological context of the data. To overcome this inadequacy we exploit the diagnostic evaluation concept of Signature Indices, in which model performance is measured using theoretically relevant characteristics of system behaviour. In our study, a Self-Organizing Map (SOM) is used to process the Signatures extracted from Monte-Carlo simulations generated by a distributed conceptual watershed model. The SOM creates a hydrologically interpretable mapping of overall model behaviour, which immediately reveals deficits and trade-offs in the ability of the model to represent the different functional behaviours of the watershed. Further, it facilitates interpretation of the hydrological functions of the model parameters and provides preliminary information regarding their sensitivities. Most notably, we use this mapping to identify the set of model realizations (among the Monte-Carlo data) that most closely approximate the observed discharge time series in terms of the hydrologically relevant characteristics, and to confine the parameter space accordingly. Our results suggest that Signature Index based SOMs could potentially serve as tools for decision makers inasmuch as model realizations with specific Signature properties can be selected according to the purpose of the model application. Moreover, given that the approach helps to represent and analyze multi-dimensional distributions, it could be used to form the basis of an optimization framework that uses SOMs to characterize the model performance response surface. As such it provides a powerful and useful way to conduct model identification and model uncertainty analyses.

scholarly journals Towards model evaluation and identification using Self-Organizing Maps

2007 ◽  
Vol 4 (6) ◽  
pp. 3953-3978 ◽  
Author(s):  
M. Herbst ◽  
M. C. Casper
Keyword(s):  
Time Series ◽  
Monte Carlo ◽  
Monte Carlo Simulations ◽  
Performance Measures ◽  
Model Identification ◽  
Self Organizing Map ◽  
Watershed Model ◽  
Self Organizing Maps ◽  
Statistical Measures ◽  
Self Organizing

Abstract. The reduction of information contained in model time series through the use of aggregating statistical measures is very high compared to the amount of information that one would like to draw from it for model identification and calibration purposes. Applied within a model identification context, aggregating statistical performance measures are inadequate to capture details on time series characteristics. It has been readily shown that this loss of information on the residuals imposes important limitations on model identification and -diagnostics and thus constitutes an element of the overall model uncertainty. In this contribution we present an approach using a Self-Organizing Map (SOM) to circumvent the identifiability problem induced by the low discriminatory power of aggregating performance measures. Instead, a Self-Organizing Map is used to differentiate the spectrum of model realizations, obtained from Monte-Carlo simulations with a distributed conceptual watershed model, based on the recognition of different patterns in time series. Further, the SOM is used instead of a classical optimization algorithm to identify the model realizations among the Monte-Carlo simulations that most closely approximate the pattern of the measured discharge time series. The results are analyzed and compared with the manually calibrated model as well as with the results of the Shuffled Complex Evolution algorithm (SCE-UA).

scholarly journals Towards model evaluation and identification using Self-Organizing Maps

2008 ◽  
Vol 12 (2) ◽  
pp. 657-667 ◽  
Author(s):  
M. Herbst ◽  
M. C. Casper
Keyword(s):  
Time Series ◽  
Monte Carlo ◽  
Performance Measures ◽  
Model Identification ◽  
Equivalent Model ◽  
Self Organizing Map ◽  
Watershed Model ◽  
Self Organizing Maps ◽  
Data Set ◽  
Self Organizing

Abstract. The reduction of information contained in model time series through the use of aggregating statistical performance measures is very high compared to the amount of information that one would like to draw from it for model identification and calibration purposes. It has been readily shown that this loss imposes important limitations on model identification and -diagnostics and thus constitutes an element of the overall model uncertainty. In this contribution we present an approach using a Self-Organizing Map (SOM) to circumvent the identifiability problem induced by the low discriminatory power of aggregating performance measures. Instead, a Self-Organizing Map is used to differentiate the spectrum of model realizations, obtained from Monte-Carlo simulations with a distributed conceptual watershed model, based on the recognition of different patterns in time series. Further, the SOM is used instead of a classical optimization algorithm to identify those model realizations among the Monte-Carlo simulation results that most closely approximate the pattern of the measured discharge time series. The results are analyzed and compared with the manually calibrated model as well as with the results of the Shuffled Complex Evolution algorithm (SCE-UA). In our study the latter slightly outperformed the SOM results. The SOM method, however, yields a set of equivalent model parameterizations and therefore also allows for confining the parameter space to a region that closely represents a measured data set. This particular feature renders the SOM potentially useful for future model identification applications.

scholarly journals Comparative analysis of model behaviour for flood prediction purposes using Self-Organizing Maps

2009 ◽  
Vol 9 (2) ◽  
pp. 373-392 ◽  
Author(s):  
M. Herbst ◽  
M. C. Casper ◽  
J. Grundmann ◽  
O. Buchholz
Keyword(s):  
Time Series ◽  
Monte Carlo ◽  
Flood Forecasting ◽  
Model Parameters ◽  
Calibration Data ◽  
Flood Prediction ◽  
Watershed Models ◽  
Prediction Problems ◽  
Peak Discharges ◽  
Self Organizing

Abstract. Distributed watershed models constitute a key component in flood forecasting systems. It is widely recognized that models because of their structural differences have varying capabilities of capturing different aspects of the system behaviour equally well. Of course, this also applies to the reproduction of peak discharges by a simulation model which is of particular interest regarding the flood forecasting problem. In our study we use a Self-Organizing Map (SOM) in combination with index measures which are derived from the flow duration curve in order to examine the conditions under which three different distributed watershed models are capable of reproducing flood events present in the calibration data. These indices are specifically conceptualized to extract data on the peak discharge characteristics of model output time series which are obtained from Monte-Carlo simulations with the distributed watershed models NASIM, LARSIM and WaSIM-ETH. The SOM helps to analyze this data by producing a discretized mapping of their distribution in the index space onto a two dimensional plane such that their pattern and consequently the patterns of model behaviour can be conveyed in a comprehensive manner. It is demonstrated how the SOM provides useful information about details of model behaviour and also helps identifying the model parameters that are relevant for the reproduction of peak discharges and thus for flood prediction problems. It is further shown how the SOM can be used to identify those parameter sets from among the Monte-Carlo data that most closely approximate the peak discharges of a measured time series. The results represent the characteristics of the observed time series with partially superior accuracy than the reference simulation obtained by implementing a simple calibration strategy using the global optimization algorithm SCE-UA. The most prominent advantage of using SOM in the context of model analysis is that it allows to comparatively evaluating the data from two or more models. Our results highlight the individuality of the model realizations in terms of the index measures and shed a critical light on the use and implementation of simple and yet too rigorous calibration strategies.

scholarly journals Hydrologic responses of the Zwalm catchment using the REW model: incorporating uncertainty of soil properties

2006 ◽  
Vol 3 (1) ◽  
pp. 69-114 ◽  
Author(s):  
A. El Ouazzani Taibi ◽  
G. P. Zhang ◽  
A. Elfeki
Keyword(s):  
Monte Carlo ◽  
Monte Carlo Method ◽  
Hydraulic Conductivity ◽  
Model Parameters ◽  
Watershed Model ◽  
The Monte Carlo Method ◽  
Physically Based ◽  
Hydrologic Responses ◽  
High Influence ◽  
Model Approach

Abstract. The research presented in this paper focuses on an application of a newly developed physically-based watershed model approach, which is called Representative Elementary Watershed (REW) approach. The study stressed the effects of uncertainty of input parameters on the watershed responses (i.e. simulated discharges). The approach was applied to the Zwalm catchment, which is an agriculture dominated watershed with a drainage area of 114.3 km2 located in East-Flanders, Belgium. Uncertainty analysis of the model parameters is limited to the saturated hydraulic conductivity because of its high influence on the watershed hydrologic behavior. The assessment of outputs uncertainty is performed using the Monte Carlo method. The ensemble statistical watershed responses and their uncertainties are calculated and compared with the measurements. The results show that the measured discharges are falling within the 95% confidence interval of the modeled discharge.

scholarly journals Quantifying location error to define uncertainty in volcanic mass flow hazard simulations

2021 ◽  
Vol 21 (8) ◽  
pp. 2447-2460
Author(s):  
Stuart R. Mead ◽  
Jonathan Procter ◽  
Gabor Kereszturi
Keyword(s):  
Mass Flow ◽  
Performance Metrics ◽  
Numerical Models ◽  
Model Performance ◽  
Flow Simulation ◽  
Model Complexity ◽  
Model Parameters ◽  
Spatial Covariance ◽  
Trade Offs ◽  
Pixel Pair

Abstract. The use of mass flow simulations in volcanic hazard zonation and mapping is often limited by model complexity (i.e. uncertainty in correct values of model parameters), a lack of model uncertainty quantification, and limited approaches to incorporate this uncertainty into hazard maps. When quantified, mass flow simulation errors are typically evaluated on a pixel-pair basis, using the difference between simulated and observed (“actual”) map-cell values to evaluate the performance of a model. However, these comparisons conflate location and quantification errors, neglecting possible spatial autocorrelation of evaluated errors. As a result, model performance assessments typically yield moderate accuracy values. In this paper, similarly moderate accuracy values were found in a performance assessment of three depth-averaged numerical models using the 2012 debris avalanche from the Upper Te Maari crater, Tongariro Volcano, as a benchmark. To provide a fairer assessment of performance and evaluate spatial covariance of errors, we use a fuzzy set approach to indicate the proximity of similarly valued map cells. This “fuzzification” of simulated results yields improvements in targeted performance metrics relative to a length scale parameter at the expense of decreases in opposing metrics (e.g. fewer false negatives result in more false positives) and a reduction in resolution. The use of this approach to generate hazard zones incorporating the identified uncertainty and associated trade-offs is demonstrated and indicates a potential use for informed stakeholders by reducing the complexity of uncertainty estimation and supporting decision-making from simulated data.

scholarly journals Quantifying location error to define uncertainty in volcanic mass flow hazard simulations

2021 ◽  
Author(s):  
Stuart R. Mead ◽  
Jonathan Procter ◽  
Gabor Kereszturi
Keyword(s):  
Mass Flow ◽  
Performance Metrics ◽  
Numerical Models ◽  
Model Performance ◽  
Flow Simulation ◽  
Model Complexity ◽  
Model Parameters ◽  
Spatial Covariance ◽  
Trade Offs ◽  
Pixel Pair

Abstract. The use of mass flow simulations in volcanic hazard zonation and mapping is often limited by model complexity (i.e. uncertainty in correct values of model parameters), a lack of model uncertainty quantification, and limited approaches to incorporate this uncertainty into hazard maps. When quantified, mass flow simulation errors are typically evaluated on a pixel-pair basis, using the difference between simulated and observed (actual) map-cell values to evaluate the performance of a model. However, these comparisons conflate location and quantification errors, neglecting possible spatial autocorrelation of evaluated errors. As a result, model performance assessments typically yield moderate accuracy values. In this paper, similarly moderate accuracy values were found in a performance assessment of three depth-averaged numerical models using the 2012 debris avalanche from the Upper Te Maari crater, Tongariro Volcano as a benchmark. To provide a fairer assessment of performance and evaluate spatial covariance of errors, we use a fuzzy set approach to indicate the proximity of similarly valued map cells. This fuzzification of simulated results yields improvements in targeted performance metrics relative to a length scale parameter, at the expense of decreases in opposing metrics (e.g. less false negatives results in more false positives) and a reduction in resolution. The use of this approach to generate hazard zones incorporating the identified uncertainty and associated trade-offs is demonstrated, and indicates a potential use for informed stakeholders by reducing the complexity of uncertainty estimation and supporting decision making from simulated data.

Sign in / Sign up

Export Citation Format

Share Document