Technical note: RAT – a robustness assessment test for calibrated and uncalibrated hydrological models

Prior to their use under future changing climate conditions, all hydrological models should be thoroughly evaluated regarding their temporal transferability (application in different time periods) and extrapolation capacity (application beyond the range of known past conditions). This note presents a straightforward evaluation framework aimed at detecting potential undesirable climate dependencies in hydrological models: the robustness assessment test (RAT). Although it is conceptually inspired by the classic differential split-sample test of Klemeš (1986), the RAT presents the advantage of being applicable to all types of models, be they calibrated or not (i.e. regionalized or physically based). In this note, we present the RAT, illustrate its application on a set of 21 catchments, verify its applicability hypotheses and compare it to previously published tests. Results show that the RAT is an efficient evaluation approach, passing it successfully can be considered a prerequisite for any hydrological model to be used for climate change impact studies.

LIMITATIONS OF DESIGN SPACE-BASED INDICATORS FOR EARLY ROBUSTNESS ASSESSMENT

This paper seeks to address the gap between qualitative Robust Design principles and parameter optimization. The former often fails to consider the challenging amount of details in embodiment and configuration design, while the latter is the widely accepted main thrust in traditional Robust Design. The gap is addressed by exploring the value of five quantitative robustness indicators for Design Space Exploration based on variables, objectives and constraints: The set level indicators, Design Space Size and Pareto Set Dispersion, and the point level indicators, Neighbourhood Performance, Failure Rate and Distance to Failure. As a background for the discussion of the limitations of these indicators an industrial case is presented. The case is an incremental encoder and includes two configurations for comparison, five objectives, eight variables, and a range of constraints. The design spaces are sampled and they show conflicting objectives, dispersed spaces and variables dependencies. Based on this it is suggested that set level indicators are more suitable than point level indicators of early robustness evaluation, but the available indicators are limited in their considerations of design space discontinuity and conflicts.