How Uncertain Is Too Uncertain? Validity Tests for Early Resilient and Risk-Based Design Processes

A number of risk and resilience-based design methods have been put forward over the years that seek to provide designers the tools to reduce the effects of potential hazards in the early design phase. However, because of the associated high level of uncertainty and low-fidelity design representations, one might justifiably wonder if using a resilient design process in the early design phase will reliably produce useful results that would improve the realized design. This paper provides a testing framework for design processes that determines the validity of the process by quantifying the epistemic uncertainty in the assumptions used to make decisions. This framework uses this quantified uncertainty to test whether three metrics are within desirable bounds: the change in the design when uncertainty is considered, the increase in the expected value of the design, and the cost of choice-related uncertainty. This approach is illustrated using two examples to demonstrate how both discrete and continuous parametric uncertainty can be considered in the testing procedure. These examples show that early design process validity is sensitive to the level of uncertainty and magnitude of design changes, suggesting that while there is a justifiable decision-theoretic case to consider high-level, high-impact design changes during the early design phase, there is less of a case to choose between relatively similar design options because the cost of making the choice under high uncertainty is greater than the expected value improvement from choosing the better design.