Sensitivity analysis of infectious disease models: methods, advances and their application

Sensitivity analysis (SA) can aid in identifying influential model parameters and optimizing model structure, yet infectious disease modelling has yet to adopt advanced SA techniques that are capable of providing considerable insights over traditional methods. We investigate five global SA methods—scatter plots, the Morris and Sobol’ methods, Latin hypercube sampling-partial rank correlation coefficient and the sensitivity heat map method—and detail their relative merits and pitfalls when applied to a microparasite (cholera) and macroparasite (schistosomaisis) transmission model. The methods investigated yielded similar results with respect to identifying influential parameters, but offered specific insights that vary by method. The classical methods differed in their ability to provide information on the quantitative relationship between parameters and model output, particularly over time. The heat map approach provides information about the group sensitivity of all model state variables, and the parameter sensitivity spectrum obtained using this method reveals the sensitivity of all state variables to each parameter over the course of the simulation period, especially valuable for expressing the dynamic sensitivity of a microparasite epidemic model to its parameters. A summary comparison is presented to aid infectious disease modellers in selecting appropriate methods, with the goal of improving model performance and design.

Download Full-text

Understanding transmission and control of the pork tapeworm with CystiAgent: a spatially explicit agent-based model

10.21203/rs.2.22078/v2 ◽

2020 ◽

Author(s):

Ian Wright Pray ◽

Wayne Wakeland ◽

William Pan ◽

William E. Lambert ◽

Hector H. Garcia ◽

...

Keyword(s):

Taenia Solium ◽

Model Performance ◽

Rank Correlation ◽

Public Health Problem ◽

Sensitivity Analyses ◽

Transmission Model ◽

Spatially Explicit ◽

Model Parameters ◽

Agent Based Model ◽

Agent Based

Abstract Background The pork tapeworm ( Taenia solium ) is a serious public health problem in rural low-resource areas of Latin America, Africa, and Asia, where the associated conditions of nuerocysticercosis (NCC) and porcine cysticercosis cause substantial health and economic harms. An accurate and validated transmission model for T. solium would serve as an important new tool for control and elimination, as it would allow for comparison of available intervention strategies, and prioritization of the most effective strategies for control and elimination efforts. Methods We developed a spatially-explicit agent-based model (ABM) for T. solium (“CystiAgent”) that differs from prior T. solium models by including a spatial framework and behavioral parameters such as pig roaming, open human defecation, and human travel. In this article, we introduce the structure and function of the model, describe the data sources used to parameterize the model, and apply sensitivity analyses (Latin hypercube sampling–partial rank correlation coefficient (LHS-PRCC)) to evaluate model parameters. Results LHS-PRCC analysis of CystiAgent found that the parameters with the greatest impact on model uncertainty were the roaming range of pigs, the infectious duration of human taeniasis, use of latrines, and the set of “tuning” parameters defining the probabilities of infection in humans and pigs given exposure to T. solium. Conclusions CystiAgent is a novel ABM that has the ability to model spatial and behavioral features of T. solium transmission not available in other models. There is a small set of impactful model parameters that contribute uncertainty to the model and may impact the accuracy of model projections. Field and laboratory studies to better understand these key components of transmission may help reduce uncertainty, while current applications of CystiAgent may consider calibration of these parameters to improve model performance. These results will ultimately allow for improved interpretation of model validation results, and usage of the model to compare available control and elimination strategies for T. solium .

Download Full-text

Understanding transmission and control of the pork tapeworm with CystiAgent: a spatially explicit agent-based model

10.21203/rs.2.22078/v1 ◽

2020 ◽

Author(s):

Ian Wright Pray ◽

Wayne Wakeland ◽

William Pan ◽

William E. Lambert ◽

Hector H. Garcia ◽

...

Keyword(s):

Taenia Solium ◽

Model Performance ◽

Rank Correlation ◽

Public Health Problem ◽

Sensitivity Analyses ◽

Transmission Model ◽

Spatially Explicit ◽

Model Parameters ◽

Agent Based Model ◽

Agent Based

Abstract BackgroundThe pork tapeworm (Taenia solium) is a serious public health problem in rural low-resource areas of Latin America, Africa, and Asia, where the associated conditions of nuerocysticercosis (NCC) and porcine cysticercosis cause substantial health and economic harms. An accurate and validated transmission model for T. solium would serve as an important new tool for control and elimination, as it would allow for comparison of available intervention strategies, and prioritization of the most effective strategies for control and elimination efforts. MethodsWe developed a spatially-explicit agent-based model (ABM) for T. solium (“CystiAgent”) that differs from prior T. solium models by including a spatial framework and behavioral parameters such as pig roaming, open human defecation, and human travel. In this article, we introduce the structure and function of the model, describe the data sources used to parameterize the model, and apply sensitivity analyses (Latin hypercube sampling–partial rank correlation coefficient (LHS-PRCC)) to evaluate model parameters. ResultsLHS-PRCC analysis of CystiAgent found that the parameters with the greatest impact on model uncertainty were the roaming range of pigs, the infectious duration of human taeniasis, use of latrines, and the set of “tuning” parameters defining the probabilities of infection in humans and pigs given exposure to T. solium.ConclusionsCystiAgent is a novel ABM that has the ability to model spatial and behavioral features of T. solium transmission not available in other models. There is a small set of impactful model parameters that contribute uncertainty to the model and may impact the accuracy of model projections. Field and laboratory studies to better understand these key components of transmission may help reduce uncertainty, while current applications of CystiAgent may consider calibration of these parameters to improve model performance. These results will ultimately allow for improved interpretation of model validation results, and usage of the model to compare available control and elimination strategies for T. solium.

Download Full-text

Understanding transmission and control of the pork tapeworm with CystiAgent: a spatially explicit agent-based model

10.21203/rs.2.22078/v3 ◽

2020 ◽

Author(s):

Ian Wright Pray ◽

Wayne Wakeland ◽

William Pan ◽

William E. Lambert ◽

Hector H. Garcia ◽

...

Keyword(s):

Taenia Solium ◽

Model Performance ◽

Rank Correlation ◽

Public Health Problem ◽

Sensitivity Analyses ◽

Transmission Model ◽

Spatially Explicit ◽

Model Parameters ◽

Agent Based Model ◽

Agent Based

Abstract Background The pork tapeworm (Taenia solium) is a serious public health problem in rural low-resource areas of Latin America, Africa, and Asia, where the associated conditions of nuerocysticercosis (NCC) and porcine cysticercosis cause substantial health and economic harms. An accurate and validated transmission model for T. solium would serve as an important new tool for control and elimination, as it would allow for comparison of available intervention strategies, and prioritization of the most effective strategies for control and elimination efforts. Methods We developed a spatially-explicit agent-based model (ABM) for T. solium (“CystiAgent”) that differs from prior T. solium models by including a spatial framework and behavioral parameters such as pig roaming, open human defecation, and human travel. In this article, we introduce the structure and function of the model, describe the data sources used to parameterize the model, and apply sensitivity analyses (Latin hypercube sampling–partial rank correlation coefficient (LHS-PRCC)) to evaluate model parameters. Results LHS-PRCC analysis of CystiAgent found that the parameters with the greatest impact on model uncertainty were the roaming range of pigs, the infectious duration of human taeniasis, use of latrines, and the set of “tuning” parameters defining the probabilities of infection in humans and pigs given exposure to T. solium.Conclusions CystiAgent is a novel ABM that has the ability to model spatial and behavioral features of T. solium transmission not available in other models. There is a small set of impactful model parameters that contribute uncertainty to the model and may impact the accuracy of model projections. Field and laboratory studies to better understand these key components of transmission may help reduce uncertainty, while current applications of CystiAgent may consider calibration of these parameters to improve model performance. These results will ultimately allow for improved interpretation of model validation results, and usage of the model to compare available control and elimination strategies for T. solium.

Download Full-text

Robust estimation of hydrological model parameters

Hydrology and Earth System Sciences Discussions ◽

10.5194/hessd-5-1641-2008 ◽

2008 ◽

Vol 5 (3) ◽

pp. 1641-1675 ◽

Cited By ~ 12

Author(s):

A. Bárdossy ◽

S. K. Singh

Keyword(s):

Measurement Errors ◽

Input Data ◽

Hydrological Model ◽

Optimal Parameter ◽

Model Performance ◽

Model Parameters ◽

Geometrical Approach ◽

State Variables ◽

Parameter Vector ◽

Robust Parameter

Abstract. The estimation of hydrological model parameters is a challenging task. With increasing capacity of computational power several complex optimization algorithms have emerged, but none of the algorithms gives an unique and very best parameter vector. The parameters of hydrological models depend upon the input data. The quality of input data cannot be assured as there may be measurement errors for both input and state variables. In this study a methodology has been developed to find a set of robust parameter vectors for a hydrological model. To see the effect of observational error on parameters, stochastically generated synthetic measurement errors were applied to observed discharge and temperature data. With this modified data, the model was calibrated and the effect of measurement errors on parameters was analysed. It was found that the measurement errors have a significant effect on the best performing parameter vector. The erroneous data led to very different optimal parameter vectors. To overcome this problem and to find a set of robust parameter vectors, a geometrical approach based on the half space depth was used. The depth of the set of N randomly generated parameters was calculated with respect to the set with the best model performance (Nash-Sutclife efficiency was used for this study) for each parameter vector. Based on the depth of parameter vectors, one can find a set of robust parameter vectors. The results show that the parameters chosen according to the above criteria have low sensitivity and perform well when transfered to a different time period. The method is demonstrated on the upper Neckar catchment in Germany. The conceptual HBV model was used for this study.

Download Full-text

Comparison between dynamic and static sensitivity analysis approaches for impact assessment of different potential evapotranspiration methods on hydrological models performance

Journal of Hydrometeorology ◽

10.1175/jhm-d-20-0192.1 ◽

2021 ◽

Author(s):

Rodric Mérimé Nonki ◽

André Lenouo ◽

Christopher J. Lennard ◽

Raphael M. Tshimanga ◽

Clément Tchawoua

Keyword(s):

Sensitivity Analysis ◽

Impact Assessment ◽

Hydrological Modeling ◽

Potential Evapotranspiration ◽

Model Performance ◽

Estimation Methods ◽

Model Parameters ◽

Hydrological Models ◽

Sensitivity Approach ◽

Static Sensitivity

AbstractPotential Evapotranspiration (PET) plays a crucial role in water management, including irrigation systems design and management. It is an essential input to hydrological models. Direct measurement of PET is difficult, time-consuming and costly, therefore a number of different methods are used to compute this variable. This study compares the two sensitivity analysis approaches generally used for PET impact assessment on hydrological model performance. We conducted the study in the Upper Benue River Basin (UBRB) located in northern Cameroon using two lumped-conceptual rainfall-runoff models and nineteen PET estimation methods. A Monte-Carlo procedure was implemented to calibrate the hydrological models for each PET input while considering similar objective functions. Although there were notable differences between PET estimation methods, the hydrological models performance was satisfactory for each PET input in the calibration and validation periods. The optimized model parameters were significantly affected by the PET-inputs, especially the parameter responsible to transform PET into actual ET. The hydrological models performance was insensitive to the PET input using a dynamic sensitivity approach, while he was significantly affected using a static sensitivity approach. This means that the over-or under-estimation of PET is compensated by the model parameters during the model recalibration. The model performance was insensitive to the rescaling PET input for both dynamic and static sensitivities approaches. These results demonstrate that the effect of PET input to model performance is necessarily dependent on the sensitivity analysis approach used and suggest that the dynamic approach is more effective for hydrological modeling perspectives.

Download Full-text

Modeling bronchial circulation with application to soluble gas exchange: description and sensitivity analysis

Journal of Applied Physiology ◽

10.1152/jappl.1998.84.6.2070 ◽

1998 ◽

Vol 84 (6) ◽

pp. 2070-2088 ◽

Cited By ~ 25

Author(s):

Thien D. Bui ◽

Donald Dabdub ◽

Steven C. George

Keyword(s):

Sensitivity Analysis ◽

Gas Exchange ◽

Pulmonary Circulation ◽

Blood Flow Rate ◽

Latin Hypercube Sampling ◽

Inert Gases ◽

Phase Iii ◽

Model Parameters ◽

Bronchial Circulation ◽

Wide Range

The steady-state exchange of inert gases across an in situ canine trachea has recently been shown to be limited equally by diffusion and perfusion over a wide range (0.01–350) of blood solubilities (βblood; ml ⋅ ml−1 ⋅ atm−1). Hence, we hypothesize that the exchange of ethanol (βblood = 1,756 at 37°C) in the airways depends on the blood flow rate from the bronchial circulation. To test this hypothesis, the dynamics of the bronchial circulation were incorporated into an existing model that describes the simultaneous exchange of heat, water, and a soluble gas in the airways. A detailed sensitivity analysis of key model parameters was performed by using the method of Latin hypercube sampling. The model accurately predicted a previously reported experimental exhalation profile of ethanol ( R 2= 0.991) as well as the end-exhalation airstream temperature (34.6°C). The model predicts that 27, 29, and 44% of exhaled ethanol in a single exhalation are derived from the tissues of the mucosa and submucosa, the bronchial circulation, and the tissue exterior to the submucosa (which would include the pulmonary circulation), respectively. Although the concentration of ethanol in the bronchial capillary decreased during inspiration, the three key model outputs (end-exhaled ethanol concentration, the slope of phase III, and end-exhaled temperature) were all statistically insensitive ( P > 0.05) to the parameters describing the bronchial circulation. In contrast, the model outputs were all sensitive ( P < 0.05) to the thickness of tissue separating the core body conditions from the bronchial smooth muscle. We conclude that both the bronchial circulation and the pulmonary circulation impact soluble gas exchange when the entire conducting airway tree is considered.

Download Full-text

Improved dynamic simulation technique for hydrodynamics and water quality of river-connected lakes

Water Science & Technology Water Supply ◽

10.2166/ws.2020.125 ◽

2020 ◽

Vol 20 (8) ◽

pp. 3752-3767

Author(s):

Bojun Liu ◽

Jun Xia ◽

Libin Yang ◽

Changyong Cui ◽

Linwei Wang ◽

...

Keyword(s):

Water Quality ◽

Latin Hypercube Sampling ◽

Resistance Coefficient ◽

Water Quality Model ◽

Roughness Coefficient ◽

Model Parameters ◽

State Variables ◽

Quality Model ◽

Rank Regression ◽

Hongze Lake

Abstract In this study, a two-dimensional hydrodynamic water-quality model is proposed for river-connected lakes in an effort to improve calibration accuracy and reduce computational burden. To achieve this, the sensitivity of parameters involved in the hydrodynamic model is analyzed using stepwise rank regression and Latin hypercube sampling (LHS), and the roughness coefficient, wind drag coefficient and wind resistance coefficient are identified as the most important parameters affecting the hydrodynamics of the Hongze Lake. Then, the ensemble Kalman filter (EnKF) is used to assimilate observations to the proposed hydrodynamic and water quality model. It is found that assimilation of both state variables and model parameters results in a significant improvement of the simulation of the water level, flow velocity and pollutant concentration in the Hongze Lake.

Download Full-text

A Sensitivity Analysis of Two Mesoscale Models: COAMPS and WRF

Monthly Weather Review ◽

10.1175/mwr-d-19-0271.1 ◽

2020 ◽

Vol 148 (7) ◽

pp. 2997-3014

Author(s):

Caren Marzban ◽

Robert Tardif ◽

Scott Sandgathe

Keyword(s):

Sensitivity Analysis ◽

Spatial Structure ◽

Latin Hypercube Sampling ◽

Forecast Model ◽

The United States ◽

Model Parameters ◽

Analysis Methodology ◽

Trigger Function ◽

Multivariate Multiple Regression

Abstract A sensitivity analysis methodology recently developed by the authors is applied to COAMPS and WRF. The method involves varying model parameters according to Latin Hypercube Sampling, and developing multivariate multiple regression models that map the model parameters to forecasts over a spatial domain. The regression coefficients and p values testing whether the coefficients are zero serve as measures of sensitivity of forecasts with respect to model parameters. Nine model parameters are selected from COAMPS and WRF, and their impact is examined on nine forecast quantities (water vapor, convective and gridscale precipitation, and air temperature and wind speed at three altitudes). Although the conclusions depend on the model parameters and specific forecast quantities, it is shown that sensitivity to model parameters is often accompanied by nontrivial spatial structure, which itself depends on the underlying forecast model (i.e., COAMPS vs WRF). One specific difference between these models is in their sensitivity with respect to a parameter that controls temperature increments in the Kain–Fritsch trigger function; whereas this parameter has a distinct spatial structure in COAMPS, that structure is completely absent in WRF. The differences between COAMPS and WRF also extend to the quality of the statistical models used to assess sensitivity; specifically, the differences are largest over the waters off the southeastern coast of the United States. The implication of these findings is twofold: not only is the spatial structure of sensitivities different between COAMPS and WRF, the underlying relationship between the model parameters and the forecasts is also different between the two models.

Download Full-text

Optimality-based non-Redfield plankton–ecosystem model (OPEM v1.1) in UVic-ESCM 2.9 – Part 2: Sensitivity analysis and model calibration

Geoscientific Model Development ◽

10.5194/gmd-13-4691-2020 ◽

2020 ◽

Vol 13 (10) ◽

pp. 4691-4712

Author(s):

Chia-Te Chien ◽

Markus Pahlow ◽

Markus Schartau ◽

Andreas Oschlies

Keyword(s):

Sensitivity Analysis ◽

Climate Model ◽

Driving Forces ◽

Ingestion Rate ◽

Latin Hypercube Sampling ◽

Ecosystem Model ◽

Model Parameters ◽

Model Assessment ◽

Model Solutions ◽

Elemental Stoichiometry

Abstract. We analyse 400 perturbed-parameter simulations for two configurations of an optimality-based plankton–ecosystem model (OPEM), implemented in the University of Victoria Earth System Climate Model (UVic-ESCM), using a Latin hypercube sampling method for setting up the parameter ensemble. A likelihood-based metric is introduced for model assessment and selection of the model solutions closest to observed distributions of NO3-, PO43-, O2, and surface chlorophyll a concentrations. The simulations closest to the data with respect to our metric exhibit very low rates of global N2 fixation and denitrification, indicating that in order to achieve rates consistent with independent estimates, additional constraints have to be applied in the calibration process. For identifying the reference parameter sets, we therefore also consider the model's ability to represent current estimates of water-column denitrification. We employ our ensemble of model solutions in a sensitivity analysis to gain insights into the importance and role of individual model parameters as well as correlations between various biogeochemical processes and tracers, such as POC export and the NO3- inventory. Global O2 varies by a factor of 2 and NO3- by more than a factor of 6 among all simulations. Remineralisation rate is the most important parameter for O2, which is also affected by the subsistence N quota of ordinary phytoplankton (Q0,phyN) and zooplankton maximum specific ingestion rate. Q0,phyN is revealed as a major determinant of the oceanic NO3- pool. This indicates that unravelling the driving forces of variations in phytoplankton physiology and elemental stoichiometry, which are tightly linked via Q0,phyN, is a prerequisite for understanding the marine nitrogen inventory.

Download Full-text

Uncertainty and Sensitivity Analysis of the In-Vessel Hydrogen Generation for Gen-III PWR and Phebus FPT-1 with MELCOR 2.2

Energies ◽

10.3390/en14164884 ◽

2021 ◽

Vol 14 (16) ◽

pp. 4884

Author(s):

Piotr Darnowski ◽

Piotr Mazgaj ◽

Mateusz Włostowski

Keyword(s):

Sensitivity Analysis ◽

Hydrogen Generation ◽

Rank Correlation ◽

Correlation Coefficients ◽

Latin Hypercube Sampling ◽

Severe Accident ◽

Sensitivity Analyses ◽

Pressurized Water ◽

Water Reactor ◽

Best Estimate

In this study, uncertainty and sensitivity analyses were performed with MELCOR 2.2.18 to study the hydrogen generation (figure-of-merit (FoM)) during the in-vessel phase of a severe accident in a light water reactor. The focus of this work was laid on a large generation-III pressurized water reactor (PWR) and a double-ended hot leg (HL) large break loss of coolant accident (LB-LOCA) without a safety injection (SI). The FPT-1 Phebus integral experiment emulating LOCA was studied, where the experiment outcomes were applied for the plant scale modelling. The best estimate calculations were supplemented with an uncertainty analysis (UA) based on 400 input-decks and Latin hypercube sampling (LHS). Additionally, the sensitivity analysis (SA) utilizing the linear regression and linear and rank correlation coefficients was performed. The study was prepared with a new open-source MELCOR sensitivity and uncertainty tool (MelSUA), which was supplemented with this work. The FPT-1 best-estimate model results were within the 10% experimental uncertainty band for the final FoM. It was shown that the hydrogen generation uncertainties in PWR were similar to the FPT-1, with the 95% percentile being covered inside a ~50% band and the 50% percentile inside a ~25% band around the FoM median. Two different power profiles for PWR were compared, indicating its impact on the uncertainty but also on the sensitivity results. Despite a similar setup, different uncertainty parameters impacted FoM, showing the difference between scales but also a significant impact of boundary conditions on the sensitivity analysis.

Download Full-text