scholarly journals Advanced Performance Metrics and Sensitivity Analysis for Model Validation and Calibration

2020 ◽  
Author(s):  
Urmila Agrawal ◽  
Pavel Etingov ◽  
Renke Huang
Keyword(s):  
Sensitivity Analysis ◽  
Power Systems ◽  
Model Validation ◽  
Real World ◽  
Model Calibration ◽  
Dynamic Models ◽  
Performance Metrics ◽  
Step Response ◽  
Response Characteristics ◽  
Oscillatory Response

<pre>High quality generator dynamic models are critical to reliable and accurate power systems studies and planning. With the availability of PMU measurements, measurement-based approach for model validation has gained significant prominence. Currently, the model validation results are analyzed by visually comparing real--world PMU measurements with the model-based response measurements, and parameter adjustments rely mostly on engineering experience. This paper proposes advanced performance metrics to systematically quantify the generator dynamic model validation results by separately taking into consideration slow governor response and comparatively fast oscillatory response. The performance metric for governor response is based on the step response characteristics of a system and the metric for oscillatory response is based on the response of generator to each system mode calculated using modal analysis. The proposed metrics in this paper is aimed at providing critical information to help with the selection of parameters to be tuned for model calibration by performing enhanced sensitivity analysis, and also help with rule-based model calibration. Results obtained using both simulated and real-world measurements validate the effectiveness of the proposed performance metrics and sensitivity analysis for model validation and calibration.</pre>

scholarly journals Advanced Performance Metrics and their Application to the Sensitivity Analysis for Model Validation and Calibration

2021 ◽  
Author(s):  
Urmila Agrawal ◽  
Pavel Etingov ◽  
Renke Huang
Keyword(s):  
Sensitivity Analysis ◽  
Power Systems ◽  
Model Validation ◽  
Real World ◽  
Model Calibration ◽  
Dynamic Models ◽  
Performance Metrics ◽  
Step Response ◽  
Response Characteristics ◽  
Oscillatory Response

<pre>High quality generator dynamic models are critical to reliable and accurate power systems studies and planning. With the availability of PMU measurements, measurement-based approach for model validation has gained significant prominence. Currently, the model validation results are analyzed by visually comparing real--world PMU measurements with the model-based response measurements, and parameter adjustments rely mostly on engineering experience. This paper proposes advanced performance metrics to systematically quantify the generator dynamic model validation results by separately taking into consideration slow governor response and comparatively fast oscillatory response. The performance metric for governor response is based on the step response characteristics of a system and the metric for oscillatory response is based on the response of generator to each system mode calculated using modal analysis. The proposed metrics in this paper is aimed at providing critical information to help with the selection of parameters to be tuned for model calibration by performing enhanced sensitivity analysis, and also help with rule-based model calibration. Results obtained using both simulated and real-world measurements validate the effectiveness of the proposed performance metrics and sensitivity analysis for model validation and calibration.</pre>

scholarly journals Advanced Performance Metrics and Sensitivity Analysis for Model Validation and Calibration

2020 ◽  
Author(s):  
Urmila Agrawal ◽  
Pavel Etingov ◽  
Renke Huang
Keyword(s):  
Sensitivity Analysis ◽  
Power Systems ◽  
Model Validation ◽  
Real World ◽  
Model Calibration ◽  
Dynamic Models ◽  
Performance Metrics ◽  
Step Response ◽  
Response Characteristics ◽  
Oscillatory Response

<pre>High quality generator dynamic models are critical to reliable and accurate power systems studies and planning. With the availability of PMU measurements, measurement-based approach for model validation has gained significant prominence. Currently, the model validation results are analyzed by visually comparing real--world PMU measurements with the model-based response measurements, and parameter adjustments rely mostly on engineering experience. This paper proposes advanced performance metrics to systematically quantify the generator dynamic model validation results by separately taking into consideration slow governor response and comparatively fast oscillatory response. The performance metric for governor response is based on the step response characteristics of a system and the metric for oscillatory response is based on the response of generator to each system mode calculated using modal analysis. The proposed metrics in this paper is aimed at providing critical information to help with the selection of parameters to be tuned for model calibration by performing enhanced sensitivity analysis, and also help with rule-based model calibration. Results obtained using both simulated and real-world measurements validate the effectiveness of the proposed performance metrics and sensitivity analysis for model validation and calibration.</pre>

scholarly journals Initial Results of Quantification of Model Validation Results Using Modal Analysis

2020 ◽  
Author(s):  
Urmila Agrawal ◽  
Pavel Etingov ◽  
Renke Huang
Keyword(s):  
Power Systems ◽  
Modal Analysis ◽  
Model Validation ◽  
Real World ◽  
Dynamic Models ◽  
Simulated Data ◽  
Real World Data ◽  
Prony Method ◽  
System Mode ◽  
Initial Results

<div>High quality generator dynamic models are critical to reliable and accurate power systems studies and planning. With the availability of PMU measurements, measurement-based approach for model validation has gained significant prominence. Currently, the model validation results are analyzed by visually comparing real–world PMU measurements with the model-based simulated data. This paper proposes metrics to quantify the generator dynamic model validation results based on the response of generators to each system mode, which includes both local and inter-area, using modal analysis approach. The metrics provide information on the inaccuracy associated with the model in terms of the characteristics of each mode. Initial results obtained using the real-world data validates the effectiveness of the proposed metrics. In this paper, modal analysis was carried out using Prony method.</div>

scholarly journals Initial Results of Quantification of Model Validation Results Using Modal Analysis

2020 ◽  
Author(s):  
Urmila Agrawal ◽  
Pavel Etingov ◽  
Renke Huang
Keyword(s):  
Power Systems ◽  
Modal Analysis ◽  
Model Validation ◽  
Real World ◽  
Dynamic Models ◽  
Simulated Data ◽  
Real World Data ◽  
Prony Method ◽  
System Mode ◽  
Initial Results

<div>High quality generator dynamic models are critical to reliable and accurate power systems studies and planning. With the availability of PMU measurements, measurement-based approach for model validation has gained significant prominence. Currently, the model validation results are analyzed by visually comparing real–world PMU measurements with the model-based simulated data. This paper proposes metrics to quantify the generator dynamic model validation results based on the response of generators to each system mode, which includes both local and inter-area, using modal analysis approach. The metrics provide information on the inaccuracy associated with the model in terms of the characteristics of each mode. Initial results obtained using the real-world data validates the effectiveness of the proposed metrics. In this paper, modal analysis was carried out using Prony method.</div>

scholarly journals Budget Impact Analysis of Diabetes Drugs: A Systematic Literature Review

2021 ◽  
Vol 9 ◽  
Author(s):  
Zejun Luo ◽  
Zhen Ruan ◽  
Dongning Yao ◽  
Carolina Oi Lam Ung ◽  
Yunfeng Lai ◽  
...  
Keyword(s):  
Sensitivity Analysis ◽  
Model Validation ◽  
Practice Guidelines ◽  
Real World ◽  
Impact Analysis ◽  
Budget Impact ◽  
Good Practice ◽  
Budget Impact Analysis ◽  
Real World Data ◽  
World Data

Background: Budget impact analysis (BIA) is an economic assessment that estimates the financial consequences of adopting a new intervention. BIA is used to make informed reimbursement decisions, as a supplement to cost-effectiveness analyses (CEAs).Objectives: We systematically reviewed BIA studies associated with anti-diabetic drugs and assessed the extent to which international BIA guidelines were followed in these studies.Methods: We conducted a literature search on PubMed, Web of Science, Econlit, Medline, China National Knowledge Infrastructure (CNKI), Wanfang Data knowledge Service platform from database inception to June 30, 2021. ISPOR good practice guidelines were used as a methodological standard for assessing BIAs. We extracted and compared the study characteristics outlined by the ISPOR BIA Task Force to evaluate the guideline compliance of the included BIA.Results: A total of eighteen studies on the BIA for anti-diabetic drugs were identified. More than half studies were from developed countries. Seventeen studies were based on model and one study was based on real-world data. Overall, analysis considered a payer perspective, reported potential budget impacts over 1–5 years. Assumptions were mainly made about target population size, market share uptake of new interventions, and scope of cost. The data used for analysis varied among studies and was rarely justified. Model validation and sensitivity analysis were lacking in the current BIA studies. Rebate analysis was conducted in a few studies to explore the price discount that was required for new interventions to demonstrate cost equivalence to comparators.Conclusion: Existing studies evaluating budget impact for anti-diabetic drugs vary greatly in methodology, some of which showed low compliance to good practice guidelines. In order for the BIA to be useful for assisting in health plan decision-making, it is important for future studies to optimize compliance to national or ISPOR good practice guidelines on BIA. Model validation and sensitivity analysis should also be improved in future BIA studies. Continued improvement of BIA using real-world data is necessary to ensure high-quality analyses and to provide reliable results.

Model Validation

2009 ◽  
pp. 231-241
Author(s):  
Ben Kei Daniel
Keyword(s):  
Social Capital ◽  
Sensitivity Analysis ◽  
Model Validation ◽  
Real World ◽  
Bayesian Model ◽  
Computational Models ◽  
Lessons Learned ◽  
Practical Application ◽  
Validation Process ◽  
The World

Though computational models take a lot of effort to build, a model is generally not useful unless it can help people to understand the world being modelled, or the problem the model is intended to solve. A useful model allows people to make useful predictions about how the world will behave now and possibly tomorrow. Validation is the last step required in developing a useful Bayesian model. The goal of validation is to gain confidence in a model and to demonstrate and prove that a model produces reliable results that are closely related to the problems or issues in which the model is intended to address. The goal of the Chapter is to provide the reader with a basic understanding of the validation process and to share with them key lessons learned from the model of social capital presented in the book. While sensitivity analysis is intended to ensure that a Bayesian model is theoretically consistent with goals and assumptions of the modeller (how the modeller views the world) or the accuracy of sources of data used for building the model, the goal of validation is to demonstrate the practical application of the model in real world settings. This Chapter presents the main steps involved in the process of validating a Bayesian model. It illustrates this process by using examples drawn from the Bayesian model of social capital.

scholarly journals Modeling Diagnostic Strategies to Manage Toxic Adverse Events following Cancer Immunotherapy

2021 ◽  
pp. 0272989X2110027
Author(s):  
Frederik van Delft ◽  
Mirte Muller ◽  
Rom Langerak ◽  
Hendrik Koffijberg ◽  
Valesca Retèl ◽  
...  
Keyword(s):  
Palliative Care ◽  
Adverse Events ◽  
Scenario Analysis ◽  
Real World ◽  
Model Calibration ◽  
Positive Test ◽  
Test Accuracy ◽  
Test Sensitivity ◽  
False Positive Test ◽  
Test Outcomes

Background Although immunotherapy (IMT) provides significant survival benefits in selected patients, approximately 10% of patients experience (serious) immune-related adverse events (irAEs). The early detection of adverse events will prevent irAEs from progressing to severe stages, and routine testing for irAEs has become common practice. Because a positive test outcome might indicate a clinically manifesting irAE that requires treatment to (temporarily) discontinue, the occurrence of false-positive test outcomes is expected to negatively affect treatment outcomes. This study explores how the UPPAAL modeling environment can be used to assess the impact of test accuracy (i.e., test sensitivity and specificity), on the probability of patients entering palliative care within 11 IMT cycles. Methods A timed automata-based model was constructed using real-world data and expert consultation. Model calibration was performed using data from 248 non–small-cell lung cancer patients treated with nivolumab. A scenario analysis was performed to evaluate the effect of changes in test accuracy on the probability of patients transitioning to palliative care. Results The constructed model was used to estimate the cumulative probabilities for the patients’ transition to palliative care, which were found to match real-world clinical observations after model calibration. The scenario analysis showed that the specificity of laboratory tests for routine monitoring has a strong effect on the probability of patients transitioning to palliative care, whereas the effect of test sensitivity was limited. Conclusion We have obtained interesting insights by simulating a care pathway and disease progression using UPPAAL. The scenario analysis indicates that an increase in test specificity results in decreased discontinuation of treatment due to suspicion of irAEs, through a reduction of false-positive test outcomes.

Sign in / Sign up

Export Citation Format

Share Document