Financially Motivated Model Performance Measures

2004 ◽  
Author(s):  
Craig A. Friedman ◽  
Sven Sandow
Keyword(s):  
Performance Measures ◽  
Model Performance

Modeling the performance measures of world class manufacturing using interpreting structural modeling

2015 ◽  
Vol 10 (1) ◽  
pp. 4-22 ◽  
Author(s):  
Abhijeet Keshaorao Digalwar ◽  
Anil Jindal ◽  
Kuldip Singh Sangwan
Keyword(s):  
Performance Measures ◽  
Model Performance ◽  
Structural Modeling ◽  
Decision Makers ◽  
Interpretive Structural Modeling ◽  
Content Type ◽  
Practical Applications ◽  
Manufacturing Organizations ◽  
World Class ◽  
World Class Manufacturing

Purpose – The purpose of this paper is to study the performance measures of world class manufacturing (WCM) and to establish relationship among them using interpretive structural modeling (ISM). Design/methodology/approach – The research paper presents a blend of theoretical framework and practical applications. In the paper, 16 performance measures are identified from literature survey and experts’ opinion, and then these are validated by questionnaire survey in India. Finally, ISM is used to obtain structural relationship among these performance measures of WCM. Findings – The results of the survey and the ISM methodology have been used to evolve the mutual relationships among these performance measures. Practical implications – The adoption of such an ISM-based model on WCM performance measures in manufacturing organizations would help managers, decision-makers and practitioners of WCM in better understanding of these performance measures and to focus on appropriate performance measures while implementing WCM in their organizations. Originality/value – Performance measures are of paramount importance for the implementation of WCM practices. Knowing the key performance measures and relationship among them can help many organizations to implement WCM practices. It is one of the foremost attempts to model performance measures of WCM. The paper provides useful insights into the WCM practitioners, consultants and researchers.

scholarly journals Analysing the temporal dynamics of model performance for hydrological models

2008 ◽  
Vol 5 (6) ◽  
pp. 3169-3211 ◽  
Author(s):  
D. E. Reusser ◽  
T. Blume ◽  
B. Schaefli ◽  
E. Zehe
Keyword(s):  
Time Series ◽  
Performance Assessment ◽  
Performance Measures ◽  
Hydrological Model ◽  
Temporal Dynamics ◽  
Model Performance ◽  
High Dimensional ◽  
High Temporal Resolution ◽  
Headwater Catchment

Abstract. The temporal dynamics of hydrological model performance gives insights into errors that cannot be obtained from global performance measures assigning a single number to the fit of a simulated time series to an observed reference series. These errors can include errors in data, model parameters, or model structure. Dealing with a set of performance measures evaluated at a high temporal resolution implies analyzing and interpreting a high dimensional data set. This paper presents a method for such a hydrological model performance assessment with a high temporal resolution and illustrates its application for two very different rainfall-runoff modeling case studies. The first is the Wilde Weisseritz case study, a headwater catchment in the eastern Ore Mountains, simulated with the conceptual model WaSiM-ETH. The second is the Malalcahuello case study, a headwater catchment in the Chilean Andes, simulated with the physics-based model Catflow. The proposed time-resolved performance assessment starts with the computation of a large set of classically used performance measures for a moving window. The key of the developed approach is a data-reduction method based on self-organizing maps (SOMs) and cluster analysis to classify the high-dimensional performance matrix. Synthetic peak errors are used to interpret the resulting error classes. The final outcome of the proposed method is a time series of the occurrence of dominant error types. For the two case studies analyzed here, 6 such error types have been identified. They show clear temporal patterns which can lead to the identification of model structural errors.

scholarly journals Empirical evidence of the impact of study characteristics on the performance of prediction models: a meta-epidemiological study

BMJ Open ◽  
2019 ◽  
Vol 9 (4) ◽  
pp. e026160 ◽  
Author(s):  
Johanna A A G Damen ◽  
Thomas P A Debray ◽  
Romin Pajouheshnia ◽  
Johannes B Reitsma ◽  
Rob J P M Scholten ◽  
...  
Keyword(s):  
Performance Measures ◽  
Systematic Reviews ◽  
Validation Studies ◽  
Prognostic Model ◽  
External Validation ◽  
Model Performance ◽  
Population Characteristics ◽  
Prognostic Models ◽  
C Statistic ◽  
Study Characteristics

ObjectivesTo empirically assess the relation between study characteristics and prognostic model performance in external validation studies of multivariable prognostic models.DesignMeta-epidemiological study.Data sources and study selectionOn 16 October 2018, we searched electronic databases for systematic reviews of prognostic models. Reviews from non-overlapping clinical fields were selected if they reported common performance measures (either the concordance (c)-statistic or the ratio of observed over expected number of events (OE ratio)) from 10 or more validations of the same prognostic model.Data extraction and analysesStudy design features, population characteristics, methods of predictor and outcome assessment, and the aforementioned performance measures were extracted from the included external validation studies. Random effects meta-regression was used to quantify the association between the study characteristics and model performance.ResultsWe included 10 systematic reviews, describing a total of 224 external validations, of which 221 reported c-statistics and 124 OE ratios. Associations between study characteristics and model performance were heterogeneous across systematic reviews. C-statistics were most associated with variation in population characteristics, outcome definitions and measurement and predictor substitution. For example, validations with eligibility criteria comparable to the development study were associated with higher c-statistics compared with narrower criteria (difference in logit c-statistic 0.21(95% CI 0.07 to 0.35), similar to an increase from 0.70 to 0.74). Using a case-control design was associated with higher OE ratios, compared with using data from a cohort (difference in log OE ratio 0.97(95% CI 0.38 to 1.55), similar to an increase in OE ratio from 1.00 to 2.63).ConclusionsVariation in performance of prognostic models across studies is mainly associated with variation in case-mix, study designs, outcome definitions and measurement methods and predictor substitution. Researchers developing and validating prognostic models should realise the potential influence of these study characteristics on the predictive performance of prognostic models.

Model-performance Measures

2021 ◽  
pp. 173-193
Author(s):  
Przemyslaw Biecek ◽  
Tomasz Burzykowski
Keyword(s):  
Performance Measures ◽  
Model Performance

Bootstrap internal validation command for predictive logistic regression models

2021 ◽  
Vol 21 (2) ◽  
pp. 498-509
Author(s):  
B. M. Fernandez-Felix ◽  
E. García-Esquinas ◽  
A. Muriel ◽  
A. Royuela ◽  
J. Zamora
Keyword(s):  
Logistic Regression ◽  
Regression Model ◽  
Predictive Model ◽  
Performance Measures ◽  
Predictive Models ◽  
Regression Models ◽  
Logistic Regression Model ◽  
Model Performance ◽  
Internal Validation ◽  
Logistic Regression Models

Overfitting is a common problem in the development of predictive models. It leads to an optimistic estimation of apparent model performance. Internal validation using bootstrapping techniques allows one to quantify the optimism of a predictive model and provide a more realistic estimate of its performance measures. Our objective is to build an easy-to-use command, bsvalidation, aimed to perform a bootstrap internal validation of a logistic regression model.

Financially motivated model performance measures

2006 ◽  
Vol 2 (2) ◽  
pp. 69-81 ◽  
Author(s):  
Craig Friedman ◽  
Sven Sandow
Keyword(s):  
Performance Measures ◽  
Model Performance

MODEL PERFORMANCE MEASURES FOR EXPECTED UTILITY MAXIMIZING INVESTORS

2003 ◽  
Vol 06 (04) ◽  
pp. 355-401 ◽  
Author(s):  
CRAIG FRIEDMAN ◽  
SVEN SANDOW
Keyword(s):  
Performance Measures ◽  
Expected Utility ◽  
Probability Model ◽  
Model Performance ◽  
Performance Measure ◽  
Relative Performance ◽  
Conditional Probability Density ◽  
New Paradigm ◽  
Discrete Probability ◽  
Wide Range

We examine model performance measures in four contexts: Discrete Probability, Continuous Probability, Conditional Discrete Probability and Conditional Probability Density Models. We consider the model performance question from the point of view of an investor who evaluates models based on the performance of the (optimal) strategies that the models suggest. Under this new paradigm, the investor selects the model with the highest estimated expected utility. We interpret our performance measures in information theoretic terms and provide new generalizations of entropy and Kullback-Leibler relative entropy. We show that the relative performance measure is independent of the market prices if and only if the investor's utility function is a member of a logarithmic family that admits a wide range of possible risk aversions. In this case, we show that the relative performance measure is equivalent to the (easily understood) differential expected growth of wealth or the (familiar) likelihood ratio. We state conditions under which relative performance measures for general utilities are well approximated by logarithmic-family-based relative performance measures. Some popular probability model performance measures (including ROC methods) are not consistent with our framework. We demonstrate that rank based performance measures can suggest model selections that are disastrous under various popular utilities.

An Algorithm and Criteria for Compressor Characteristics Real Time Modeling and Approximation

1991 ◽  
Vol 113 (1) ◽  
pp. 112-118 ◽  
Author(s):  
A. M. El-Gammal
Keyword(s):  
Real Time ◽  
Performance Measures ◽  
Systematic Approach ◽  
Model Performance ◽  
Aircraft Engine ◽  
Quantitative Model ◽  
Multiple Objectives ◽  
Time Model ◽  
Time Modeling ◽  
Engine Component

Numerous problems are encountered in realizing an adequate real time model for an aircraft engine. The purpose of this article is to propose a systematic approach for modeling and approximating the characteristics of an engine or engine component parts, and to apply this approach to the Viper compressor (VC) characteristics. The proposed approach introduces a set of quantitative model-performance measures. Monitoring these measures makes it possible to take care of the multiple objectives of the model simultaneously and individually, and to attach a guarantee level to the model behavior. A set of (66) models are considered, from which the adequate VC model for real time (RT) simulation purposes is determined.

Sign in / Sign up

Export Citation Format

Share Document