scholarly journals Applying a General Analytic Method for Assessing Bias Sensitivity to Bias-Adjusted Threat and Equitable Threat Scores

2009 ◽  
Vol 24 (6) ◽  
pp. 1748-1754 ◽  
Author(s):  
Keith F. Brill ◽  
Fedor Mesinger

Abstract Bias-adjusted threat and equitable threat scores were designed to account for the effects of placement errors in assessing the performance of under- or overbiased forecasts. These bias-adjusted performance measures exhibit bias sensitivity. The critical performance ratio (CPR) is the minimum fraction of added forecasts that are correct for a performance measure to indicate improvement if bias is increased. In the opposite case, the CPR is the maximum fraction of removed forecasts that are correct for a performance measure to indicate improvement if bias is decreased. The CPR is derived here for the bias-adjusted threat and equitable threat scores to quantify bias sensitivity relative to several other measures of performance including conventional threat and equitable threat scores. The CPR for a bias-adjusted equitable threat score may indicate the likelihood of preserving or increasing the conventional equitable threat score if forecasts are bias corrected based on past performance.

2010 ◽  
Vol 25 (4) ◽  
pp. 1307-1314 ◽  
Author(s):  
Keith F. Brill ◽  
Matthew Pyle

Abstract Critical performance ratio (CPR) expressions for the eight conditional probabilities associated with the 2 × 2 contingency table of outcomes for binary (dichotomous “yes” or “no”) forecasts are derived. Two are shown to be useful in evaluating the effects of hedging as it approaches random change. The CPR quantifies how the probability of detection (POD) must change as frequency bias changes, so that a performance measure (or conditional probability) indicates an improved forecast for a given value of frequency bias. If yes forecasts were to be increased randomly, the probability of additional correct forecasts (hits) is given by the detection failure ratio (DFR). If the DFR for a performance measure is greater than the CPR, the forecast is likely to be improved by the random increase in yes forecasts. Thus, the DFR provides a benchmark for the CPR in the case of frequency bias inflation. If yes forecasts are decreased randomly, the probability of removing a hit is given by the frequency of hits (FOH). If the FOH for a performance measure is less than the CPR, the forecast is likely to be improved by the random decrease in yes forecasts. Therefore, the FOH serves as a benchmark for the CPR if the frequency bias is decreased. The closer the FOH (DFR) is to being less (greater) than or equal to the CPR, the more likely it may be to enhance the performance measure by decreasing (increasing) the frequency bias. It is shown that randomly increasing yes forecasts for a forecast that is itself better than a randomly generated forecast can improve the threat score but is not likely to improve the equitable threat score. The equitable threat score is recommended instead of the threat score whenever possible.


2009 ◽  
Vol 24 (1) ◽  
pp. 307-318 ◽  
Author(s):  
Keith F. Brill

Abstract Performance measures computed from the 2 × 2 contingency table of outcomes for dichotomous forecasts are sensitive to bias. The method presented here evaluates how the probability of detection (POD) must change as bias changes so that a performance measure improves at a given value of bias. A critical performance ratio (CPR) of the change of POD to the change in bias is derived for a number of performance measures. If a change in POD associated with a bias change satisfies the CPR condition, the performance measure will indicate an improved forecast. If a perfect measure of performance existed, it would always exhibit its optimal value at bias equal to one. Actual measures of performance are susceptible to bias, indicating a better forecast for bias values not equal to one. The CPR is specifically applied to assess the conditions for an improvement toward a more favorable value of several commonly used performance measures as bias increases or decreases through the value one. All performance measures evaluated are found to have quantifiable bias sensitivity. The CPR is applied to analyzing a performance requirement and bias sensitivity in a geometric model.


Author(s):  
Mara Madaleno ◽  
Elisabete S. Vieira ◽  
João P. C. Teodósio

Using a sample of 47 Portuguese and Spanish firms for the period 2010 to 2017, the authors study the relationship between female presence on board and firm's accounting (ROA and ROE) and market-based (MTB and Tobin's Q) performance. They find that women on the board of directors is positively related to firm's performance, as well as the gender of the CFO and the proportion of women on the listed key professionals, when we consider the market measures of performance, not being so consistent for accounting performance measures. Results were sensitive to the performance measure used. The results reinforce the political options of European Commission gender established quotas, revealing that in the Iberian countries these quotas are not being effectively implemented, even if results suggest that women on board in fact exert positive influence over market performance. This also led us to think that financial markets may also react in a positive way when the CFO of the company is a woman instead of a man, despite the sample limitations both in terms of gender and number of firms.


Author(s):  
Tracy M. Maylett

This case study describes an initiative to change a long-standing performance management process at a large manufacturing facility within General Mills that emphasized the attainment of objective performance measures (the “what” of performance) to one that also included the “how” of goal achievement. The organization embarked on a 3-year pilot evaluation of the use of 360 Feedback as a possible solution to replace or supplement their traditional single-source (supervisor) performance appraisal process. The two systems ran in parallel using 140 randomly selected employees. Results showed little correlation between the what measures of performance from the traditional appraisals and the how data collected using the 360 Feedback, supporting the view that job performance should be viewed as requiring both aspects of evaluation, using different methods of assessment. Ultimately, the organization maintained both systems but integrated 360 Feedback into the traditional appraisals as well, creating complementary processes that looked “forward” (development) and “past” (performance).


Author(s):  
Edward Connelly

Performance assessments via institutionalized performance measures are a key part of the ongoing, global restructuring of corporate, industrial, government, and military ways of doing business. Corporations, recognizing that the “game” is played for keeps, are reorganizing to survive and prosper in the national and international environment. Often, this restructuring is not just adding new technology, although it typically takes advantage of new technology. Instead, this is a work environment change supporting effective work interactions. Led by the training and human resources personnel, employees learn to share goals, to communicate effectively and to help each other solve problems in order to function as an effective team. Results of these efforts are truly inspiring. Organizations such as Kodak, IBM, Ford, and others have turned things around, producing profitable units providing quality products and services, and having fun while doing it (Anfuso 1994). A key part of this turn-a-round is the common understanding of how performance is evaluated, a result achieved via instutionalized performance measures. A performance measure is a definition of how performance is to be assessed. The definition is always based on an individual's subjective preference of the worth of demonstrated performances or proposed activities. When the individual is an authority whose performance assessments significantly impact the performance of other individuals and the organization, then that individual's assessment concept demands attention. A performance measure, acceptable to that authority by virtue of it rating performances the same way the authority does, can systematize the assessment process by communicating what data are to be collected and how those data are to be processed to determine the performance rating. Extracting information from authorities for building performance measures does not require advances in computer technology or mathematics. Instead, it requires creating an environment in which the authority can interact with a facilitator to consider alternative workplace outcomes and can provide ratings of the desirability of each outcome. To demonstrate this interaction as a human factors problem, this paper describes the environments necessary to extract the definitions of good performance from authorities. These definitions are the basis for building the equivalent performance measures.


2008 ◽  
Vol 16 ◽  
pp. 137-142 ◽  
Author(s):  
F. Mesinger

Abstract. Among the wide variety of performance measures available for the assessment of skill of deterministic precipitation forecasts, the equitable threat score (ETS) might well be the one used most frequently. It is typically used in conjunction with the bias score. However, apart from its mathematical definition the meaning of the ETS is not clear. It has been pointed out (Mason, 1989; Hamill, 1999) that forecasts with a larger bias tend to have a higher ETS. Even so, the present author has not seen this having been accounted for in any of numerous papers that in recent years have used the ETS along with bias "as a measure of forecast accuracy". A method to adjust the threat score (TS) or the ETS so as to arrive at their values that correspond to unit bias in order to show the model's or forecaster's accuracy in \\textit{placing} precipitation has been proposed earlier by the present author (Mesinger and Brill, the so-called dH/dF method). A serious deficiency however has since been noted with the dH/dF method in that the hypothetical function that it arrives at to interpolate or extrapolate the observed value of hits to unit bias can have values of hits greater than forecast when the forecast area tends to zero. Another method is proposed here based on the assumption that the increase in hits per unit increase in false alarms is proportional to the yet unhit area. This new method removes the deficiency of the dH/dF method. Examples of its performance for 12 months of forecasts by three NCEP operational models are given.


Author(s):  
Mara Madaleno ◽  
Elisabete S. Vieira ◽  
João P. C. Teodósio

Using a sample of 47 Portuguese and Spanish firms for the period 2010 to 2017, the authors study the relationship between female presence on board and firm's accounting (ROA and ROE) and market-based (MTB and Tobin's Q) performance. They find that women on the board of directors is positively related to firm's performance, as well as the gender of the CFO and the proportion of women on the listed key professionals, when we consider the market measures of performance, not being so consistent for accounting performance measures. Results were sensitive to the performance measure used. The results reinforce the political options of European Commission gender established quotas, revealing that in the Iberian countries these quotas are not being effectively implemented, even if results suggest that women on board in fact exert positive influence over market performance. This also led us to think that financial markets may also react in a positive way when the CFO of the company is a woman instead of a man, despite the sample limitations both in terms of gender and number of firms.


Water Policy ◽  
2011 ◽  
Vol 13 (5) ◽  
pp. 677-692 ◽  
Author(s):  
Jonathan R. Winsten ◽  
Claire Baffaut ◽  
Joseph Britt ◽  
Tatiana Borisova ◽  
Chad Ingels ◽  
...  

Current federal and state soil and water conservation programs consist primarily of cost-sharing or compensating farmers for implementing a set of pre-defined best management practices. This approach does not consider specific environmental outcomes or the cost-effectiveness of the program at the farm or watershed level. Performance-based incentives are payments that are attached to a specified environmental performance measure, for example a reduction in nutrient, sediment, pesticide, or bacteria loss from a field, or reduction in loading at the mouth of a watershed. Assessing environmental performance requires the definition of a performance measure, which is used to calculate resulting incentive payments. In this paper, we discuss issues related to the selection of a performance measure, such as the scale at which the performance is measured, modeling versus monitoring, and the ability of the measure to foster farmer learning and adaptive management. Numerous examples of performance measures adopted or considered by watershed stakeholders in several states are presented. Watershed managers, policy makers, and extension agents can consider this information to make informed choices among proposed approaches toward improving water quality in streams affected by agricultural nonpoint source pollution.


2020 ◽  
Vol 23 (02) ◽  
pp. 2050014
Author(s):  
DIPANKAR MONDAL ◽  
N. SELVARAJU

This paper proposes a set of desirable axioms to characterize performance measures in the context of portfolio management. A performance measure consistent with the axioms is called “ideal”. We observe that a popular performance measure, Farinelli–Tibiletti (FT) ratio [S. Farinelli & L. Tibiletti (2008) Sharpe thinking in asset ranking with one-sided measures, European Journal of Operational Research 185 (3), 1542–1547], which captures potential-seeking behavior, is not ideal. It violates a very important property of portfolio theory, the diversification. As an alternative, we propose a new ideal performance measure, upside beta ratio (UBR). To examine its performance, we rank mutual funds for UBR and other four performance measures — Sharpe, Sortino, FT and Jensen’s alpha — and then we compare the rankings of UBR with the rankings of other ratios. In addition, the performance of top-ranked funds are compared through back-testing and out-of-sample analysis. Our findings reveal that the UBR performs significantly better than the other ratios in most scenarios. Finally, in order to check robustness of the new measure, a parameter sensitivity analysis is presented.


2019 ◽  
Author(s):  
Rishal Asri

Sunlight is energy that can be converted into electrical energy. One of the uses is by applying it to the roof ofthe building. The application in this building has restrictions such as the placement of the PV moduleshorizontally and vertically. In the study comparing the results of energy obtained from the PV system withhorizontal and vertical positions with a standard degree angle in the direction of azimuth sunlight. Positionusing the horizontal produces more energy and reaches a performance ratio of more than 80%.


Sign in / Sign up

Export Citation Format

Share Document