scholarly journals Evaluating one-shot tournament predictions

2020 ◽  
pp. 1-10
Author(s):  
Claus Thorn Ekstrøm ◽  
Hans Van Eetvelde ◽  
Christophe Ley ◽  
Ulf Brefeld

We introduce the Tournament Rank Probability Score (TRPS) as a measure to evaluate and compare pre-tournament predictions, where predictions of the full tournament results are required to be available before the tournament begins. The TRPS handles partial ranking of teams, gives credit to predictions that are only slightly wrong, and can be modified with weights to stress the importance of particular features of the tournament prediction. Thus, the Tournament Rank Prediction Score is more flexible than the commonly preferred log loss score for such tasks. In addition, we show how predictions from historic tournaments can be optimally combined into ensemble predictions in order to maximize the TRPS for a new tournament.

2008 ◽  
Vol 99 (01) ◽  
pp. 202-207 ◽  
Author(s):  
Louis-Rachid Salmi ◽  
Marie-Antoinette Sevestre-Pietri ◽  
Sophie Perusat ◽  
Monika Nguon ◽  
Maryse Degeilh ◽  
...  

SummaryIt was the objective of this study to design a clinical prediction score for the diagnosis of upper extremity deep venous thrombosis (UEDVT).A score was built by multivariate logistic regression in a sample of patients hospitalized for suspicion of UEDVT (derivation sample). It was validated in a second sample in the same university hospital, then in a sample from the multicenter OPTIMEV study that included both outpatients and inpatients. In these three samples, UEDVT diagnosis was objectively confirmed by ultrasound. The derivation sample included 140 patients among whom 50 had confirmed UEDVT, the validation sample included 103 patients among whom 46 had UEDVT, and the OPTIMEV sample included 214 patients among whom 65 had UEDVT. The clinical score identified a combination of four items (venous material, localized pain, unilateral pitting edema and other diagnosis as plausible). One point was attributed to each item (positive for the first 3 and negative for the other diagnosis). A score of –1 or 0 characterized low probability patients, a score of 1 identified intermediate probability patients, and a score of 2 or 3 identified patients with high probability. Low probability score identified a prevalence of UEDVT of 12, 9 and 13%, respectively, in the derivation, validation and OPTIMEV samples. High probability score identified a prevalence of UEDVT of 70, 64 and 69% respectively. In conclusion we propose a simple score to calculate clinical probability of UEDVT. This score might be a useful test in clinical trials as well as in clinical practice.


1989 ◽  
Author(s):  
J. Frank Yates ◽  
Ilan Yaniv ◽  
Juwhei Lee ◽  
J. E. Keith Smith
Keyword(s):  

2020 ◽  
Vol 30 (5) ◽  
pp. 746-753
Author(s):  
Ning Dong ◽  
Hulin Piao ◽  
Yu Du ◽  
Bo Li ◽  
Jian Xu ◽  
...  

Abstract OBJECTIVES Acute kidney injury (AKI) is a common complication of cardiovascular surgery that is associated with increased mortality, especially after surgeries involving the aorta. Early detection and prevention of AKI in patients with aortic dissection may help improve outcomes. The objective of this study was to develop a practical prediction score for AKI after surgery for Stanford type A acute aortic dissection (TAAAD). METHODS This was a retrospective cohort study that included 2 independent hospitals. A larger cohort of 326 patients from The Second Hospital of Jilin University was used to identify the risk factors for AKI and to develop a risk score. The derived risk score was externally validated in a separate cohort of 102 patients from the other hospital. RESULTS The scoring system included the following variables: (i) age >45 years; (ii) body mass index >25 kg/m2; (iii) white blood cell count >13.5 × 109/l; and (iv) lowest perioperative haemoglobin <100 g/l, cardiopulmonary bypass duration >150 min and renal malperfusion. On receiver operating characteristic curve analysis, the score predicted AKI with fair accuracy in both the derivation [area under the curve 0.778, 95% confidence interval (CI) 0.726–0.83] and the validation (area under the curve 0.747, 95% CI 0.657–0.838) cohorts. CONCLUSIONS We developed a convenient scoring system to identify patients at high risk of developing AKI after surgery for TAAAD. This scoring system may help identify patients who require more intensive postoperative management and facilitate appropriate interventions to prevent AKI and improve patient outcomes.


2021 ◽  
Vol 0 (0) ◽  
Author(s):  
Edward Wheatcroft

Abstract A scoring rule is a function of a probabilistic forecast and a corresponding outcome used to evaluate forecast performance. There is some debate as to which scoring rules are most appropriate for evaluating forecasts of sporting events. This paper focuses on forecasts of the outcomes of football matches. The ranked probability score (RPS) is often recommended since it is ‘sensitive to distance’, that is it takes into account the ordering in the outcomes (a home win is ‘closer’ to a draw than it is to an away win). In this paper, this reasoning is disputed on the basis that it adds nothing in terms of the usual aims of using scoring rules. A local scoring rule is one that only takes the probability placed on the outcome into consideration. Two simulation experiments are carried out to compare the performance of the RPS, which is non-local and sensitive to distance, the Brier score, which is non-local and insensitive to distance, and the Ignorance score, which is local and insensitive to distance. The Ignorance score outperforms both the RPS and the Brier score, casting doubt on the value of non-locality and sensitivity to distance as properties of scoring rules in this context.


Symmetry ◽  
2021 ◽  
Vol 13 (2) ◽  
pp. 192
Author(s):  
Hui Lin ◽  
Jianxin You ◽  
Tao Xu

Evaluation of online teaching quality has become an important issue because many universities are turning to online classes due to the Corona Virus Disease 2019 (COVID-19) pandemic. In this paper, online teaching quality evaluation is considered as a linguistic multi-attribute group decision-making (MAGDM) problem. Generally, the evaluation sematic information can be symmetrically or asymmetrically distributed in linguistic term sets. Thus, an extended linguistic MAGDM framework is proposed for evaluating online teaching quality. As the main contribution, the proposed method takes into account the risk preferences of assessment experts (AEs) and unknown weight information of attributes and sub-attributes. To be specific, the Delphi method is employed to establish a multi-level evaluation indicator system (EIS) of online teaching quality. Then, by introducing the group generalized linguistic term set (GLTS) with two risk preference parameters, a two-stage optimization model is developed to calculate the weights of attributes and sub-attributes objectively. Subsequently, the linguistic MAGDM framework was divided into two stages. The first stage maximizes the group comprehensive rating values of teachers on different attributes to obtain partial ranking results for teachers on each attribute. The latter stage maximizes the group comprehensive rating values of teachers to evaluate the overall quality. Finally, a case study is provided to illustrate how to apply the framework to evaluate online teaching quality.


Sign in / Sign up

Export Citation Format

Share Document