Assessing the credibility of a clinical ladder review process: An interrater reliability study

2001 ◽  
Vol 49 (1) ◽  
pp. 27-29 ◽  
Author(s):  
Robin Dawn Froman
2020 ◽  
Vol 25 (1) ◽  
pp. 37-42 ◽  
Author(s):  
Ros Whelan ◽  
Eric Prince ◽  
David M. Mirsky ◽  
Robert Naftel ◽  
Aashim Bhatia ◽  
...  

OBJECTIVEPediatric adamantinomatous craniopharyngiomas (ACPs) are histologically benign brain tumors that confer significant neuroendocrine morbidity. Previous studies have demonstrated that injury to the hypothalamus is associated with worsened quality of life and a shorter lifespan. This insight helps many surgeons define the goals of surgery for patients with ACP. Puget and colleagues proposed a 3-tiered preoperative and postoperative grading system based on the degree of hypothalamic involvement identified on MRI. In a prospective cohort from their institution, the authors found that use of the system to guide operative goals was associated with decreased morbidity. To date, however, the Puget system has not been externally validated. Here, the authors present an interrater reliability study that assesses the generalizability of this system for surgeons planning initial operative intervention for children with craniopharyngiomas.METHODSA panel of 6 experts, consisting of pediatric neurosurgeons and pediatric neuroradiologists, graded 30 preoperative and postoperative MRI scans according to the Puget system. Interrater reliability was calculated using Fleiss’ κ and Krippendorff’s α statistics.RESULTSInterrater reliability in the preoperative context demonstrated moderate agreement (κ = 0.50, α = 0.51). Interrater reliability in the postoperative context was 0.27 for both methods of statistical evaluation.CONCLUSIONSInterrater reliability for the system as defined is moderate. Slight refinements of the Puget MRI grading system, such as collapsing the 3 grades into 2, may improve its reliability, making the system more generalizable.


Author(s):  
I. Joa ◽  
J. O. Johannessen ◽  
K. S. Heiervang ◽  
A. A. Sviland ◽  
H. A. Nordin ◽  
...  

Abstract This study examined psychometric properties and feasibility of the Family Psychoeducation (FPE) Fidelity Scale. Fidelity assessors conducted reviews using the FPE fidelity scale four times over 18 months at five sites in Norway. After completing fidelity reviews, assessors rated feasibility of the fidelity review process. The FPE fidelity scale showed excellent interrater reliability (.99), interrater item agreement (88%), and internal consistency (mean = .84 across four time points). By the 18-month follow-up, all five sites increased fidelity and three reached adequate fidelity. Fidelity assessors rated feasibility as excellent. The FPE fidelity scale has good psychometric properties and is feasible for evaluating the implementation of FPE programs. Trial registration ClinicalTrials.gov Identifier: NCT03271242.


2015 ◽  
Vol 10 (2) ◽  
pp. 183-190 ◽  
Author(s):  
Joel B. Chidley ◽  
Alexandra L. MacGregor ◽  
Caoimhe Martin ◽  
Calum A. Arthur ◽  
Jamie H. Macdonald

Purpose:To identify physiological, psychological, and skill characteristics that explain performance in downhill (DH) mountainbike racing.Methods:Four studies were used to (1) identify factors potentially contributing to DH performance (using an expert focus group), (2) develop and validate a measure of rider skill (using video analysis and expert judge evaluation), (3) evaluate whether physiological, psychological, and skill variables contribute to performance at a DH competition, and (4) test the specific contribution of aerobic capacity to DH performance.Results:Study 1 identified aerobic capacity, handgrip endurance, anaerobic power, rider skill, and self-confidence as potentially important for DH. In study 2 the rider-skill measure displayed good interrater reliability. Study 3 found that rider skill and handgrip endurance were significantly related to DH ride time (β = –0.76 and –0.14, respectively; R2 = .73), with exploratory analyses suggesting that DH ride time may also be influenced by self-confidence and aerobic capacity. Study 4 confirmed aerobic capacity as an important variable influencing DH performance (for a DH ride, mean oxygen uptake was 49 ± 5 mL · kg−1 · min−1, and 90% of the ride was completed above the 1st ventilatory threshold).Conclusions:In order of importance, rider skill, handgrip endurance, self-confidence, and aerobic capacity were identified as variables influencing DH performance. Practically, this study provides a novel assessment of rider skill that could be used by coaches to monitor training and identify talent. Novel intervention targets to enhance DH performance were also identified, including self-confidence and aerobic capacity.


2020 ◽  
Vol 100 (3) ◽  
pp. 468-476 ◽  
Author(s):  
Bolette S Rafn ◽  
Chiara A Singh ◽  
Julie Midtgaard ◽  
Pat G Camp ◽  
Margaret L McNeely ◽  
...  

Abstract Background Early identification of breast cancer–related upper body issues is important to enable timely physical therapist treatment. Objective This study evaluated the feasibility and reliability of women performing self-managed prospective surveillance for upper body issues in the early postoperative phase as part of a hospital-based physical therapy program. Design This was a prospective, single-site, single-group feasibility and reliability study. Methods Presurgery arm circumference measurements were completed at home and at the hospital by participants and by a physical therapist. Instruction in self-measurement was provided using a video guide. After surgery, all circumference measurements were repeated along with self-assessment and therapist assessment for shoulder flexion and abduction active range of motion. Feasibility was determined by recruitment/retention rates and participant-reported ease of performing self-measurements (1 [very difficult] to 10 [very easy]). Reliability was determined as intrarater reliability, interrater reliability, and agreement. Results Thirty-three women who were 53.4 (SD = 11.4) years old participated, with recruitment and retention rates of 79% and 94%, respectively. Participant-reported ease of measurement was 8.2 (SD = 2.2) before surgery and 8.0 (SD = 1.9) after surgery. The intrarater reliability and interrater reliability were excellent before surgery (intraclass correlation coefficient [ICC] ≥ 0.94; 95% confidence interval = 0.87–0.97) and after surgery (ICC ≥ 0.91; 95% confidence interval = 0.76–0.96). Agreement between self-assessed and therapist-assessed active shoulder flexion (κ = 0.79) and abduction (κ = 0.71) was good. Limitations Further testing is needed using a prospective design with a longer follow-up to determine whether self-managed prospective surveillance and timely treatment can hinder the development of chronic breast cancer–related upper body issues Conclusions Self-measured arm circumference and shoulder range of motion are reliable, and their inclusion in a hospital-based program of prospective surveillance for upper body issues seems feasible. This approach may improve early detection and treatment


2006 ◽  
Vol 40 (6-7) ◽  
pp. 606-607 ◽  
Author(s):  
Therese Garrick ◽  
Xanthe Glaw ◽  
Clive G. Harper ◽  
Amanda North

Sign in / Sign up

Export Citation Format

Share Document