The Effect of Visual Sort and Rate Versus Visual Analog Scales on the Reliability of Judgments of Dysphonia

Purpose The reliability of auditory-perceptual judgments between listeners is a long-standing problem in the assessment of voice disorders. The purpose of this study was to determine whether a relatively novel experimental scaling method, called visual sort and rate (VSR), yielded stronger reliability than the more frequently used method of visual analog scales (VAS) for ratings of overall severity (OS) and breathiness (BR) in speakers with voicedisorders. Method Fifty speech samples were selected from a database of speakers with voice disorders. Twenty-two inexperienced listeners provided ratings of OS or BR in four rating blocks: VSR-OS, VSR-BR, VAS-OS, and VSR-BR. For the VAS task, listeners rated each speaker for BR or OS using a vertically oriented 100-mm VAS. For the VSR task, stimuli were distributed into sets of samples with a range of speaker severities in each set. Listeners sorted and ranked samples for OS or BR within each set, and final ratings were captured on a vertically oriented 100-mm VAS. Interrater variability, defined as the mean of the squared differences between a listener's ratings and group mean ratings, and intrarater reliability (Pearson r ) were compared across rating tasks for OS and BR using paired t tests. Results Results showed that listeners had significantly less interrater variability (better reliability) when using VSR methods compared to VAS for judgments of both OS and BR. Intrarater reliability was high across rating tasks and dimensions; however, ratings of BR were significantly more consistent within individual listeners when using VAS than when using VSR. Conclusions VSR is an experimental method that decreases variability of auditory-perceptual judgments between inexperienced listeners when rating speakers with a range of dysphonic severities and disorders. Future research should determine whether a clinically viable tool may be developed based on VSR principles and whether such benefits extend to experienced listeners.

Near Adult Heights and Predictions of Prospective Adult Heights using Automated and Conventional Greulich-Pyle Bone Age Determinations in children with chronic endocrine diseases

Calculation of prospective adult heights (PAH) is associated with considerable bone age interrater variability. Therefore, the new PAH method based on automated bone age (BA) determination (BoneXpert™) was compared to the conventional PAH method by Bayley- Pinneau (BP) based on BA determination according to Greulich and Pyle (GP) and to observed near adult heights. Heights and near adult heights were measured in 82 patients (48 females) with chronic endocrinopathies at age of 10.45 ± 2.12 years and at time of transition to adult care (17.98 ± 3.02 years). Further, BA were assessed according to conventional GP - by three experts- and by BoneXpert™. PAH were calculated using conventional BP tables and BoneXpert™. The conventional and the automated BA determinations revealed a mean difference of 0.25 ± 0.72 years (p = 0.0027). The automated PAH by BoneXpert™ were 156.96 ± 0.86 cm in females and 171.75 ± 1.6 cm in males compared to 153.95 ± 1.12 cm in females and 169.31 ± 1.6 cm in males by conventional BP, respectively, and in comparison to near adult heights 156.38 ± 5.84 cm in females and 168.94 ± 8.18 cm in males, respectively. Conclusion: BA ratings and adult height predictions by BoneXpert™ in children with chronic endocrinopathies abolish rater dependent variability and enhance reproducibility of estimates thereby refining care in growth disorders. Conventional methods may outperform automated analyses in specific cases.

Reliability of the Clinical Frailty Scale in very elderly ICU patients: a prospective European study

Purpose Frailty is a valuable predictor for outcome in elderly ICU patients, and has been suggested to be used in various decision-making processes prior to and during an ICU admission. There are many instruments developed to assess frailty, but few of them can be used in emergency situations. In this setting the clinical frailty scale (CFS) is frequently used. The present study is a sub-study within a larger outcome study of elderly ICU patients in Europe (the VIP-2 study) in order to document the reliability of the CFS. Materials and methods From the VIP-2 study, 129 ICUs in 20 countries participated in this sub-study. The patients were acute admissions ≥ 80 years of age and frailty was assessed at admission by two independent observers using the CFS. Information was obtained from the patient, if not feasible, from the family/caregivers or from hospital files. The profession of the rater and source of data were recorded along with the score. Interrater variability was calculated using linear weighted kappa analysis. Results 1923 pairs of assessors were included and background data of patients were similar to the whole cohort (n = 3920). We found a very high inter-rater agreement (weighted kappa 0.86), also in subgroup analyses. The agreement when comparing information from family or hospital records was better than using only direct patient information, and pairs of raters from same profession performed better than from different professions. Conclusions Overall, we documented a high reliability using CFS in this setting. This frailty score could be used more frequently in elderly ICU patients in order to create a more holistic and realistic impression of the patient´s condition prior to ICU admission.

Interrater Agreement of The Copenhagen Triage Algorithm

Introduction Systematic triage is performed in the Emergency Department (ED) to assess the urgency of care for each patient. The Copenhagen Triage Algorithm (CTA) is a newly developed, evidence-based triage system, however the interrater agreement remains unknown. Method This was a prospective cohort study. The collection of data was conducted in the three sections (Acute/Cardiology, Medicine and Surgery) of the ED of Herlev Hospital. Patients were assessed independently by two different nurses using CTA. The interrater variability of CTA was calculated using Fleiss kappa. The analysis was stratified according to less or more than 2 years of ED experience. Results A total of 110 patients were included of which 10 were excluded due to incomplete data. The raters agreed on triage category 80 % of the time corresponding to a kappa value of 0.70 (95% confidence interval 0.57-0.83). Stratified on ED sections, the agreement was 83 % in the Acute/Cardiology section corresponding to a kappa value of 0.73 (0.55-0.91), 79 % in the Medicine section corresponding to a kappa value of 0.64 (0.39-0.89) and 0.56 % in the Surgery section corresponding to a kappa value of 0.56 (0.21-0.90). The experienced raters had an interrater agreement of 0.73 (0.56-0.90), while the less experienced raters had an agreement of 0.76, (0.28-1.24). Conclusion A substantial interrater agreement was found for the Copenhagen triage algorithm.

Systematic review and meta-analysis of diagnostic agreement in suspected TIA

ObjectiveTo determine the interrater variability for TIA diagnostic agreement among expert clinicians (neurologists/stroke physicians), administrative data, and nonspecialists.MethodsWe performed a meta-analysis of studies from January 1984 to January 2019 using MEDLINE, EMBASE, and PubMed. Two reviewers independently screened for eligible studies and extracted interrater variability measurements using Cohen's kappa scores to assess diagnostic agreement.ResultsNineteen original studies consisting of 19,421 patients were included. Expert clinicians demonstrate good agreement for TIA diagnosis (κ = 0.71, 95% confidence interval [CI] = 0.62–0.81). Interrater variability between clinicians' TIA diagnosis and administrative data also demonstrated good agreement (κ = 0.68, 95% CI = 0.62–0.74). There was moderate agreement (κ = 0.41, 95% CI = 0.22–0.61) between referring clinicians and clinicians at TIA clinics receiving the referrals. Sixty percent of 748 patient referrals to TIA clinics were TIA mimics.ConclusionsOverall agreement between expert clinicians was good for TIA diagnosis, although variation still existed for a sizeable proportion of cases. Diagnostic agreement for TIA decreased among nonspecialists. The substantial number of patients being referred to TIA clinics with other (often neurologic) diagnoses was large, suggesting that clinicians, who are proficient in managing TIAs and their mimics, should run TIA clinics.

Key Histopathology Features of Cutaneous Acute Graft-Versus-Host Disease Can be Detected Noninvasively

BACKGROUND. Cutaneous erythema and histopathology features in patients post allogeneic hematopoietic cell transplantation (HCT) are nonspecific, making it difficult to distinguish acute graft-versus-host disease (aGVHD) from engraftment syndrome, drug reactions, viral infections, and other etiologies. Biopsies from different locations on the same patient frequently differ in aGVHD grade, and serial biopsies of a single lesion are not possible. Proper biopsy site selection, timing, and serial noninvasive microscopic monitoring may improve our understanding and management of aGVHD. In this study, we tested the feasibility of noninvasive reflectance confocal microscopy (RCM) to detect key histopathology features of cutaneous aGVHD. STUDY POPULATION. We enrolled 11 patients with high clinical suspicion of cutaneous aGVHD, as determined by a transplant physician. In total, 16 lesions of cutaneous aGVHD-affected site were imaged by RCM (6x6 mm2en face images at four different depths) and subsequently biopsied (4x4 mm punch biopsy). 9 out of 11 patients had only skin involvement, and 2 patients had skin and gut aGVHD. 5 patients had >50% body surface area (BSA) affected, 1 had 18-50% BSA, and 5 had <18% BSA. 8 out of 11 patients required systemic corticosteroids and 3 patients received topical corticosteroids. All patients were imaged before starting treatment. METHODS. We used a clinical confocal microscope (Vivascope 1500, Caliber I.D.), which enables real-time evaluation of epidermal and superficial dermal tissue at sub-cellular resolution. Four reflectance confocal microscopists blinded to histopathology independently evaluated the presence or absence of 18 RCM features1. Concurrently, four dermatopathologists blinded to clinical and confocal information determined the presence or absence of 19 histopathology features, as well as the Lerner aGVHD grade. A histologic feature was determined as "present" in a lesion when marked by most experts (expert vote) or resolved by a fifth expert in case of a disagreement among four experts. The reflectance confocal microscopist vote was then correlated to the dermatopathologist expert vote for 17 overlapping features. We also evaluated the interrater variability among microscopists and among dermatopathologists. RESULTS. The main aGVHD features by Lerner grade had > 88% correlation between RCM and histopathology: (1) basal vacuolar change, (2) presence of dyskeratotic keratinocytes, and (3) dermal inflammation. By contrast, dyskeratotic cells and inflammation at the adnexal structures had <50% correlation. This may be attributed to the fact that more adnexal structures are typically visible in the 6x6 mm2 RCM horizontal tissue view compared to vertical section histopathology. We found a similar interrater variability among RCM experts (70%) and dermatopathologists (68%). CONCLUSIONS. In this pilot study, we show that noninvasive label-free RCM of skin enables detection of the main Lerner grade features of cutaneous aGVHD. We found a similar interrater variability among reflectance confocal microscopists and among dermatopathologists who each independently evaluated the presence or absence of a list of aGVHD features. Future studies can build on this work to evaluate the feasibility of RCM to determine cutaneous aGVHD grade, as well as distinguish between rash due to aGVHD and drug reactions. ACKNOWLEDGEMENT. This work was supported in part by Career Development Award Number IK2 CX001785 from the United Sates Department of Veterans Affairs Clinical Science R&D (CSRD) Service. Saknite I, Gill M, Alessi-Fox C, Byrne M, Jagasia M, Gonzalez S, Ardigo M, Tkaczyk ER. Features of cutaneous acute graft-versus-host disease by reflectance confocal microscopy. Br J Dermatol (2019). Disclosures Christi: Caliber I.D.: Employment, Equity Ownership. Byrne:Karyopharm: Research Funding. Jagasia:Kadmon: Consultancy; Incyte: Consultancy; Janssen: Research Funding. Tkaczyk:Incyte: Consultancy.