scholarly journals Comparison of prehospital professional accuracy, speed, and interrater reliability of six pediatric triage algorithms

2022 ◽  
Vol 3 (1) ◽  
Author(s):  
Tabitha Cheng ◽  
Katherine Staats ◽  
Amy H. Kaji ◽  
Nicole D'Arcy ◽  
Kian Niknam ◽  
...  
Keyword(s):  
Interrater Reliability ◽  
Accuracy Speed

Cultural and Linguistic Adaptation of the Consensus Auditory-Perceptual Evaluation of Voice (CAPE-V) Into Hindi

2020 ◽  
Vol 63 (12) ◽  
pp. 3974-3981
Author(s):  
Ashwini Joshi ◽  
Isha Baheti ◽  
Vrushali Angadi
Keyword(s):  
Strong Correlation ◽  
Concurrent Validity ◽  
Interrater Reliability ◽  
Voice Quality ◽  
Weak Correlation ◽  
Voice Assessment ◽  
Perceptual Evaluation ◽  
Severity Grade ◽  
Normal Voice ◽  
Group A

Aim The purpose of this study was to develop and assess the reliability of a Hindi version of the Consensus Auditory-Perceptual Evaluation of Voice (CAPE-V). Reliability was assessed by comparing Hindi CAPE-V ratings with English CAPE-V ratings and by the Grade, Roughness, Breathiness, Asthenia and Strain (GRBAS) scale. Method Hindi sentences were created to match the phonemic load of the corresponding English CAPE-V sentences. The Hindi sentences were adapted for linguistic content. The original English and adapted Hindi CAPE-V and GRBAS were completed for 33 bilingual individuals with normal voice quality. Additionally, the Hindi CAPE-V and GRBAS were completed for 13 Hindi speakers with disordered voice quality. The agreement of CAPE-V ratings was assessed between language versions, GRBAS ratings, and two rater pairs (three raters in total). Pearson product–moment correlation was completed for all comparisons. Results A strong correlation ( r > .8, p < .01) was found between the Hindi CAPE-V scores and the English CAPE-V scores for most variables in normal voice participants. A weak correlation was found for the variable of strain ( r < .2, p = .400) in the normative group. A strong correlation ( r > .6, p < .01) was found between the overall severity/grade, roughness, and breathiness scores in the GRBAS scale and the CAPE-V scale in normal and disordered voice samples. Significant interrater reliability ( r > .75) was present in overall severity and breathiness. Conclusions The Hindi version of the CAPE-V demonstrates good interrater reliability and concurrent validity with the English CAPE-V and the GRBAS. The Hindi CAPE-V can be used for the auditory-perceptual voice assessment of Hindi speakers.

Interobserver Reliability Using the Phonetic Level Evaluation With Severely and Profoundly Hearing-Impaired Children

1991 ◽  
Vol 34 (5) ◽  
pp. 989-999 ◽  
Author(s):  
Stephanie Shaw ◽  
Truman E. Coggins
Keyword(s):  
Interrater Reliability ◽  
Interobserver Reliability ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Hearing Impaired ◽  
Intraclass Correlation Coefficients ◽  
Assessment Measure ◽  
Impaired Children ◽  
Speech Assessment ◽  
Hearing Impaired Children

This study examines whether observers reliably categorize selected speech production behaviors in hearing-impaired children. A group of experienced speech-language pathologists was trained to score the elicited imitations of 5 profoundly and 5 severely hearing-impaired subjects using the Phonetic Level Evaluation (Ling, 1976). Interrater reliability was calculated using intraclass correlation coefficients. Overall, the magnitude of the coefficients was found to be considerably below what would be accepted in published behavioral research. Failure to obtain acceptably high levels of reliability suggests that the Phonetic Level Evaluation may not yet be an accurate and objective speech assessment measure for hearing-impaired children.

CODEM Instrument

GeroPsych ◽  
2014 ◽  
Vol 27 (1) ◽  
pp. 23-31 ◽  
Author(s):  
Anne Kuemmel (This author contributed eq ◽  
Julia Haberstroh (This author contributed ◽  
Johannes Pantel
Keyword(s):  
Convergent Validity ◽  
Interrater Reliability ◽  
Discriminant Validity ◽  
Assessment Tool ◽  
Intraclass Correlation ◽  
Well Being ◽  
Communication Behavior ◽  
People With Dementia ◽  
Pearson's R ◽  
Pearson’S R

Communication and communication behaviors in situational contexts are essential conditions for well-being and quality of life in people with dementia. Measuring methods, however, are limited. The CODEM instrument, a standardized observational communication behavior assessment tool, was developed and evaluated on the basis of the current state of research in dementia care and social-communicative behavior. Initially, interrater reliability was examined by means of videoratings (N = 10 people with dementia). Thereupon, six caregivers in six German nursing homes observed 69 residents suffering from dementia and used CODEM to rate their communication behavior. The interrater reliability of CODEM was excellent (mean κ = .79; intraclass correlation = .91). Statistical analysis indicated that CODEM had excellent internal consistency (Cronbach’s α = .95). CODEM also showed excellent convergent validity (Pearson’s R = .88) as well as discriminant validity (Pearson’s R = .63). Confirmatory factor analysis verified the two-factor solution of verbal/content aspects and nonverbal/relationship aspects. With regard to the severity of the disease, the content and relational aspects of communication exhibited different trends. CODEM proved to be a reliable, valid, and sensitive assessment tool for examining communication behavior in the field of dementia. CODEM also provides researchers a feasible examination tool for measuring effects of psychosocial intervention studies that strive to improve communication behavior and well-being in dementia.

Determinants of Interrater Reliability of Job Analysis: A Meta-analysis

2002 ◽  
Vol 18 (1) ◽  
pp. 52-62 ◽  
Author(s):  
Olga F. Voskuijl ◽  
Tjarda van Sliedregt
Keyword(s):  
Data Collection ◽  
Interrater Reliability ◽  
Meta Analysis ◽  
Job Analysis ◽  
Job Descriptions ◽  
Job Description ◽  
Research And Practice ◽  
Descriptive Information ◽  
Interrater Reliability Coefficients ◽  
Reliability Coefficients

Summary: This paper presents a meta-analysis of published job analysis interrater reliability data in order to predict the expected levels of interrater reliability within specific combinations of moderators, such as rater source, experience of the rater, and type of job descriptive information. The overall mean interrater reliability of 91 reliability coefficients reported in the literature was .59. The results of experienced professionals (job analysts) showed the highest reliability coefficients (.76). The method of data collection (job contact versus job description) only affected the results of experienced job analysts. For this group higher interrater reliability coefficients were obtained for analyses based on job contact (.87) than for those based on job descriptions (.71). For other rater categories (e.g., students, organization members) neither the method of data collection nor training had a significant effect on the interrater reliability. Analyses based on scales with defined levels resulted in significantly higher interrater reliability coefficients than analyses based on scales with undefined levels. Behavior and job worth dimensions were rated more reliable (.62 and .60, respectively) than attributes and tasks (.49 and .29, respectively). Furthermore, the results indicated that if nonprofessional raters are used (e.g., incumbents or students), at least two to four raters are required to obtain a reliability coefficient of .80. These findings have implications for research and practice.

Assessing Stressors Related to Migration in Patients With Psychiatric Disorders

2012 ◽  
Vol 28 (4) ◽  
pp. 262-269 ◽  
Author(s):  
Matthias Johannes Müller ◽  
Suzan Kamcili-Kubach ◽  
Songül Strassheim ◽  
Eckhardt Koch
Keyword(s):  
Interrater Reliability ◽  
Item Analysis ◽  
Global Assessment Of Functioning ◽  
Family Conflicts ◽  
Response Options ◽  
Reliability Indices ◽  
Dsm Iv ◽  
Functional Relevance ◽  
Sum Scores ◽  
Gaf Scores

A 10-item instrument for the assessment of probable migration-related stressors was developed based on previous work (MIGSTR10) and interrater reliability was tested in a chart review study. The MIGSTR10 and nine nonspecific stressors of the DSM-IV Axis IV (DSMSTR9) were put into a questionnaire format with categorical and dimensional response options. Charts of 100 inpatients (50 Turkish migrants [MIG], 50 native German patients [CON]) with affective or anxiety disorder were reviewed by three independent raters and MIGSTR10, DSMSTR9, and Global Assessment of Functioning scale (GAF) scores were obtained. Interrater reliability indices (ICC) of items and sum scores were calculated. The prevalence of single migration-related stressors in MIG ranged from 15% to 100% (CON 0–92%). All items of the MIGSTR10 (ICC 0.58–0.92) and the DSMSTR9 (ICC 0.56–0.96) reached high to very high interrater agreement (p < .0005). The item analysis of the MIGSTR10 revealed sufficient internal consistency (Cronbach’s α = 0.68/0.69) and only one item (“family conflicts”) without substantial correlation with the remaining scale. Correlation analyses showed a significant overlap of dimensional MIGSTR10 scores (r² = 0.25; p < .01) and DSMSTR9 scores (r² = 9%; p < .05) with GAF scores in MIG indicating functional relevance. MIGSTR10 is considered a feasible, economic, and reliable instrument for the assessment of stressors potentially related to migration.

Panel Ratings of Tape-Recorded Interview Responses: Interrater Reliability? Racial Differences?

2005 ◽  
Author(s):  
Patrick F. McKay ◽  
John R. Curtis ◽  
David J. Snyder ◽  
Robert C. Satterwhite
Keyword(s):  
Racial Differences ◽  
Interrater Reliability

Interrater reliability of a method to assess hypothalamic involvement in pediatric adamantinomatous craniopharyngioma

2020 ◽  
Vol 25 (1) ◽  
pp. 37-42 ◽  
Author(s):  
Ros Whelan ◽  
Eric Prince ◽  
David M. Mirsky ◽  
Robert Naftel ◽  
Aashim Bhatia ◽  
...  
Keyword(s):  
Quality Of Life ◽  
Brain Tumors ◽  
Interrater Reliability ◽  
Statistical Evaluation ◽  
Grading System ◽  
Reliability Study ◽  
Postoperative Mri ◽  
Mri Scans ◽  
Adamantinomatous Craniopharyngioma

OBJECTIVEPediatric adamantinomatous craniopharyngiomas (ACPs) are histologically benign brain tumors that confer significant neuroendocrine morbidity. Previous studies have demonstrated that injury to the hypothalamus is associated with worsened quality of life and a shorter lifespan. This insight helps many surgeons define the goals of surgery for patients with ACP. Puget and colleagues proposed a 3-tiered preoperative and postoperative grading system based on the degree of hypothalamic involvement identified on MRI. In a prospective cohort from their institution, the authors found that use of the system to guide operative goals was associated with decreased morbidity. To date, however, the Puget system has not been externally validated. Here, the authors present an interrater reliability study that assesses the generalizability of this system for surgeons planning initial operative intervention for children with craniopharyngiomas.METHODSA panel of 6 experts, consisting of pediatric neurosurgeons and pediatric neuroradiologists, graded 30 preoperative and postoperative MRI scans according to the Puget system. Interrater reliability was calculated using Fleiss’ κ and Krippendorff’s α statistics.RESULTSInterrater reliability in the preoperative context demonstrated moderate agreement (κ = 0.50, α = 0.51). Interrater reliability in the postoperative context was 0.27 for both methods of statistical evaluation.CONCLUSIONSInterrater reliability for the system as defined is moderate. Slight refinements of the Puget MRI grading system, such as collapsing the 3 grades into 2, may improve its reliability, making the system more generalizable.

Measuring interrater reliability among multiple raters: An example of methods for nominal data

1990 ◽  
Vol 9 (9) ◽  
pp. 1103-1115 ◽  
Author(s):  
Karen L. Posner ◽  
Paul D. Sampson ◽  
Robert A. Caplan ◽  
Richard J. Ward ◽  
Frederick W. Cheney
Keyword(s):  
Interrater Reliability ◽  
Nominal Data

scholarly journals The Family Psychoeducation Fidelity Scale: Psychometric Properties

2020 ◽  
Vol 47 (6) ◽  
pp. 894-900 ◽  
Author(s):  
I. Joa ◽  
J. O. Johannessen ◽  
K. S. Heiervang ◽  
A. A. Sviland ◽  
H. A. Nordin ◽  
...  
Keyword(s):  
Psychometric Properties ◽  
Internal Consistency ◽  
Interrater Reliability ◽  
Review Process ◽  
Trial Registration ◽  
Family Psychoeducation ◽  
Time Points ◽  
The Family ◽  
Fidelity Scale

Abstract This study examined psychometric properties and feasibility of the Family Psychoeducation (FPE) Fidelity Scale. Fidelity assessors conducted reviews using the FPE fidelity scale four times over 18 months at five sites in Norway. After completing fidelity reviews, assessors rated feasibility of the fidelity review process. The FPE fidelity scale showed excellent interrater reliability (.99), interrater item agreement (88%), and internal consistency (mean = .84 across four time points). By the 18-month follow-up, all five sites increased fidelity and three reached adequate fidelity. Fidelity assessors rated feasibility as excellent. The FPE fidelity scale has good psychometric properties and is feasible for evaluating the implementation of FPE programs. Trial registration ClinicalTrials.gov Identifier: NCT03271242.

Reliability of a Modified Medication Appropriateness Index in Community Pharmacies

2003 ◽  
Vol 37 (1) ◽  
pp. 40-46
Author(s):  
Rosemin Kassam ◽  
Linda G Martin ◽  
Karen B Farris ◽  
Homero A Monsanto ◽  
Jean-Marie Kaiser
Keyword(s):  
Community Pharmacy ◽  
Interrater Reliability ◽  
Intraclass Correlation ◽  
Community Setting ◽  
Medication Appropriateness Index ◽  
Paired Samples ◽  
Psychometric Data ◽  
Ambulatory Patients ◽  
Pharmacy Setting ◽  
Medication Appropriateness

Background The medication appropriateness index (MAI) has demonstrated reliability in selected outpatient clinics where medical data were easily accessible from medical charts. However, its use in the community setting where patient data may be limited has not been examined. Objective To evaluate the usefulness of a modified MAI for use in the community pharmacy setting by testing interrater reliability using 3 different rating schemes. Methods Two raters evaluated 160 medications for 32 elderly ambulatory patients. Patient information was acquired using community pharmacist-collected medication histories. A summated MAI score, percent agreement, κ, positive agreement, negative agreement, and intraclass correlation coefficient were calculated for each criterion using 3 scoring schemes. A paired samples t-test (95% CI) was used to test interrater reliability. Results The κ statistics were >0.75 for indication and effectiveness, but good (0.41–0.66) for the remaining criteria using the Hanlon scoring scheme. The intraclass coefficients (0.82, 0.86, 0.87) and overall κ (0.65, 0.66, 0.61) were similar for the 3 schemes. Conclusions This study suggests that the modified MAI has the potential to detect medication appropriateness and inappropriateness in the community pharmacy setting; however, it is not without limitations. Because the MAI has the most clinimetric and psychometric data available, the instrument should be studied further to increase its reliability and generalizability.

Sign in / Sign up

Export Citation Format

Share Document