Psychometric properties of the Cumulated Ambulation Score French translation

Objective: To examine measurement properties (inter-rater reliability, agreement, validity, and responsiveness) of the Cumulated Ambulation Score French translation in patients with hip fracture. Design: Methodological study. Setting: A 20-bed orthopedic unit and 20-bed geriatric unit. Subjects: About 140 consecutive patients with a mean (SD) age of 83 (12) years. Interventions: The English version of the Cumulated Ambulation Score used to evaluate basic mobility was translated into French following international guidelines. Two raters independently assessed all patients on postoperative days one, two, three, and 30 after a hip fracture surgery. Relative and absolute inter-rater reliability and responsiveness (effect size for improvement from postoperative day two to 30) were evaluated. Convergent validity was analyzed by Spearman’s correlation coefficient comparing the Cumulated Ambulation Score with two other measures on postoperative day two and 30. Main measure: Cumulated Ambulation Score. Results: The weighted Kappa value ranged from 0.89 to 1.0. The standard error of measurement and the smallest real difference of the Cumulated Ambulation Score ranged, respectively, from 0.12 to 0.23 and from 0.32 to 0.6 points, while the effect size reached 1.03 (95% CI 0.87–1.26). There was a strong positive correlation comparing the Cumulated Ambulation Score with the French Tinetti Assessment Tool ( r ⩾ 0.83) and the French Mini Motor Test ( r ⩾ 0.79). Conclusions: Our findings indicate that the Cumulated Ambulation Score’s French version is a reliable and valid tool to assess patients’ basic mobility with hip fractures.

Download Full-text

Measurement properties of the Physical Literacy Assessment for Youth (PLAY) Tools

Applied Physiology Nutrition and Metabolism ◽

10.1139/apnm-2020-0648 ◽

2020 ◽

Author(s):

Hilary Ann Toccacelli Caldwell ◽

Natascja A Di Cristofaro ◽

John Cairney ◽

Steven R Bray ◽

Brian W. Timmons

Keyword(s):

Internal Consistency ◽

Convergent Validity ◽

Measurement Properties ◽

School Age Children ◽

School Age ◽

Literacy Assessment ◽

Physical Literacy ◽

Rater Reliability ◽

Method Of Evaluation ◽

Acceptable Internal Consistency

The Physical Literacy Assessment for Youth (PLAY) Tools are a suite of tools to assess an individual’s physical literacy. The purpose of this study is to examine the psychometric properties of the PLAY Tools, including inter-rater reliability, internal consistency, validity and the associations between the tools. In this study, 218 children and youth (8.4- to- 13.7-years old) and a parent/ guardian completed the appropriate physical literacy assessments (i.e., PLAYbasic, PLAYfun, PLAYparent and PLAYself) and the Bruiniks-Oseretsky Test of Motor Proficiency (BOT-2). Inter-rater reliability for PLAYfun was excellent (ICC=0.94). The PLAYbasic, PLAYfun total, running and object control scores, and PLAYparent motor competence domain were higher in males than females, and PLAYfun locomotor skills were lower in males than females (p<0.05). Age was positively correlated with PLAYbasic and PLAYfun (r=0.14-0.32, p<0.05). BOT-2 was positively correlated with PLAYfun and PLAYbasic (r=0.19-0.59, p<0.05). PLAYbasic is a significant predictor of PLAYfun (R<sup>2</sup>=0.742, p<0.001). PLAYfun, PLAYparent and PLAYself were moderately correlated with one another. PLAYfun, PLAYparent and PLAYself demonstrated acceptable internal consistency (α=0.74-0.87, ω=0.73-0.87). The PLAY Tools demonstrated moderate associations between one another, strong inter-rater reliability and good construct and convergent validity. Continued evaluation of these tools with other populations, such as adolescents, is necessary. •In school-age children, the PLAY Tools demonstrated strong inter-rater reliability, moderate associations with one another, acceptable internal consistency and good construct and convergent validity.•The results suggest that that PLAY Tools are an acceptable method of evaluation for physical literacy in school-age children.

Download Full-text

Inter-rater reliability of paediatric emergency assessment: physiological and clinical features

Archives of Disease in Childhood ◽

10.1136/archdischild-2019-318664 ◽

2020 ◽

pp. archdischild-2019-318664

Author(s):

Calvin Heal ◽

Sarah Cotterill ◽

Andrew Graeme Rowland ◽

Natalie Garratt ◽

Tony Long ◽

...

Keyword(s):

Clinical Features ◽

Emergency Departments ◽

Assessment Tool ◽

Vital Signs ◽

Scoring Systems ◽

Weighted Kappa ◽

Urgent Care ◽

Convenience Sample ◽

Rater Reliability ◽

North West

ObjectiveThe Paediatric Admission Guidance in the Emergency Department (PAGE) score is an assessment tool currently in development that helps predict hospital admission using components including patient characteristics, vital signs (heart rate, temperature, respiratory rate and oxygen saturation) and clinical features (eg, breathing, behaviour and nurse judgement). It aims to assist in safe admission and discharge decision making in environments such as emergency departments and urgent care centres. Determining the inter-rater reliability of scoring tools such as PAGE can be difficult. The aim of this study was to determine the inter-rater reliability of seven clinical components of the PAGE Score.DesignInter-rater reliability was measured by each patient having their clinical components recorded by two separate raters in succession. The first rater was the assessing nurse, and the second rater was a research nurse.SettingTwo emergency departments and one urgent care centre in the North West of England. Measurements were recorded over 1 week; data were collected for half a day at each of the three sites.PatientsA convenience sample of 90 paediatric attendees (aged 0–16 years), 30 from each of the three sites.Main outcome measuresTwo independent measures for each child were compared using kappa or prevalence-adjusted bias-adjusted kappa (PABAK). Bland-Altman plots were also constructed for continuous measurements.ResultsInter-rater reliability ranged from moderate (0.62 (95% CI 0.48 to 0.74) weighted kappa) to very good (0.98 (95% CI 95 to 0.99) weighted kappa) for all measurements except ‘nurse judgement’ for which agreement was fair (0.30, 95% CI 0.09 to 0.50 PABAK). Complete information from both raters on all the clinical components of the PAGE score were available for 73 children (81%). These total scores showed good’ inter-rater reliability (0.64 (95% CI 0.53 to 0.74) weighted kappa).ConclusionsOur findings suggest different nurses would demonstrate good inter-rater reliability when collecting acute assessments needed for the PAGE score, reinforcing the applicability of the tool. The importance of determining reliability in scoring systems is highlighted and a suitable methodology was presented.

Download Full-text

Development of the Huddle Observation Tool for structured case management discussions to improve situation awareness on inpatient clinical wards

BMJ Quality & Safety ◽

10.1136/bmjqs-2017-006513 ◽

2017 ◽

Vol 27 (5) ◽

pp. 365-372 ◽

Cited By ~ 8

Author(s):

Julian Edbrooke-Childs ◽

Jacqueline Hayes ◽

Evelyn Sharples ◽

Dawid Gondek ◽

Emily Stapley ◽

...

Keyword(s):

Case Management ◽

Correlation Coefficient ◽

Intraclass Correlation Coefficient ◽

Situation Awareness ◽

Assessment Tool ◽

Intraclass Correlation ◽

Weighted Kappa ◽

Observational Assessment ◽

Rater Reliability ◽

Team Processes

Background‘Situation Awareness For Everyone’ (SAFE) was a 3-year project which aimed to improve situation awareness in clinical teams in order to detect potential deterioration and other potential risks to children on hospital wards. The key intervention was the ‘huddle’, a structured case management discussion which is central to facilitating situation awareness. This study aimed to develop an observational assessment tool to assess the team processes occurring during huddles, including the effectiveness of the huddle.MethodsA cross-sectional observational design was used to psychometrically develop the ‘Huddle Observation Tool’ (HOT) over three phases using standardised psychometric methodology. Huddles were observed across four NHS paediatric wards participating in SAFE by five researchers; two wards within specialist children hospitals and two within district general hospitals, with location, number of beds and length of stay considered to make the sample as heterogeneous as possible. Inter-rater reliability was calculated using the weighted kappa and intraclass correlation coefficient.ResultsInter-rater reliability was acceptable for the collaborative culture (weighted kappa=0.32, 95% CI 0.17 to 0.42), environment items (weighted kappa=0.78, 95% CI 0.52 to 1) and total score (intraclass correlation coefficient=0.87, 95% CI 0.68 to 0.95). It was lower for the structure and risk management items, suggesting that these were more variable in how observers rated them. However, agreement on the global score for huddles was acceptable.ConclusionWe developed an observational assessment tool to assess the team processes occurring during huddles, including the effectiveness of the huddle. Future research should examine whether observational evaluations of huddles are associated with other indicators of safety on clinical wards (eg, safety climate and incidents of patient harm), and whether scores on the HOT are associated with improved situation awareness and reductions in deterioration and adverse events in clinical settings, such as inpatient wards.

Download Full-text

CODEM Instrument

GeroPsych ◽

10.1024/1662-9647/a000100 ◽

2014 ◽

Vol 27 (1) ◽

pp. 23-31 ◽

Cited By ~ 4

Author(s):

Anne Kuemmel (This author contributed eq ◽

Julia Haberstroh (This author contributed ◽

Johannes Pantel

Keyword(s):

Convergent Validity ◽

Interrater Reliability ◽

Discriminant Validity ◽

Assessment Tool ◽

Intraclass Correlation ◽

Well Being ◽

Communication Behavior ◽

People With Dementia ◽

Pearson's R ◽

Pearson’S R

Communication and communication behaviors in situational contexts are essential conditions for well-being and quality of life in people with dementia. Measuring methods, however, are limited. The CODEM instrument, a standardized observational communication behavior assessment tool, was developed and evaluated on the basis of the current state of research in dementia care and social-communicative behavior. Initially, interrater reliability was examined by means of videoratings (N = 10 people with dementia). Thereupon, six caregivers in six German nursing homes observed 69 residents suffering from dementia and used CODEM to rate their communication behavior. The interrater reliability of CODEM was excellent (mean κ = .79; intraclass correlation = .91). Statistical analysis indicated that CODEM had excellent internal consistency (Cronbach’s α = .95). CODEM also showed excellent convergent validity (Pearson’s R = .88) as well as discriminant validity (Pearson’s R = .63). Confirmatory factor analysis verified the two-factor solution of verbal/content aspects and nonverbal/relationship aspects. With regard to the severity of the disease, the content and relational aspects of communication exhibited different trends. CODEM proved to be a reliable, valid, and sensitive assessment tool for examining communication behavior in the field of dementia. CODEM also provides researchers a feasible examination tool for measuring effects of psychosocial intervention studies that strive to improve communication behavior and well-being in dementia.

Download Full-text

Structured Interview for Hoarding Disorder (SIHD)

European Journal of Psychological Assessment ◽

10.1027/1015-5759/a000433 ◽

2019 ◽

Vol 35 (4) ◽

pp. 512-520

Author(s):

Caterina Novara ◽

Paolo Cavedini ◽

Stella Dorz ◽

Susanna Pardini ◽

Claudio Sica

Keyword(s):

Convergent Validity ◽

Structured Interview ◽

Diagnostic Process ◽

Schizophrenia Spectrum Disorders ◽

Clinical Environment ◽

Obsessive Compulsive ◽

Rater Reliability ◽

Hoarding Disorder ◽

Compulsive Disorder ◽

Schizophrenia Spectrum

Abstract. The Structured Interview for Hoarding Disorder (SIHD) is a semi-structured interview designed to assist clinicians in diagnosing a hoarding disorder (HD). This study aimed to validate the Italian version of the SIHD. For this purpose, its inter-rater reliability has been analyzed as well as its ability to differentiate HD from other disorders often comorbid. The sample was composed of 74 inpatients who had been diagnosed within their clinical environment: 9 with HD, 11 with obsessive-compulsive disorder (OCD) and HD, 22 with OCD, 19 with major depressive disorder (MDD), and 13 with schizophrenia spectrum disorders (SSD). The results obtained indicated “substantial” or “perfect” inter-rater reliability for all the core HD criteria, HD diagnosis, and specifiers. The SIHD differentiated between subjects suffering from and not suffering from a HD. Finally, the results indicated “good” convergent validity and high scores were shown in terms of both sensitivity and specificity for HD diagnosis. Altogether, the SIHD represents a useful instrument for evaluating the presence of HD and is a helpful tool for the clinician during the diagnostic process.

Download Full-text

Inter-rater Reliability of Preceptors on Clinical Pharmacy Competency Evaluation

Current Drug Therapy ◽

10.2174/1574885515999201209202624 ◽

2020 ◽

Vol 15 ◽

Author(s):

Dixon Thomas ◽

Sherief Khalifa ◽

Jayadevan Sreedharan ◽

Rucha Bond

Keyword(s):

United Arab Emirates ◽

Clinical Pharmacy ◽

Clinical Competence ◽

Assessment Tool ◽

Clinical Skills ◽

Group Discussion ◽

Pharmacy Students ◽

Rater Reliability ◽

Quasi Experimental ◽

Competency Evaluation

Background:: Clinical competence of pharmacy students is better evaluated at their practice sites. compared to the classroom. A clinical pharmacy competency evaluation rubric like that of the American College of Clinical Pharmacy (ACCP)is an effective assessment tool for clinical skills and can be used to show item reliability. The preceptors should be trained on how to use the rubrics as many inherent factors could influence inter-rater reliability. Objective:: To evaluate inter-rater reliability among preceptors on evaluating clinical competence of pharmacy students, before and after a group discussion intervention. Methods:: In this quasi experimental study in a United Arab Emirates teaching hospital, Seven clinical pharmacy preceptors rated clinical pharmacy competencies of ten recent PharmD graduates referring to their portfolios and preceptorship. Clinical pharmacy competencies were adopted from ACCP and mildly modified to be relevant for the local settings. Results:: Inter-rater reliability (Cronbach's Alpha) among preceptors was reasonable being practitioners at a single site for 2-4 years. At domain level, inter-rater reliability ranged from 0.79 - 0.93 before intervention and 0.94 - 0.99 after intervention. No inter-rater reliability was observed in relation to certain competency elements ranging from 0.31 – 0.61 before intervention, but improved to 0.79 – 0.97 after intervention. Intra-class correlation coefficient improved among all individual preceptors being reliable with each other after group discussion though some had no reliability with each other before group discussion. Conclusion:: Group discussion among preceptors at the training site was found to be effective in improving inter-rater reliability on all elements of the clinical pharmacy competency evaluation. Removing a preceptor from analysis did not affect inter-rater reliability after group discussion.

Download Full-text

The Inflammatory Bowel Disease Symptom Inventory: A Patient-report Scale for Research and Clinical Application

Inflammatory Bowel Diseases ◽

10.1093/ibd/izz038 ◽

2019 ◽

Vol 25 (8) ◽

pp. 1277-1290 ◽

Cited By ~ 7

Author(s):

Kathryn A Sexton ◽

John R Walker ◽

Laura E Targownik ◽

Lesley A Graff ◽

Clove Haviva ◽

...

Keyword(s):

Inflammatory Bowel Disease ◽

Factor Analysis ◽

Bowel Disease ◽

Convergent Validity ◽

Short Form ◽

Self Report ◽

Measurement Properties ◽

Systemic Complications ◽

Patient Reported ◽

Inflammatory Bowel

Abstract Objectives Existing measures of inflammatory bowel disease (IBD) symptoms are not well suited to self-report, inadequate in measurement properties, insufficiently specific, or burdensome for brief or repeated administration. We aimed to develop a patient-reported outcome measure to assess a broader range of IBD symptoms. Methods The IBD Symptoms Inventory (IBDSI) was developed by adapting symptom items from existing clinician-rated or diary-format inventories; after factor analysis, 38 items were retained on 5 subscales: bowel symptoms, abdominal discomfort, fatigue, bowel complications, and systemic complications. Participants completed the IBDSI and other self-report measures during a clinic visit. A nurse administered the Harvey Bradshaw Index (HBI) for Crohn’s disease (CD) or the Powell-Tuck Index (PTI) for ulcerative colitis (UC), and a gastroenterologist completed a global assessment of disease severity (PGA). Results The 267 participants with CD (n = 142) or UC (n = 125), ages 18 to 81 (M = 43.4, SD = 14.6) were 58.1% female, with a mean disease duration of 13.9 (SD = 10.5) years. Confirmatory factor analysis supported the 5 subscales. The total scale and subscales showed good reliability and significant correlations with self-report symptom and IBD quality of life measures, the HBI, PTI, and PGA. Conclusions The IBDSI showed strong measurement properties: a supported factor structure, very good internal consistency, convergent validity, and excellent sensitivity and specificity to clinician-rated active disease. Self-report HBI and PTI items, when extracted from this measure, produced scores comparable to clinician-administered versions. The 38-item IBDSI, or 26-item short form, can be used as a brief survey of common IBD symptoms in clinic or research settings.

Download Full-text

The measurement performance of the EQ-5D-5L versus EQ-5D-3L in patients with hidradenitis suppurativa

Quality of Life Research ◽

10.1007/s11136-020-02732-x ◽

2021 ◽

Cited By ~ 1

Author(s):

Alex Bató ◽

Valentin Brodszky ◽

L. Hunor Gergely ◽

Krisztián Gáspár ◽

Norbert Wikonkál ◽

...

Keyword(s):

Convergent Validity ◽

Hidradenitis Suppurativa ◽

Clinical Care ◽

Intraclass Correlation ◽

Life Quality ◽

Measurement Properties ◽

Economic Evaluations ◽

Health State ◽

Cross Sectional Survey ◽

Cross Sectional

Abstract Purpose Hidradenitis suppurativa (HS) is a chronic inflammatory skin disease that affects up to 1% of the population in Europe. The EQ-5D is the most commonly used generic instrument for measuring health-related quality of life among HS patients. This study aims to compare the measurement properties of the two adult versions of EQ-5D (EQ-5D-3L and EQ-5D-5L) in patients with HS. Methods We recruited 200 consecutive patients with HS (mean age 37 years, 38% severe or very severe HS) to participate in a multicentre cross-sectional survey. Patients completed the EQ-5D-3L, EQ-5D-5L, Dermatology Life Quality Index (DLQI) and Skindex-16 questionnaires. Results More than twice as many different health state profiles occurred in the EQ-5D-5L compared to the EQ-5D-3L (101 vs. 43). A significant reduction in ceiling effect was found for the mobility, self-care and usual activities dimensions. A good agreement was established between the EQ-5D-3L and EQ-5D-5L with an intraclass correlation coefficient of 0.872 (95% CI 0.830–0.903; p < 0.001) that was confirmed by a Bland-Altman plot. EQ-5D-5L improved both the absolute and relative informativity in all dimensions except for anxiety/depression. EQ-5D-3L and EQ-5D-5L demonstrated similar convergent validity with DLQI and Skindex-16. EQ-5D-5L was able to better discriminate between known groups of patients based on the number of comorbidities and disease severity (HS-Physician's Global Assessment). Conclusion In patients with HS, the EQ-5D-5L outperformed the EQ-5D-3L in feasibility, ceiling effects, informativity and known-groups validity for many important clinical characteristics. We recommend using the EQ-5D-5L in HS patients across various settings, including clinical care, research and economic evaluations.

Download Full-text

Citizenship matters: Translating and adapting the Citizenship Measure to Norwegian

Nordic Studies on Alcohol and Drugs ◽

10.1177/14550725211018604 ◽

2021 ◽

pp. 145507252110186

Author(s):

Linda Nesse ◽

Marianne Thorsen Gonzalez ◽

Michael Rowe ◽

Ruth Kjærsti Raanaas

Keyword(s):

Internal Consistency ◽

Convergent Validity ◽

Mental Health Problems ◽

Supported Housing ◽

Measurement Properties ◽

Convenience Sample ◽

Marginalised Groups ◽

Two Factors ◽

Back Translation ◽

High Internal Consistency

Citizenship is considered intertwined with recovery, and may be a useful perspective for advancing quality of life among marginalised groups. Yet, matters of citizenship among persons with co-occurring substance use and mental health problems both in research and practice. Aims: In order to measure citizenship among persons with co-occurring problems in a Norwegian study, a measure of citizenship was translated from English to Norwegian. The aims of the study were to 1) translate and adapt the Citizenship Measure, developed by Rowe and colleagues at the Yale Program for Recovery and Community Health, to Norwegian, and 2) to assess the internal consistency and convergent validity of the Norwegian translated measure. Methods: The translation process was carried out using forward and back translation procedures. To examine measurement properties, a convenience sample of 104 residents with co-occurring problems living in supported housing completed the measure. Results: Two factors were identified, related to rights, and to relational citizenship. The Norwegian translation of the Citizenship Measure showed has high internal consistency and adequate convergent validity. Conclusions: We argue that the measure can be useful in assessing perceived citizenship, and in initiating efforts to support citizenship among persons with co-occurring problems.

Download Full-text

Patient-reported anxiety and depression measures for use in Indian head and neck cancer populations: a psychometric evaluation

Journal of Patient-Reported Outcomes ◽

10.1186/s41687-021-00316-y ◽

2021 ◽

Vol 5 (1) ◽

Author(s):

Chindhu Shunmugasundaram ◽

Haryana M. Dhillon ◽

Phyllis N. Butow ◽

Puma Sundaresan ◽

Mahati Chittem ◽

...

Keyword(s):

Head And Neck ◽

Convergent Validity ◽

Psychometric Evaluation ◽

Measurement Properties ◽

Anxiety And Depression ◽

Factor Analyses ◽

Factor Solution ◽

Confirmatory Factor Analyses ◽

Culturally Adapted ◽

Confirmatory Factor

Abstract Background Head and neck cancers (HNC) are one of the most traumatic forms of cancer because they affect essential aspects of life such as speech, swallowing, eating and disfigurement. HNCs are common in India, with over 100,000 cases being registered each year. HNC and treatment are both associated with considerable anxiety and depression. With increasing multinational research, no suitable measures in Indian languages are available to assess anxiety and depression in Indian HNC patients. This study evaluated the psychometric properties of cross-culturally adapted versions of Zung’s self-rating Anxiety Scale (SAS) and the Patient health questionnaire – 9 (PHQ-9) in Tamil, Telugu and Hindi speaking Indian HNC populations. Methods HNC patients were recruited from three tertiary cancer centres in India. Patients completed the cross-culturally adapted versions of SAS and PHQ-9. We assessed targeting, scaling assumptions, construct validity (exploratory and confirmatory factor analyses), convergent validity, and internal consistency reliability. Results The study sample included 205 Tamil, 216 Telugu and 200 Hindi speaking HNC patients. Exploratory and confirmatory factor analyses indicated a two-factor solution for PHQ-9 and four-factor solution for SAS in all three languages. Cronbach’s alpha coefficients ranged between 0.717 and 0.890 for PHQ-9 and between 0.803 and 0.868 for SAS, indicating good reliability. Correlations between hypothesized scales were as expected providing evidence towards convergent validity. Conclusions This first psychometric evaluation of the measurement properties of Tamil, Telugu and Hindi versions of the SAS and PHQ-9 in large, Indian HNC populations supported their use as severity and outcome measures across the disease and treatment continuum.

Download Full-text