scholarly journals Development and Psychometric Testing of Coding Scheme for Dementia Family Communication Using Video Observation

2021 ◽  
Vol 5 (Supplement_1) ◽  
pp. 747-747
Author(s):  
Sohyun Kim
Keyword(s):  
Social Interaction ◽  
Psychometric Properties ◽  
Family Caregivers ◽  
Dementia Care ◽  
Video Observation ◽  
Family Interactions ◽  
Cohen’S Kappa ◽  
Rater Reliability ◽  
Coding Scheme ◽  
Cohen's Kappa

Abstract Understanding communication behaviors between persons living with dementia and family caregivers is essential for meaningful social interaction and decrease problematic behaviors and caregiving burden. The purpose of this study was to develop and test the psychometric properties of a coding scheme for dementia care interactions. The coding scheme items were developed from literature and expert review, and the pilot testing on 16 video-recorded interactions. A secondary analysis was conducted using 77 videos from 21 dyads of dementia family interactions naturally occurred in the participant’s home. The final coding scheme consists of 11 codes for persons living with dementia (6 nonverbal and 5 verbal) and 12 codes for family caregivers (7 nonverbal and 5 verbal). Content validity was excellent (I-CVI = .93, S-CVI/UA = .71, S-CVI/Ave = .93 with 6 experts). Inter-item correlation was acceptable for both caregiver codes (positive nonverbal = .21, positive verbal = .15, negative nonverbal = .36, negative verbal = .29), and patient codes (positive nonverbal = .13, positive verbal = .27, negative nonverbal = .15, negative verbal = .18). Intra-rater reliability (Cohen’s Kappa = .83, percentage of agreement = 83.88%) and inter-rater reliability (Cohen’s Kappa = .81, percentage of agreement = 81.75%) were excellent. Findings suggest the preliminary psychometric properties of the newly developed coding scheme to assess dyadic interactions of persons living with dementia and their informal caregiver in-home care situations. Future testing of the coding scheme for application in communication interventions to improve quality social interaction in dementia care is discussed.

scholarly journals Core Conflictual Relationship Theme: The reliability of a simplified scoring procedure.

2019 ◽  
Author(s):  
Peter Tallberg ◽  
Randi Ulberg ◽  
Hanne-Sofie Johnsen Dahl ◽  
Per Andreas Høglend
Keyword(s):  
Case Formulation ◽  
Validity And Reliability ◽  
Cohen’S Kappa ◽  
Rater Reliability ◽  
Core Conflictual Relationship Theme ◽  
Cohen's Kappa ◽  
Simplified Method ◽  
Core Conflictual Relationship ◽  
Therapy Sessions ◽  
Conflictual Relationship

Abstract Background: Creating a case formulation is an important and basic skill in psychotherapy meant to guide treatment. A patient’s interpersonal pattern is an essential part of a case formulation. Core Conflictual Relationship Theme (CCRT) is a well-known structured method to describe interpersonal patterns. The CCRT method is based on the assumption that humans display a central relationship theme, which is shown in most relationships as well as in the patient-therapist relation. The CCRT scoring is based on how the patient describes interactions with others, in therapy sessions or in a specific interview. These descriptions are transcribed. Raters then score the identified relational episodes by choosing elements from the clustered categories of Wishes, Response from Others and Response from Self. The method has shown high validity and reliability. Inter rater reliability is generally good: Cohen’s kappa ranging from 0.55 to 0.70. To decide CCRT pattern from transcribed material is time consuming and labour intensive This study investigates a labour- and timesaving version of the method.Methods: This study aimed to investigate rater agreement in a simplified method of scoring the CCRT, based directly on live semi-structured dynamic interviews without transcribing the material. 52 patients referred for psychotherapy in a clinical trial, were scored for CCRT pattern. Based on information that came forth during the two-hour interview, raters scored the patients choosing elements from the clustered categories of Wishes, Response from Others and Response from Self. More than one category in each component could be chosen without ranking. 5 raters compared two by two were investigated. Inter rater reliability was measured by Cohen’s kappa.Results: Mean kappa for Wishes, Response from Others and Response from Self was .33, .44 and .45 respectively. Mean kappa for CCRT in total was .44 among 5 raters.Conclusion: In this simplified method to score the CCRT based on oral dynamic interviews, fair to moderate IRR was obtained.

InterRAI Acute Care Instrument for Seniors in Canadian Hospitals: Findings of an Inter-Rater Reliability Pilot Study

2020 ◽  
pp. 084456212092051
Author(s):  
Veronique Boscart ◽  
Linda Sheiban Taucar ◽  
Michelle Heyer ◽  
Tabitha Kellendonk ◽  
Keia Johnson ◽  
...  
Keyword(s):  
Older Adults ◽  
Acute Care ◽  
Adverse Outcomes ◽  
Cohen’S Kappa ◽  
Nursing Assessment ◽  
Kappa Score ◽  
Rater Reliability ◽  
Cohen's Kappa ◽  
Interrai Acute Care ◽  
Hospitalized Older Adults

Background Older adults are the biggest users of emergency departments and hospitals. However, healthcare professionals are often ill equipped to conduct comprehensive geriatric assessments causing missed opportunities for preventing adverse outcomes. Purpose To evaluate the inter-rater reliability of the interRAI Acute Care (AC) instrument for hospitalized older adults in two acute care hospitals in Ontario, Canada. Methods This descriptive study focused on evaluating the interRAI AC instrument, which was designed to facilitate a comprehensive nursing assessment for hospitalized seniors. Sample characteristics were described, and Cohen’s Kappa was calculated to derive the inter-rater reliability. Assessment times to complete the instrument were collected as well. Results The Cohen’s Kappa score for the instrument was 0.96. Many older adults who were interviewed had several challenges, including multimorbidity, polypharmacy, and lack of home support. The average time required for nurses to complete the interRAI AC instrument was 22 min. Conclusions The interRAI AC instrument is reliable for use by trained nurses to conduct a comprehensive assessment. This instrument offers a standardized and efficient approach to assess for care and intervention priorities and could prevent adverse outcomes in hospitalized older adults.

scholarly journals A standardised tool for assessing needs in forensic psychiatric population: clinical validation of the Italian CANFOR, staff version

2014 ◽  
Vol 24 (3) ◽  
pp. 274-281 ◽  
Author(s):  
L. Castelletti ◽  
A. Lasalvia ◽  
E. Molinari ◽  
S. D. M. Thomas ◽  
E. Straticò ◽  
...  
Keyword(s):  
Psychometric Properties ◽  
Convergent Validity ◽  
Unmet Needs ◽  
Psychiatric Patients ◽  
Unmet Need ◽  
Italian Version ◽  
Cohen’S Kappa ◽  
Cohen's Kappa ◽  
Forensic Psychiatric Patients ◽  
Forensic Psychiatric

Background.The Camberwell Assessment of Need – Forensic Version (CANFOR) is a standardised assessment tool specifically designed to assess needs for care in forensic psychiatric populations. The original English version of the instrument has shown good psychometric properties. The aim of this study was to validate the Italian version of the CANFOR-staff tool.Method.After translation and back-translation, the Italian CANFOR tool was administered to a sample of 50 forensic psychiatric patients. Convergent validity was tested using the Brief Psychiatric Rating Scale (BPRS) and the Global Assessment of Functioning (GAF) by applying Kendall's tau-b. Inter-rater and test–retest reliabilities were measured by ICCs for need dimensions (total and unmet) and Cohen's kappa coefficients for individual need items.Results.Regarding convergent validity, a higher number of needs (total and unmet) were associated with more severe psychiatric symptoms (BPRS). Higher numbers of unmet needs were also associated with lower levels of global functioning (GAF). ICCs for total and unmet needs scores indicated a good level of agreement for inter-rater reliability and a very good level for test–retest, respectively. Regarding the specific items, inter-rater Cohen's kappa was high (moderate to very good agreement) for 18 items in relation to the presence of a need and for 15 items in the rating of an unmet need, whereas Cohen's kappa for test–retest reliability was very high for all the items in the presence of a need and high for 18 of the unmet need domains.Conclusions.The Italian version of CANFOR has adequate psychometric properties. It can be considered a promising instrument for the assessment of needs of forensic psychiatric patients.

scholarly journals Application of the intervention Complexity Assessment Tool for Systematic Reviews within a Cochrane review: an illustrative case study

2020 ◽  
Vol 3 ◽  
pp. 31
Author(s):  
Cathal A. Cadogan ◽  
Audrey Rankin ◽  
Simon Lewin ◽  
Carmel M. Hughes
Keyword(s):  
Systematic Reviews ◽  
Assessment Tool ◽  
Cochrane Review ◽  
Guidance Document ◽  
Cohen’S Kappa ◽  
Rater Reliability ◽  
Cohen's Kappa ◽  
Holistic Understanding ◽  
Complexity Assessment

Background: The intervention Complexity Assessment Tool for Systematic Reviews (iCAT_SR) has been developed to facilitate detailed assessments of intervention complexity in systematic reviews. Worked examples of the tool’s application are needed to promote its use and refinement. The aim of this case study was to apply the iCAT_SR to a subset of 20 studies included in a Cochrane review of interventions aimed at improving appropriate polypharmacy in older people. Methods: Interventions were assessed independently by two authors using the six core iCAT_SR dimensions: (1) ‘Target organisational levels/categories’; (2) ‘Target behaviour/actions’; (3) ‘Active intervention components’; (4) ‘Degree of tailoring’; (5) ‘Level of skill required by intervention deliverers’; (6) ‘Level of skill required by intervention recipients’. Attempts were made to apply four optional dimensions: ‘Interaction between intervention components’; ‘Context/setting’; ‘Recipient/provider factors’; ‘Nature of causal pathway’. Inter-rater reliability was assessed using Cohen’s Kappa coefficient. Disagreements were resolved by consensus discussion. The findings are presented narratively. Results: Assessments involving the core iCAT_SR dimensions showed limited consistency in intervention complexity across included studies, even when categorised according to clinical setting. Interventions were delivered across various organisational levels and categories (i.e. healthcare professionals and patients) and typically comprised multiple components. Intermediate skill levels were required by those delivering and receiving the interventions across all studies. A lack of detail in study reports precluded application of the iCAT_SR’s optional dimensions. The inter-rater reliability was substantial (Cohen's Kappa = 0.75) Conclusions: This study describes the application of the iCAT_SR to studies included in a Cochrane systematic review. Future intervention studies need to ensure more detailed reporting of interventions, context and the causal pathways underlying intervention effects to allow a more holistic understanding of intervention complexity and facilitate replication in other settings. The experience gained has helped to refine the original guidance document relating to the application of iCAT_SR.

Core Conflictual Relationship Theme: The reliability of a simplified scoring procedure. 

2020 ◽  
Author(s):  
Peter Tallberg ◽  
Randi Ulberg ◽  
Hanne-Sofie Johnsen Dahl ◽  
Per Andreas Høglend
Keyword(s):  
Case Formulation ◽  
Validity And Reliability ◽  
Cohen’S Kappa ◽  
Rater Reliability ◽  
Core Conflictual Relationship Theme ◽  
Cohen's Kappa ◽  
Simplified Method ◽  
Registration Number ◽  
Core Conflictual Relationship ◽  
Conflictual Relationship

Abstract Background: Creating a case formulation is an important and basic skill in psychotherapy meant to guide treatment. A patient’s interpersonal pattern is an essential part of a case formulation. Core Conflictual Relationship Theme (CCRT) is a well-known structured method to describe interpersonal patterns. The CCRT method is based on the assumption that humans display a central relationship theme, which is shown in most relationships as well as in the patient-therapist relation. The CCRT scoring is based on how the patient describes interactions with others, in therapy sessions or in a specific interview. These descriptions are transcribed. Raters then score the identified relational episodes by choosing elements from the clustered categories of Wishes, Response from Others and Response from Self. The method has shown high validity and reliability. Inter rater reliability is generally good: Cohen’s kappa ranging from 0.55 to 0.70.To decide CCRT pattern from transcribed material is time consuming and labour intensive This study investigates a labour- and timesaving version of the method.Methods: This study aimed to investigate rater agreement in a simplified method of scoring the CCRT, based directly on live semi-structured dynamic interviews without transcribing the material. 52 patients referred for psychotherapy in a clinical trial, were scored for CCRT pattern. Based on information that came forth during the two-hour interview, raters scored the patients choosing elements from the clustered categories of Wishes, Response from Others and Response from Self. More than one category in each component could be chosen without ranking. 5 raters compared two by two were investigated. Inter rater reliability was measured by Cohen’s kappa.Results: Mean kappa for Wishes, Response from Others and Response from Self was .33, .44 and .45 respectively. Mean kappa for CCRT in total was .44 among 5 raters.Conclusion: In this simplified method to score the CCRT based on oral dynamic interviews, fair to moderate IRR was obtained. Trial Registration: First Experimental Study of Transference-interpretations (FEST307/95) Registration number: ClinicalTrials.gov Identifier: NCT00423462.

scholarly journals Using Video Recording in Evaluating Skills of Medical Students in the Performance of the Orthopedic Examination

2021 ◽  
Vol 55 (3) ◽  
Author(s):  
Jose Ma D. Bautista ◽  
Peter B. Bernardo ◽  
Mark Anthony R. Ruanto
Keyword(s):  
Clinical Skills ◽  
Video Recording ◽  
Kappa Coefficient ◽  
Cohen’S Kappa ◽  
Perfect Agreement ◽  
Rater Reliability ◽  
Video Recordings ◽  
Cohen's Kappa ◽  
Improve Reliability ◽  
Cohen’S Kappa Coefficient

Objective. The study aims to assess the similarity between the results of the evaluation of students during an Objective Structured Clinical Examination (OSCE) and a video recording of the same OSCE (VOSCE). Methods. All Orthopedic surgeon preceptors in the actual OSCE were recruited to the study. Video recordings of the students taking the OSCE were collected and later reviewed and re-evaluated by the same preceptor after at least four weeks. The grades of actual OSCE and VOSCE were collected and analyzed using Cohen’s kappa coefficient. Results. High variability of intra-rater reliability was observed in different preceptors and station (slight agreement to perfect agreement). Overall intra-rater reliability between actual and video OSCE showed moderate agreement with Cohen’s kappa coefficient equal to 0.43 (n-219). Conclusion. Video OSCE is a reliable tool in assessing student clinical skills and knowledge in the musculoskeletal examination. Some factors have been suggested to further improve reliability.

5.14 Psychometric Properties of a Novel Vineland-II 2-Domain Composite Score to Assess Social Communication and Social Interaction in ASD

2018 ◽  
Vol 57 (10) ◽  
pp. S231-S232
Author(s):  
Robert L. Findling ◽  
Stephanie Le Scouiller ◽  
Lisa Squassante ◽  
Kevin Sanders ◽  
Janice Smith ◽  
...  
Keyword(s):  
Social Interaction ◽  
Psychometric Properties ◽  
Social Communication ◽  
Composite Score

scholarly journals Family caregiver emotional distress in advanced cancer: the DME-C scale psychometric properties

2020 ◽  
pp. bmjspcare-2020-002608
Author(s):  
Joaquín T Limonero ◽  
Jorge Maté-Méndez ◽  
María José Gómez-Romero ◽  
Dolors Mateo-Ortega ◽  
Jesús González-Barboteo ◽  
...  
Keyword(s):  
Psychometric Properties ◽  
End Of Life ◽  
Family Caregivers ◽  
Advanced Cancer ◽  
Emotional Distress ◽  
Family Caregiver ◽  
Screening Tools ◽  
Anxiety And Depression ◽  
The Family ◽  
End Stage

BackgroundFamily caregivers of patients with advanced illness at end of life often report high levels of emotional distress. To address this emotional distress is necessary to have adequate and reliable screening tools.AimThis study analyses the psychometric properties and clinical utility of the Family Caregiver Emotional Detection Scale for caregivers of patients with end-stage cancer (DME-C, Spanish acronym) who are receiving palliative care (PC).DesignMulticentre, cross-sectional study.Settings/participantsFamily caregivers of patients with advanced cancer at end of life receiving palliative treatment were interviewed to explore their emotional distress through the DME-C scale and other instruments measuring anxiety and depression (Hospital Anxiety and Depression Scale (HADS)), distress thermometer (DT) and overload (B), as well as a clinical psychological assessment (CPA).Results138 family caregivers, 85 (61.6%) female and 53 (38.4%) male, with an average age of 59.69±13.3 participated in the study. The reliability of the scale, as measured by Cronbach’s alpha, was 0.76, and its stability over time was 0.734. Positive, significant correlations were found between the DME-C and the scores for anxiety and depression registered on the HADS scale, as well as with the total result of this latter scale and the results for B, the DT and the CPA. A statistical analysis of the receiver-operating characteristic curves showed that the scale has a sensitivity and specificity of 75%, and that the cut-off point for the detection of emotional distress was a score ≥11. Fifty-four per cent of the caregivers displayed emotional distress according to this scale.ConclusionsThe DME-C displays good psychometric properties. It is simple, short, reliable and easy to administer. We believe that the instrument is useful for the detection of emotional distress in the family caregivers of hospitalised patients suffering from end-stage illnesses and receiving PC.

scholarly journals Measuring Caregiver Person-Centered Communication for Persons With Dementia via Video Observations

2020 ◽  
Vol 4 (Supplement_1) ◽  
pp. 152-152
Author(s):  
Ying-Ling Jao ◽  
Carissa Coleman ◽  
Kristine Williams ◽  
Diane Berish ◽  
Wen Liu ◽  
...  
Keyword(s):  
Sequential Analysis ◽  
Controlled Trial ◽  
Verbal Communication ◽  
Rater Reliability ◽  
Coding Scheme ◽  
Video Observations ◽  
Interactive Nature ◽  
The Impact ◽  
Mealtime Interactions ◽  
And Task

Abstract Communication is fundamental for daily care activities in nursing homes (NHs). Second-by-second behavioral coding of video observations is an ideal approach to examine the interactive nature of communication but requires a reliable coding scheme. Recent studies have adapted the Peron-Centered Behavioral Inventory (PCBI) and Task-Centered Behavioral Inventory (TCBI) to analyze caregiver communication during mealtime interactions, but their use for coding general daily caregiving activities has not been widely evaluated. This pilot study adapted the PCBI and TCBI of video observations and determined their inter-rater reliability for measuring caregiver verbal communication with persons with dementia (PwD). We analyzed videos from a randomized controlled trial of an intervention to improve caregiver communication in NHs. We selected one 1-minute segment from 12 videos that included interactions of caregiver-resident dyads. One research assistant transcribed caregivers’ verbal communication and segmented the communication into utterances. Two other research assistants independently coded each utterance using the adapted PCBI and TCBI. The coding scheme was expanded by modifying the existing operational definitions, adding three new codes, and developing a coding decision guide. Residents were Caucasian (100%), mean age 86 years with dementia and resistive behaviors. The adapted PCBI and TCBI had an inter-rater reliability of Kappa=0.656 (p<.001) across the 12 videos. Overall, our adapted PCBI and TCBI showed substantial inter-rater reliability. The results support the use of our adapted PCBI and TCBI to distinguish between person-centered and task-centered communication in video observations, which, in turn, allows for sequential analysis to examine the impact of caregiver communication on PwD.

Sign in / Sign up

Export Citation Format

Share Document