Assessing Therapeutic Competence in Cognitive Therapy for Social Phobia: Psychometric Properties of the Cognitive Therapy Competence Scale for Social Phobia (CTCS-SP)

Background: There has been considerable acknowledgement in treatment outcome research that, although the assessment of treatment integrity is essential in many respects, it requires great effort as well as resources and is therefore often neglected. Aims: In order to fill this gap, the Cognitive Therapy Competence Scale for Social Phobia (CTCS-SP) was developed, based on the Cognitive Therapy Scale, to measure therapist competence in delivering cognitive therapy for social phobia. The aim of the present study was to investigate interrater reliability, internal consistency and retest reliability of the scale. Method: Raters evaluated therapist competence from 161 videotaped sessions (98 patients) selected from 234 cognitive treatments within a multi-centre study. Results: Interrater-reliability was found to be high for the overall score (ICC = .81) and moderate for individual items (ICC = .62–.92). Internal consistency and retest reliability were also found to be high (Cronbach's alpha = .89; (ICCretest = .86). Conclusions: The results indicate that the CTCS-SP is highly reliable. As even individual items yield satisfactory reliability, the scale can be used in various fields of research, including the measurement of changes in skill acquisition and the impact of competence on outcome criteria.

Download Full-text

The Mandarin Voice Handicap Index for Laryngectomees With Pneumatic Artificial Laryngeal and Esophageal Speech

American Journal of Speech-Language Pathology ◽

10.1044/2021_ajslp-20-00241 ◽

2021 ◽

pp. 1-12

Author(s):

Chen-Chi Wang ◽

Jia-Shiou Liao ◽

Hsiu-Chin Lai ◽

Yi-Hsuan Lo

Keyword(s):

Internal Consistency ◽

Intraclass Correlation ◽

Correlation Coefficients ◽

Voice Handicap Index ◽

Good Test ◽

Retest Reliability ◽

Related Quality ◽

Valid Instrument ◽

Test Retest Reliability ◽

The Impact

Purpose The Voice Handicap Index (VHI) questionnaire assesses the impact of total laryngectomy on voice-related quality of life. This study evaluates the Mandarin VHI, including its internal consistency, test–retest reliability, content validity, and differences in scores for Mandarin alaryngeal patients with pneumatic artificial laryngeal (PA) and esophageal (ES) speech. Method Translation and validation of the VHI questionnaire was performed through the forward–backward translation technique. This study used a sample of 78 PA and 23 ES participants from Taiwan who completed the Mandarin VHI. Forty-two of the alaryngeal participants completed the Mandarin VHI twice over a period of 7–63 days. Results The measurement of the internal consistency of the Mandarin VHI showed a high Cronbach's alpha coefficient for the total score (.975) and the functional (.930), physical (.939), and emotional (.938) subscales. Based on the results of the intraclass correlation coefficients, good test–retest reliability for the total and domain scores was found (intraclass correlation coefficient = .827–.863). Conclusion The Mandarin VHI was validated as an instrument with proper internal consistency and reliability, which supports the Mandarin VHI as a valid instrument for the self-evaluation of handicaps related to voice problems in PA and ES speakers.

Download Full-text

Development and use of a Biological Rhythm Interview

European Psychiatry ◽

10.1016/s0924-9338(09)70793-x ◽

2009 ◽

Vol 24 (S1) ◽

pp. 1-1 ◽

Cited By ~ 1

Author(s):

L.M. Giglio ◽

P.V.d.S. Magalhães ◽

A. Andreazza ◽

J. Walz ◽

L. Jakobsen ◽

...

Keyword(s):

Bipolar Disorder ◽

Internal Consistency ◽

Biological Rhythms ◽

Biological Rhythm ◽

Eating Pattern ◽

Retest Reliability ◽

New Instrument ◽

Social Rhythm ◽

Test Retest Reliability ◽

The Impact

Introduction:As several lines of evidence point to irregular biological rhythms in bipolar disorder, and its disruption may lead to new illness episodes, having an instrument that measures biological rhythms is critical. This report describes the validation of a new instrument, the Biological Rhythms Interview of Assessment in Neuropsychiatry (BRIAN), designed to assess biological rhythms in the clinical setting.Methods:Eighty-one outpatients with a diagnosis of bipolar disorder and 79 control subjects matched for type of health service used, sex, age and educational level were consecutively recruited. After a pilot study, 18 items evaluating sleep, activities, social rhythm and eating pattern were probed for discriminant, content and construct validity, concurrent validity with the Pittsburgh Sleep Quality Index (PSQI), internal consistency and test-retest reliability.Results:A three-factor solution, termed sleep/ social rhythm factor, activity factor and feeding factor, provided the best theoretical and most parsimonious account of the data; items essentially loaded in factors as theoretically intended, with the exception of the sleep and social scales, which formed a single factor. Test-retest reliability and internal consistency were excellent. Highly significant differences between the two groups were found for the whole scale and for each BRIAN factor. Total BRIAN scores were highly correlated with the global PSQI score.Discussion:The BRIAN scale presents a consistent profile of validity and reliability. Its use may help clinicians to better assess their patients and researchers to improve the evaluation of the impact of novel therapies targeting biological rhythm pathways.

Download Full-text

Development of Chinese Versions of Quality of Life in Late-Stage Dementia and Cognitive Tests for Severe Dementia

Dementia and Geriatric Cognitive Disorders Extra ◽

10.1159/000511703 ◽

2020 ◽

pp. 172-181

Author(s):

Suet-Lai Leung ◽

Hiroyuki Tanaka ◽

Timothy C.Y. Kwok

Keyword(s):

Quality Of Life ◽

Older Adults ◽

Internal Consistency ◽

Interrater Reliability ◽

Severe Dementia ◽

Good Test ◽

Retest Reliability ◽

Chinese Older Adults ◽

Test Retest Reliability

Introduction: Valid assessments of quality of life (QoL) and cognition are important in caring for individuals with severe dementia; there is an urgent need for validated assessment tools for specific populations. This study aimed to develop and validate Chinese versions of the Quality of Life in Late-Stage Dementia (QUALID-C) scale and the Cognitive Test for Severe Dementia (CTSD-C) for Chinese older adults. Methods: This was a cross-sectional validation study comprised of 93 Chinese older adults with severe dementia recruited from 6 residential homes. The content and cultural validity of the QUALID-C and CTSD-C were evaluated by a 7-member expert panel, and interrater reliability, test-retest reliability, internal consistency, concurrent validity, and factorial structure were examined. Results: The QUALID-C showed acceptable internal consistency (Cronbach α = 0.65), good interrater reliability (intraclass correlation coefficient [ICC] = 0.99), and good test-retest reliability (ICC = 0.96). Principal component analysis yielded 3 factors; the items loaded on the factors were comparable to those in previous studies and suggested the scale’s multidimensionality to measure QoL. The CTSD-C showed satisfactory internal consistency (Cronbach α = 0.862), good interrater reliability (ICC = 0.99), and good test-retest reliability (ICC = 0.958). Principal component analysis yielded 3 factors; the items loaded on factors 1 and 2 resembled the items of the automatic response and attentional control factors of the original study. Conclusion: The QUALID-C and the CTSD-C are reliable and valid scales to measure the QoL and cognitive functions of Chinese older adults with severe dementia. These assessments can be utilized to evaluate the effectiveness of treatment and future research work.

Download Full-text

Using Generalizability Theory and the ERP Reliability Analysis (ERA) Toolbox for Assessing Test-Retest Reliability of ERP Scores Part 1: Algorithms, Framework, and Implementation

10.31234/osf.io/kcven ◽

2020 ◽

Author(s):

Peter E Clayson ◽

Kaylie Amanda Carbine ◽

Scott Baldwin ◽

Joseph A. Olsen ◽

Michael J. Larson

Keyword(s):

Reliability Analysis ◽

Internal Consistency ◽

Temporal Stability ◽

Test Theory ◽

Theory Approach ◽

Retest Reliability ◽

Score Reliability ◽

Test Retest Reliability ◽

The Impact ◽

Reliability Coefficients

The reliability of event-related brain potential (ERP) scores depends on study context and how those scores will be used, and reliability must be routinely evaluated. Many factors can influence ERP score reliability, and generalizability (G) theory provides a multifaceted approach to estimating the internal consistency and temporal stability of scores that is well suited for ERPs. G-theory’s approach possesses a number of advantages over classical test theory that make it ideal for pinpointing sources of error in scores. The current primer outlines the G-theory approach to estimating internal consistency (coefficients of equivalence) and test-retest reliability (coefficients of stability). This approach is used to evaluate the reliability of ERP measurements. The primer outlines how to estimate reliability coefficients that consider the impact of the number of trials, events, occasion, and group. The uses of two different G-theory reliability coefficients (i.e., generalizability and dependability) in ERP research are elaborated, and a dataset from the companion manuscript, which examines N2 amplitudes to Go/NoGo stimuli, is used as an example of the application of these coefficients to ERPs. The developed algorithms are implemented in the ERP Reliability Analysis (ERA) Toolbox, which is open-source software designed for estimating score reliability using G theory. The toolbox facilitates the application of G theory in an effort to simplify the study-by-study evaluation of ERP score reliability. The formulas provided in this primer should enable researchers to pinpoint the sources of measurement error in ERP scores from multiple recording sessions and subsequently plan studies that optimize score reliability.

Download Full-text

Measurement Properties of Headache-Specific Outcomes Scales in Adolescent Athletes

Journal of Sport Rehabilitation ◽

10.1123/jsr.20.1.129 ◽

2011 ◽

Vol 20 (1) ◽

pp. 129-142 ◽

Cited By ~ 12

Author(s):

Sarah K. Piebes ◽

Alison R. Snyder ◽

R. Curtis Bay ◽

Tamara C. Valovich McLeod

Keyword(s):

High School ◽

Psychometric Properties ◽

Outcome Measures ◽

Internal Consistency ◽

Measurement Properties ◽

Adolescent Athletes ◽

Healthy Adolescent ◽

Retest Reliability ◽

Test Retest Reliability ◽

The Impact

Context:Recurrent headaches significantly affect health-related quality of life (HRQOL) in adults; the impact of headache on HRQOL among adolescents is unknown, and the psychometric properties of headache-specific outcomes instruments have not been adequately studied in this population.Objective:To evaluate the psychometric properties of the Headache Impact Test (HIT-6) and Pediatric Migraine Disability Assessment (PedMIDAS) in healthy adolescent athletes.Design:Descriptive survey.Setting:High school athletic training facilities during the fall sports season.Participants:177 high school athletes (89 males and 88 females).Interventions:A survey consisting of a demographic and concussion-history questionnaire, a graded symptom scale, the HIT-6, and the PedMIDAS. Internal consistency (α), test–retest reliability (rs), Bland-Altman analyses, and the Mann-Whitney U test were used to evaluate psychometric properties and age and gender differences.Main Outcome Measures:The HIT-6 and PedMIDAS item and total scores.Results:Test–retest reliability for the HIT-6 total score was rs = .72, and reliability of individual items ranged from rs = .52 to .67. The test–retest reliability for the PedMIDAS total score was rs = .61, and reliability of individual items ranged from rs = .23 to .62. Both scales demonstrated acceptable internal consistency: HIT-6 α = .89−.90 and PedMIDAS α = .71−.75.Conclusions:The authors found moderate test–retest reliability for the HIT-6 and the PedMIDAS in a healthy adolescent athlete population. Research on the applicability and utility of the HIT-6 and PedMIDAS in concussed adolescents is warranted.

Download Full-text

High-Level Mobility Assessment Tool (HiMAT): Interrater Reliability, Retest Reliability, and Internal Consistency

Physical Therapy ◽

10.1093/ptj/86.3.395 ◽

2006 ◽

Vol 86 (3) ◽

pp. 395-400 ◽

Cited By ~ 51

Author(s):

Gavin P Williams ◽

Kenneth M Greenwood ◽

Val J Robertson ◽

Patricia A Goldie ◽

Meg E Morris

Keyword(s):

Internal Consistency ◽

Interrater Reliability ◽

Assessment Tool ◽

Physical Therapists ◽

Intraclass Correlation ◽

Systematic Change ◽

Retest Reliability ◽

Mobility Assessment ◽

High Level ◽

Very High

Abstract Background and Purpose. The High-Level Mobility Assessment Tool (HiMAT) assesses high-level mobility in people who have sustained a traumatic brain injury (TBI). The purpose of this study was to investigate the interrater reliability, retest reliability, and internal consistency of data obtained with the HiMAT. Subjects. Three physical therapists and 103 people with TBI were recruited from a rehabilitation hospital. Methods. Three physical therapists concurrently assessed a subset of 17 subjects with TBI to investigate interrater reliability. One physical therapist assessed a different subset of 20 subjects with TBI on 2 occasions, 2 days apart, to investigate retest reliability. Data from the entire sample of 103 subjects were used to investigate the internal consistency of this new scale. Results. Both the interrater reliability (intraclass correlation coefficient [ICC]=.99) and the retest reliability (ICC=.99) of the HiMAT data were very high. For retest reliability, a small systematic change was detected (t=3.82, df=19), indicating a marginal improvement of 1 point at retest. Internal consistency also was very high (Cronbach alpha=.97). Discussion and Conclusion. The HiMAT is a new tool specifically designed to measure high-level mobility, which currently is not a component of existing scales used in TBI. This study demonstrated that the HiMAT is a reliable tool for measuring high-level mobility. [Williams GP, Greenwood KM, Robertson VJ, et al. High-Level Mobility Assessment Tool (HiMAT): interrater reliability, retest reliability, and internal consistency. Phys Ther. 2006;86:395–400.]

Download Full-text

Assessing panic disorder-specific competencies: evaluation of the Cognitive Therapy Competence Scale for panic disorder

Behavioural and Cognitive Psychotherapy ◽

10.1017/s1352465820000612 ◽

2020 ◽

pp. 1-9

Author(s):

Sheena Liness ◽

Sarah Beale ◽

David M. Clark ◽

Paul M. Salkovskis ◽

Anke Ehlers ◽

...

Keyword(s):

Panic Disorder ◽

Psychometric Properties ◽

Cognitive Therapy ◽

Clinical Outcomes ◽

Convergent Validity ◽

Reliability And Validity ◽

Behavioural Therapy ◽

Specific Patient ◽

Therapist Competence ◽

Competence Scale

Abstract Background: Evidence-based treatment for panic disorder consists of disorder-specific cognitive behavioural therapy (CBT) protocols. However, most measures of CBT competence are generic and there is a clear need for disorder-specific assessment measures. Aims: To fill this gap, we evaluated the psychometric properties of the Cognitive Therapy Competence Scale (CTCP) for panic disorder. Method: CBT trainees (n = 60) submitted audio recordings of CBT for panic disorder that were scored on a generic competence measure, the Cognitive Therapy Scale – Revised (CTS-R), and the CTCP by markers with experience in CBT practice and evaluation. Trainees also provided pre- to post-treatment clinical outcomes on disorder-specific patient report measures for cases corresponding to their therapy recordings. Results: The CTCP exhibited strong internal consistency (α = .79–.91) and inter-rater reliability (ICC = .70–.88). The measure demonstrated convergent validity with the CTS-R (r = .40–.54), although investigation into competence classification indicated that the CTCP may be more sensitive at detecting competence for panic disorder-specific CBT skills. Notably, the CTCP demonstrated the first indication of a relationship between therapist competence and clinical outcome for panic disorder (r = .29–.35); no relationship was found for the CTS-R. Conclusions: These findings provide initial support for the reliability and validity of the CTCP for assessing therapist competence in CBT for panic disorder and support the use of anxiety disorder-specific competence measures. Further investigation into the psychometric properties of the measure in other therapist cohorts and its relationship with clinical outcomes is recommended.

Download Full-text

The Hearing Impairment Impact–Significant Other Profile (HII-SOP): A Tool to Measure Hearing Loss-Related Quality of Life in Spouses of People with Hearing Loss

Journal of the American Academy of Audiology ◽

10.3766/jaaa.23.10.6 ◽

2012 ◽

Vol 23 (10) ◽

pp. 807-823 ◽

Cited By ~ 12

Author(s):

Jill E. Preminger ◽

Suzanne Meeks

Keyword(s):

Quality Of Life ◽

Hearing Loss ◽

Internal Consistency ◽

Internal Consistency Reliability ◽

Third Party ◽

Retest Reliability ◽

Related Quality ◽

Test Retest Reliability ◽

The Impact

Background: Third-party hearing loss-related quality of life (HLQoL) reports measure the third-party disability as a result of communicating regularly with someone with hearing loss. Scales with known psychometric characteristics validated on a diverse subject population are needed in order to describe the activity limitations and participation restrictions experienced by spouses as a result of living with an individual with hearing loss. Purpose: The purpose of the present study was to develop a scale to measure third-party HLQoL in spouses of people with hearing loss with acceptable psychometric characteristics. This scale is known as the Hearing Impairment Impact–Significant Other Profile (HII-SOP). Research Design: In Study 1, the initial test items were developed and evaluated with a factor analysis for adequate construct validity. In Study 2, the internal consistency reliability, the validity, and the test-retest reliability of the revised test were evaluated. Study Sample: In Study 1, 120 people between 34 and 87 yr of age participated and in Study 2, 164 people between 23 and 88 yr of age participated. Data Collection and Analysis: In Study 1, a 41-item questionnaire was developed based on five content areas: physical adjustment, social activities, emotional reaction, intimate relationship, and change in roles. The scale was submitted to a factor analysis to analyze interrelationships among items, determine the underlying dimensions, and select items for the final scale. In Study 2, the internal consistency reliability, construct validity, and test-retest reliability were evaluated in the revised 20-item questionnaire. The internal-consistency reliability was measured using Cronbach's alpha. Validity was assessed by observing the correlations of the new scale with well-established scales measuring related constructs: overall health-related quality of life, marital communication, HLQoL, and negative affect, in both the person with hearing loss and the spouse. Test-retest reliability was measured in a subset of the spouses who completed the HII-SOP between 2 and 4 wk after the initial scale was completed. Results: The HII-SOP is a 20-item scale with three subscales which measure: (1) the emotions that arise when having a spouse with hearing loss as well as the impact of the hearing loss on the marital relationship, (2) the impact of the hearing loss on the social life of the spouse, and (3) the communication strategies used by the spouse. The scale and its subscales have adequate internal-consistency reliability suggesting that the 20 items do measure a single construct and the subscales do measure distinct subconstructs. The HII-SOP scale was significantly correlated with measures expected to relate to the construct of third-party disability associated with hearing loss. Finally, the HII-SOP scale has adequate test-retest reliability (r = 0.90) and the 95% critical differences is 19.7 points. Conclusions: The HII-SOP is a scale to measure third-party HLQoL in spouses of individuals with hearing loss. Scores of 20–39 reflect mild third-party disability, scores of 40–59 reflect moderate third-party disability, and scores >60 reflect severe third-party disability associated with hearing loss.

Download Full-text