scholarly journals Development and Validation of a Computerized Adaptive Assessment Tool for Discrimination and Measurement of Psychotic Symptoms

2020 ◽  
Author(s):  
Daniel Guinart ◽  
Renato de Filippis ◽  
Stella Rosson ◽  
Bhagyashree Patil ◽  
Lara Prizgint ◽  
...  
Keyword(s):  
Convergent Validity ◽  
Computerized Adaptive Testing ◽  
Rating Scale ◽  
Item Bank ◽  
Self Report ◽  
Psychotic Symptoms ◽  
Healthy Controls ◽  
Retest Reliability ◽  
Test Retest Reliability ◽  
Measurement Efficiency

Abstract Objective Time constraints limit the use of measurement-based approaches in research and routine clinical management of psychosis. Computerized adaptive testing (CAT) can reduce administration time, thus increasing measurement efficiency. This study aimed to develop and test the capacity of the CAT-Psychosis battery, both self-administered and rater-administered, to measure the severity of psychotic symptoms and discriminate psychosis from healthy controls. Methods An item bank was developed and calibrated. Two raters administered CAT-Psychosis for inter-rater reliability (IRR). Subjects rated themselves and were retested within 7 days for test-retest reliability. The Brief Psychiatric Rating Scale (BPRS) was administered for convergent validity and chart diagnosis, and the Structured Clinical Interview (SCID) was used to test psychosis discriminant validity. Results Development and calibration study included 649 psychotic patients. Simulations revealed a correlation of r = .92 with the total 73-item bank score, using an average of 12 items. Validation study included 160 additional patients and 40 healthy controls. CAT-Psychosis showed convergent validity (clinician: r = 0.690; 95% confidence interval [95% CI]: 0.610–0.757; self-report: r = .690; 95% CI: 0.609–0.756), IRR (intraclass correlation coefficient [ICC] = 0.733; 95% CI: 0.611–0.828), and test-retest reliability (clinician ICC = 0.862; 95% CI: 0.767–0.922; self-report ICC = 0.815; 95%CI: 0.741–0.871). CAT-Psychosis could discriminate psychosis from healthy controls (clinician: area under the receiver operating characteristic curve [AUC] = 0.965, 95% CI: 0.945–0.984; self-report AUC = 0.850, 95% CI: 0.807–0.894). The median length of the clinician-administered assessment was 5 minutes (interquartile range [IQR]: 3:23–8:29 min) and 1 minute, 20 seconds (IQR: 0:57–2:09 min) for the self-report. Conclusion CAT-Psychosis can quickly and reliably assess the severity of psychosis and discriminate psychotic patients from healthy controls, creating an opportunity for frequent remote assessment and patient/population-level follow-up.

scholarly journals O8.2. VALIDATION OF A COMPUTERIZED ADAPTIVE TESTING TOOL FOR PSYCHOSIS: THE CAT-PSYCHOSIS BATTERY

2020 ◽  
Vol 46 (Supplement_1) ◽  
pp. S19-S19
Author(s):  
Daniel Guinart ◽  
Renato de Filippis ◽  
Stella Rosson ◽  
Lara Prizgint ◽  
Bhagyashree Patil ◽  
...  
Keyword(s):  
Roc Curve ◽  
Mixed Models ◽  
Convergent Validity ◽  
Computerized Adaptive Testing ◽  
Correlation Coefficients ◽  
The Self ◽  
Self Report ◽  
Healthy Controls ◽  
Administration Time ◽  
Test Retest Reliability

Abstract Background Time constraints limit the use of measurement-based approaches in the routine clinical management of schizophrenia. Computerized Adaptive Testing (CAT) uses computational algorithms (item response theory - IRT) to match individual subjects with only the most relevant questions for them, reducing administration time and increasing measurement efficiency and scalability. This study aimed to test the psychometric properties of the newly developed CAT-Psychosis battery, both self-administered and rater-administered versions. Methods Patients rated themselves with the self-administered CAT-Psychosis which yields a current psychotic severity score. The CAT-Psychosis is based on a multidimensional extension of traditional IRT-based CAT that is suitable for complex traits and disorders such as psychosis. Two different raters independently conducted the rater-administered CAT-Psychosis to test inter-rater reliability (IRR). The Brief Psychiatric Rating Scale (BPRS) was administered to test convergent validity. Subjects were re-tested within 7 days to assess test-retest reliability. Generalized linear mixed models and Pearson product moment correlation coefficients were used to test for correlations between individual ratings and average CAT-Psychosis severity scores respectively and the BPRS. Intraclass correlation coefficients (ICCs) were used to test for reliability. Generalized linear and non-linear (logistic) mixed models were used to estimate diagnostic discrimination capacity (lifetime ratings) and to estimate diagnostic sensitivity, specificity and area under the ROC curve with 10-fold cross validation. Results 135 subjects with psychosis and 25 healthy controls were included in the study. Mean age of the sample was 33.1 years, standard deviation (SD)=12.2years; 62% were males. No significant differences were detected between groups (p=0.9064 and p=0.2684 respectively). Mean length of assessment was 7 minutes and 9 seconds (SD: 5:04min) for the clinician-administered version and 1 minute and 49 seconds (SD: 1:35min) for the self-administered version, averaging 11.4 and 12.6 questions each. Convergent validity against BPRS was moderate for both rater-administered (r=0.65 (0.55–0.73); Marginal Maximum Likelihood Estimation (MMLE)=0.052, Standard Error (SE)=0.005, p<0.00001) and self-administered (r=0.66; MMLE=0.057, SE=0.005, p<0.00001) versions. Clinician version’s IRR was strong (ICC=0.67 (Confidence Interval (CI): 0.51–0.80)), and test-retest reliability was strong for both self-report (ICC=0.83 (CI: 0.76–0.87) and clinician (ICC=0.87 (CI: 0.75–0.94) version. The CAT-Psychosis clinician version was able to discriminate psychosis vs. healthy controls (Area Under the ROC Curve (AUC)=0.96 (CI: 0.90–0.97)). CAT-Psychosis self-report yielded similar results (AUC= 0.85 (CI: 0.77–0.88)). Discussion CAT-Psychosis provides valid severity ratings that mirror BPRS total scores, even as a self-report, yielding a dramatic reduction in administration time, while maintaining reliable psychometric properties. Furthermore, CAT-Psychosis, both clinician and self-report versions, is able to reliably discriminate psychotic patients based on a lifetime diagnosis from healthy controls after a brief assessment of current symptomatology.

Psychometric Properties of the Persian Version of the Weiss Functional Impairment Rating Scale–Self-Report Form in Iranian Adolescents

2017 ◽  
Vol 23 (13) ◽  
pp. 1600-1609 ◽  
Author(s):  
Habib Hadianfard ◽  
Behnaz Kiani ◽  
Margaret D. Weiss
Keyword(s):  
Psychometric Properties ◽  
Functional Impairment ◽  
Convergent Validity ◽  
Rating Scale ◽  
Self Report ◽  
Retest Reliability ◽  
Persian Version ◽  
Adolescents And Adults ◽  
Test Retest Reliability

Objective: The Weiss Functional Impairment Rating Scale–Self-Report Form (WFIRS-S) was developed as a measure of functional impairment in adolescents and adults with ADHD. This study reports the psychometric properties of the Persian version of the WFIRS-S in a sample of normal Iranian adolescents. Method: Internal consistency and factor structure of the WFIRS-S were tested on a sample of 386 students (Grades 7-12). The test-retest reliability and the convergent validity of the WFIRS-S were evaluated by using two different subsamples including 50 and 100 students, respectively. Results: The Cronbach’s alpha values were between .72 and .94 for the WFIRS-S subdomains and total scale. The test-retest reliability was .80 for the total scale. The WFIRS-S subdomains had moderate to high significant correlations with the Pediatric Quality of Life total scale. Conclusion: The Persian version of the WFIRS-S has acceptable psychometric properties and could be used as a functional impairment assessment for adolescents.

Validation of the NARCOMS Registry: pain assessment

2005 ◽  
Vol 11 (3) ◽  
pp. 338-342 ◽  
Author(s):  
Ruth Ann Marrie ◽  
Gary Cutter ◽  
Tuula Tyry ◽  
Olympia Hadjimichael ◽  
Timothy Vollmer
Keyword(s):  
Multiple Sclerosis ◽  
Convergent Validity ◽  
Disease Duration ◽  
Self Report ◽  
Research Committee ◽  
Retest Reliability ◽  
The North ◽  
Disability Status ◽  
Pain Question ◽  
Test Retest Reliability

The North American Research Committee on Multiple Sclerosis (NARCOMS) Registry is a multiple sclerosis (MS) self-report registry with more than 24 000 participants. Participants report disability status upon enrolment, and semi-annually using Performance Scales (PS), Patient Determined Disease Steps (PDDS) and a pain question. In November 2000 and 2001, we also collected the Pain Effects Scale (PES). Our aim was to validate the NARCOMS pain question using the PES as our criterion measure. We measured correlations between the pain question and age, disease duration, various PS subscales and PDDS to assess construct validity. We correlated pain question responses in participants who reported no change in PDSS or the PS subscales between questionnaires to determine test—retest reliability. We measured responsiveness in participants who reported a substantial change in the sensory, spasticity PS subscales. The correlation between the pain question and PES was r=0.61 in November 2000, and r=0.64 in November 2001 (both P<0.0001). Correlations between the pain question and age, and disease duration were low, indicating divergent validity. Correlations between the pain question and spasticity, sensory PS subscales and PDSS were moderate, indicating convergent validity. Test—retest reliability was r=0.84 (P<0.0001). Responsiveness was 70.7%. The pain question is a valid self-report measure of pain in MS.

Psychometric Properties of a Structured Diagnostic Interview for DSM-5 Anxiety, Mood, and Obsessive-Compulsive and Related Disorders

Assessment ◽  
2016 ◽  
Vol 25 (1) ◽  
pp. 3-13 ◽  
Author(s):  
David F. Tolin ◽  
Christina Gilliam ◽  
Bethany M. Wootton ◽  
William Bowe ◽  
Laura B. Bragdon ◽  
...  
Keyword(s):  
Convergent Validity ◽  
Interrater Reliability ◽  
Diagnostic Interview ◽  
Self Report ◽  
Obsessive Compulsive ◽  
Reliability Data ◽  
Retest Reliability ◽  
Dsm 5 ◽  
Structured Diagnostic Interview ◽  
Test Retest Reliability

Three hundred sixty-two adult patients were administered the Diagnostic Interview for Anxiety, Mood, and OCD and Related Neuropsychiatric Disorders (DIAMOND). Of these, 121 provided interrater reliability data, and 115 provided test–retest reliability data. Participants also completed a battery of self-report measures that assess symptoms of anxiety, mood, and obsessive-compulsive and related disorders. Interrater reliability of DIAMOND anxiety, mood, and obsessive-compulsive and related diagnoses ranged from very good to excellent. Test–retest reliability of DIAMOND diagnoses ranged from good to excellent. Convergent validity was established by significant between-group comparisons on applicable self-report measures for nearly all diagnoses. The results of the present study indicate that the DIAMOND is a promising semistructured diagnostic interview for DSM-5 disorders.

scholarly journals Psychometric Properties of Persian Version of the Amphetamine Withdrawal Questionnaire Version 2 (AWQV2) in Patients with Methamphetamine-Type Substance Use Disorder

2020 ◽  
Vol 14 (4) ◽  
Author(s):  
Vahid Farnia ◽  
Mehdi Moradinazar ◽  
Nasrin Abdoli ◽  
Mostafa Alikhani ◽  
Mansour Rezaei ◽  
...  
Keyword(s):  
Psychometric Properties ◽  
Convergent Validity ◽  
Self Report ◽  
Cronbach’S Alpha ◽  
Retest Reliability ◽  
Cronbach's Alpha ◽  
Persian Version ◽  
Amphetamine Withdrawal ◽  
Questionnaire Version ◽  
Test Retest Reliability

Background: No standard self-report instrument for withdrawal symptoms is available in Iran. Objectives: This study aimed to evaluate the psychometric properties of the Persian version of the 10-item Amphetamine Withdrawal questionnaire version 2 (AWQV2). Methods: A sample of 388 methamphetamine addicts (215 females and 173 males) referred to addiction recovery centers and psychiatric ward of Farabi Hospital in Kermanshah. A two-stage random sampling method was used. The reliability and internal consistency of the AWQV2 items were examined using Cronbach’s alpha and test-retest reliability, respectively, and the instrument validity of the AWQV2 was measured using construct validity and convergent validity. Results: The AWQV2 had a Cronbach’s alpha of 0.72. Factor analysis using the main component analysis with a varimax rotation introduced three factors of hyperarousal, anxiety, and reversed vegetative symptoms. These factors explained 0.58 of the total variance. The coefficient of test-retest reliability at a 2-week interval was equal to 0.77. The convergent validity of the AWQV2 was examined by simultaneously administering the Advanced Warning of Relapse (AWARE) questionnaire to 40 subjects, with a correlation coefficient of 0.81. Conclusions: Based on the results, the AWQV2 has very good psychometric properties and may be used in research and therapeutic interventions.

scholarly journals Reduced parietal alpha power and psychotic symptoms: Test-retest reliability of resting-state magnetoencephalography in schizophrenia and healthy controls

2020 ◽  
Vol 215 ◽  
pp. 229-240 ◽  
Author(s):  
Felicha T. Candelaria-Cook ◽  
Megan E. Schendel ◽  
Cesar J. Ojeda ◽  
Juan R. Bustillo ◽  
Julia M. Stephen
Keyword(s):  
Resting State ◽  
Psychotic Symptoms ◽  
Alpha Power ◽  
Healthy Controls ◽  
Retest Reliability ◽  
Test Retest Reliability

Validation of a Computerized Use of Time Recall for Activity Measurement in Advanced-Age Adults

2014 ◽  
Vol 22 (2) ◽  
pp. 245-254 ◽  
Author(s):  
Casey Jordan Mace ◽  
Ralph Maddison ◽  
Timothy Olds ◽  
Ngaire Kerse
Keyword(s):  
Physical Activity ◽  
Convergent Validity ◽  
Vigorous Physical Activity ◽  
Activity Level ◽  
Self Report ◽  
Advanced Age ◽  
Activity Measurement ◽  
Retest Reliability ◽  
Use Of Time ◽  
Test Retest Reliability

Background:The Multimedia Activity Recall for Children and Adults (MARCA) is a computerized recall instrument that records use of time during 24 hr the previous day and has been developed to address limitations of current self-report physical activity measures for those in advanced age.Methods:Test–retest reliability and convergent validity of the adult MARCA were assessed in a sample of 45 advanced-age adults (age 84.9SD ±1.62 yr) as a subsample of the Life and Living in Advanced-Age Cohort Study New Zealand (LiLACS NZ). Test–retest methods required participants to recall the previous day’s activity using the MARCA twice within the same day. Convergent validity was assessed against accelerometry.Results:Test–retest reliability was high, with ICCs greater than .99 for moderate to vigorous physical activity (MVPA) and physical activity level (PAL). Compared with accelerometry, the MARCA demonstrated validity comparable to other self-report instruments with Spearman’s coefficients of .34 and .59 for time spent in nonsedentary physical activity and PAL.Conclusion:The MARCA is a valid and reliable self-report tool for physical activity behaviors in advanced-age adults.

Self-Administered Instrument to Measure the Patient's Experience of Recovery After First-Episode Psychosis: Development and Validation of the Psychosis Recovery Inventory

2005 ◽  
Vol 39 (6) ◽  
pp. 493-499 ◽  
Author(s):  
Eric Y.H. Chen ◽  
Dennise K.P. Tam ◽  
Josephine W.S. Wong ◽  
C.W Law ◽  
Cindy P.Y. Chiu
Keyword(s):  
Internal Consistency ◽  
Validation Study ◽  
Convergent Validity ◽  
Rating Scale ◽  
First Episode Psychosis ◽  
First Episode ◽  
Recovery Stage ◽  
Retest Reliability ◽  
Episode Psychosis ◽  
Test Retest Reliability

Objective: Patients who are recovering from a first-episode psychosis face specific and complex issues that are related to their illness and treatment experiences, such as the appraisal of the extent of their recoveryand the risk ofrelapse. Currently, no instrument provides a comprehensive assessment of these related attitudes. A novel self-administered rating scale for the measurement of key perceptions during the recovery stage after a firstepisode psychosis is presented. The Psychosis Recovery Inventory (PRI) is designed to specifically address a number of closely related issues that are faced by patients who are recovering from a first-episode psychosis. Method: The process of development of the PRI involved the generation of items from qualitative interviews, the construction and refinement of these items and a validation study. The longitudinal stability of the PRI items was assessed in a test–retest reliability study in which 20 patients completed the retest within 4 weeks. The internal consistency and convergent validity of the PRI were evaluated by a comparison of the PRI subscale scores and the Scale to Assess Unawareness of Mental Disorder and Drug Attitude Inventory scores in a sample of 48 first-episode psychosis patients. Results: The validation study shows that the PRI is an instrument with a good test–retest reliability, internal consistency and convergent validity. Conclusions: This pragmatic, low burden, self-administered scale can be applied in clinical and research settings to obtain reliable information on the attitudes of patients on a range of interrelated issues in the recovery stage that follows a first-episode psychosis.

scholarly journals Monitoring Positive Mental Health in Swedish Adolescents: Psychometric Evaluation of the Mental Health Continuum – Short Form

2020 ◽  
Author(s):  
Fredrik Söderqvist ◽  
Peter Larm
Keyword(s):  
Mental Health ◽  
Convergent Validity ◽  
Short Form ◽  
Well Being ◽  
Self Report ◽  
Specific Group ◽  
Retest Reliability ◽  
Model Based ◽  
Test Retest Reliability ◽  
Mental Well Being

Abstract BackgroundThe Mental Health Continuum – Short form (MHC-SF) is a self-report measure that has been increasingly used to monitor mental well-being at the population level. The aim of this study was to evaluate, for the first time, the psychometric properties of the MHC-SF in a Swedish population, more specifically adolescents. MethodsFirst, the evaluation was performed by examining face validity and test–retest reliability obtained in a pre-study (n = 93). Then using data from the Survey of Adolescent Life in Vestmanland 2020 (n = 3880; participation rate = 71%; females = 51%; mean age = 16.23 years), we performed confirmatory factor analysis on different factor structures based on theory and previous research. Model-based estimates were calculated for assessing the internal reliability of the factor structure with the best fit. Convergent validity was assessed by bivariate as well as model-based correlations, and test–retest reliability was evaluated by intra-class correlation coefficients. ResultsThis study on Swedish adolescents found that the MHC-SF is essentially unidimensional and best described with a bifactor model consisting of a dominant general well-being factor and three specific group factors of emotional, social and psychological well-being. Its overall reliability and the reliability of the general well-being factor were good to excellent, while the reliability of its subscales (specific group factor) was poor, and thus should not be used alone. Test–retest reliability of the total scale was good, and convergent validity was supported by strong to very strong correlations with the Short Warwick–Edinburg Mental Well-being Scale. ConclusionsIn conclusion, we consider the Swedish MHC-SF to be a psychometrically sound instrument for monitoring overall mental well-being in Swedish adolescents.

Sign in / Sign up

Export Citation Format

Share Document