Abstract
Background
Time constraints limit the use of measurement-based approaches in the routine clinical management of schizophrenia. Computerized Adaptive Testing (CAT) uses computational algorithms (item response theory - IRT) to match individual subjects with only the most relevant questions for them, reducing administration time and increasing measurement efficiency and scalability. This study aimed to test the psychometric properties of the newly developed CAT-Psychosis battery, both self-administered and rater-administered versions.
Methods
Patients rated themselves with the self-administered CAT-Psychosis which yields a current psychotic severity score. The CAT-Psychosis is based on a multidimensional extension of traditional IRT-based CAT that is suitable for complex traits and disorders such as psychosis. Two different raters independently conducted the rater-administered CAT-Psychosis to test inter-rater reliability (IRR). The Brief Psychiatric Rating Scale (BPRS) was administered to test convergent validity. Subjects were re-tested within 7 days to assess test-retest reliability. Generalized linear mixed models and Pearson product moment correlation coefficients were used to test for correlations between individual ratings and average CAT-Psychosis severity scores respectively and the BPRS. Intraclass correlation coefficients (ICCs) were used to test for reliability. Generalized linear and non-linear (logistic) mixed models were used to estimate diagnostic discrimination capacity (lifetime ratings) and to estimate diagnostic sensitivity, specificity and area under the ROC curve with 10-fold cross validation.
Results
135 subjects with psychosis and 25 healthy controls were included in the study. Mean age of the sample was 33.1 years, standard deviation (SD)=12.2years; 62% were males. No significant differences were detected between groups (p=0.9064 and p=0.2684 respectively). Mean length of assessment was 7 minutes and 9 seconds (SD: 5:04min) for the clinician-administered version and 1 minute and 49 seconds (SD: 1:35min) for the self-administered version, averaging 11.4 and 12.6 questions each. Convergent validity against BPRS was moderate for both rater-administered (r=0.65 (0.55–0.73); Marginal Maximum Likelihood Estimation (MMLE)=0.052, Standard Error (SE)=0.005, p<0.00001) and self-administered (r=0.66; MMLE=0.057, SE=0.005, p<0.00001) versions. Clinician version’s IRR was strong (ICC=0.67 (Confidence Interval (CI): 0.51–0.80)), and test-retest reliability was strong for both self-report (ICC=0.83 (CI: 0.76–0.87) and clinician (ICC=0.87 (CI: 0.75–0.94) version. The CAT-Psychosis clinician version was able to discriminate psychosis vs. healthy controls (Area Under the ROC Curve (AUC)=0.96 (CI: 0.90–0.97)). CAT-Psychosis self-report yielded similar results (AUC= 0.85 (CI: 0.77–0.88)).
Discussion
CAT-Psychosis provides valid severity ratings that mirror BPRS total scores, even as a self-report, yielding a dramatic reduction in administration time, while maintaining reliable psychometric properties. Furthermore, CAT-Psychosis, both clinician and self-report versions, is able to reliably discriminate psychotic patients based on a lifetime diagnosis from healthy controls after a brief assessment of current symptomatology.
O8.2. VALIDATION OF A COMPUTERIZED ADAPTIVE TESTING TOOL FOR PSYCHOSIS: THE CAT-PSYCHOSIS BATTERY
