scholarly journals Performance validity testing via telehealth and failure rate in veterans with moderate-to-severe traumatic brain injury: A veterans affairs TBI model systems study

2021 ◽  
pp. 1-9
Author(s):  
Robert Kanser ◽  
Justin O’Rourke ◽  
Marc A. Silva
Keyword(s):  
Traumatic Brain Injury ◽  
Brain Injury ◽  
Severe Traumatic Brain Injury ◽  
Digit Span ◽  
Model Systems ◽  
Performance Validity ◽  
Face To Face ◽  
Neuropsychological Evaluations ◽  
Validity Tests ◽  
Positive Classification

BACKGROUND: The COVID-19 pandemic has led to increased utilization of teleneuropsychology (TeleNP) services. Unfortunately, investigations of performance validity tests (PVT) delivered via TeleNP are sparse. OBJECTIVE: The purpose of this study was to examine the specificity of the Reliable Digit Span (RDS) and 21-item test administered via telephone METHOD: Participants were 51 veterans with moderate-to-severe traumatic brain injury (TBI). All participants completed the RDS and 21-item test in the context of a larger TeleNP battery. Specificity rates were examined across multiple cutoffs for both PVTs. RESULTS: Consistent with research employing traditional face-to-face neuropsychological evaluations, both PVTs maintained adequate specificity (i.e., >  90%) across previously established cutoffs. Specifically, defining performance invalidity as RDS <  7 or 21-item test forced choice total correct <  11 led to <  10%false positive classification errors. CONCLUSIONS: Findings add to the limited body of research examining and provide preliminary support for the use of the RDS and 21-item test in TeleNP via telephone. Both measures maintained adequate specificity in veterans with moderate-to-severe TBI. Future investigations including clinical or experimental “feigners” in a counter-balanced cross-over design (i.e., face-to-face vs. TeleNP) are recommended.

A-102 Performance Validity Testing in Telehealth Research: A VA Traumatic Brain Injury Model Systems Study

2021 ◽  
Vol 36 (6) ◽  
pp. 1151-1151
Author(s):  
Justin O'Rourke ◽  
Robert J Kanser ◽  
Marc A Silva
Keyword(s):  
Traumatic Brain Injury ◽  
Brain Injury ◽  
Digit Span ◽  
Secondary Analysis ◽  
Primary Objective ◽  
Model Systems ◽  
Performance Validity ◽  
Cut Scores ◽  
Chi Square ◽  
Severe Tbi

Abstract Objective Studies on Performance Validity Tests (PVTs) for tele-neuropsychology (TeleNP) are sparse. Verbal PVTs appear to better translate to TeleNP, so the primary objective of this study was to provide initial data on two well-established, verbal PVTs administered via TeleNP for research participants with traumatic brain injury (TBI). Methods This secondary analysis of the Veterans Affairs TBI Model Systems data included 53 participants enrolled in a PVT module study (3/01/2020–09/20/2020) with documented moderate-to-severe TBI per Glasgow Coma Score (M = 6.5, SD = 4.4), posttraumatic amnesia duration (M = 42.7 days, SD = 47.1), and/or time to follow commands (M = 10.5 days, SD = 16.3). Participants completed two PVTs—Reliable Digit Span (RDS) and the 21-Item Test (21-IT)—alongside telephone-based cognitive assessment 1–7 years after TBI. Descriptive analyses were performed to compare PVT performances to previously established cut scores. Chi square analyses were employed to examine 21-IT and RDS as dichotomous outcomes (pass/fail) at selected cutoffs. Results RDS ranged from 5 to 16 (M = 10.5, SD = 2.4). 21-IT ranged from 7 to 21 (M = 16.4, SD = 3.1). For RDS, 9.8% were invalid with a cutscore of ≤7 and 19.6% using a cutscore of ≤8. For the 21-IT, 7.8% were in invalid using a cutscore of ≤11, and 13.7% using a cutscore of ≤12. Conclusion(s) Using previously established cut scores, telephone-administered RDS and 21-IT resulted in relatively low rates of invalid performance among individuals with moderate-to-severe TBI. These findings provide preliminary support for the RDS and 21-IT in TeleNP.

Digit Span Forward as a Performance Validity Test in Dementia Evaluations: Specificity in Mild Cognitive Impairment, Mild Dementia, and Moderate Dementia

2019 ◽  
Vol 34 (6) ◽  
pp. 837-837
Author(s):  
H Clark ◽  
P Martin ◽  
R Schroeder
Keyword(s):  
Cognitive Impairment ◽  
Mild Cognitive Impairment ◽  
Digit Span ◽  
Future Research ◽  
Performance Validity ◽  
Mild Dementia ◽  
Dementia Patients ◽  
Moderate Dementia ◽  
Neuropsychological Evaluations ◽  
Validity Tests

Abstract Objective Traditional performance validity tests (PVTs) often yield high false positive rates in dementia evaluations. The current study examined the frequency of extremely low scores (≤ 2 percentile) on WAIS-IV Digit Span Forward (DSF) in older adults with Mild Cognitive Impairment (MCI) or dementia to evaluate its possible utility as a PVT in these populations. Method Archival data from outpatient neuropsychological evaluations were analyzed. Individuals who were not diagnosed with a neurocognitive disorder, had missing data, or were believed to be invalidly performing were excluded. Participants (n = 195; mean age = 72.8; mean education = 13.2 years) were classified according to their evaluation diagnosis of MCI (n = 72; mean RBANS Total Score = 86.8) or dementia. Dementia patients were further divided by MoCA score into groups of mild dementia (n = 90; MoCA≥15; mean RBANS Total Score = 71.0) or moderate dementia (n = 33; MoCA < 15; mean RBANS Total Score = 55.9). Frequencies of scaled scores were analyzed to calculate specificity values for each group. Results A WAIS-IV DSF scaled score of ≤4 (≤ 2 percentile) resulted in specificity values of 0.99 and 0.94 in MCI and mild dementia, respectively. Conversely, in moderate dementia, ≥0.90 specificity was achieved only when using a more conservative cutoff of ≤2. Conclusions Low DSF scaled scores occurred infrequently in MCI and mild dementia, indicating strong specificity and potential utility as a PVT in these populations. However, in moderate dementia, low DSF scores were more common, requiring use of a more stringent cutoff. Future research should examine DSF sensitivity to invalid performance, as well as DSF specificity according to specific etiologies of MCI and dementia.

scholarly journals A-205 Digit Span Raw, A Newly Derived Embedded Performance Validity Test

2020 ◽  
Vol 35 (6) ◽  
pp. 1000-1000
Author(s):  
Schroeder R ◽  
Soden D ◽  
Clark H ◽  
Martin P
Keyword(s):  
Digit Span ◽  
Area Under The Curve ◽  
Performance Validity ◽  
Adult Intelligence Scale ◽  
Wechsler Adult Intelligence Scale ◽  
Sensitivity Rate ◽  
Neuropsychological Evaluations ◽  
Validity Tests ◽  
Wechsler Adult Intelligence ◽  
Performance Validity Tests

Abstract Objective Outside of Reliable Digit Span (RDS), there has been minimal research examining the utility of Digit Span (DS) score combinations from the Wechsler Adult Intelligence Scale—4th Edition (WAIS-IV) as possible performance validity tests (PVTs). We sought to determine if other DS scores/score combinations might work more effectively than RDS as a PVT. Method Patients included 318 individuals who completed neuropsychological evaluations. Individuals were excluded if they were not administered DS; were not administered at least 4 criterion PVTs; had diagnoses of dementia, intellectual disability, or left hemisphere cerebrovascular accident; or had indeterminate validity results (i.e., failure of one PVT). Valid performers (n = 248) were those who passed all criterion PVTs while invalid performers (n = 70) failed two or more criterion PVTs. Receiver operating characteristic curves were conducted for multiple DS indices. Results Area under the curve (AUC) was highest for the DS index that combined raw scores from all three trials (Digit Span Raw; AUC = .821). Likewise, when examining cutoffs that maintained 90% specificity for each DS index, a Digit Span Raw cutoff of &lt; 20 produced the highest sensitivity rate (52%) of all indices. For comparison, RDS, RDS with sequencing, and DS scaled score had AUC values of .758, .802, and .811, respectively. When maintaining specificity at 90%, sensitivity rates were 28%, 43%, and 43%, respectively. Conclusions Results suggest that the most effective embedded DS index might be a new one, which we term Digit Span Raw. Cross-validation of these findings could provide support for using this index instead of the more commonly examined RDS.

Sign in / Sign up

Export Citation Format

Share Document