scholarly journals Cumulative False Positive Rates Given Multiple Performance Validity Tests: Commentary on Davis and Millis (2014) and Larrabee (2014)

2014 ◽  
Vol 28 (8) ◽  
pp. 1212-1223 ◽  
Author(s):  
Robert M. Bilder ◽  
Catherine A. Sugar ◽  
Gerhard S. Hellemann
Keyword(s):  
False Positive ◽  
Performance Validity ◽  
Validity Tests ◽  
Performance Validity Tests

scholarly journals Are Established Embedded Performance Validity Test Cut-Offs Generalizable to Patients With Multiple Sclerosis?

2020 ◽  
Vol 35 (5) ◽  
pp. 511-516
Author(s):  
Christopher H Domen ◽  
Michael R Greher ◽  
Patrick W Hosokawa ◽  
Sierra L Barnes ◽  
Brian D Hoyt ◽  
...  
Keyword(s):  
Multiple Sclerosis ◽  
Retrospective Study ◽  
Initial Data ◽  
False Positive ◽  
Large Scale ◽  
Advanced Disease ◽  
Performance Validity ◽  
Relapsing Remitting ◽  
Validity Tests ◽  
Performance Validity Tests

Abstract Objective Data for the use of embedded performance validity tests (ePVTs) with multiple sclerosis (MS) patients are limited. The purpose of the current study was to determine whether ePVTs previously validated in other neurological samples perform similarly in an MS sample. Methods In this retrospective study, the prevalence of below-criterion responding at different cut-off scores was calculated for each ePVT of interest among patients with MS who passed a stand-alone PVT. Results Previously established PVT cut-offs generally demonstrated acceptable specificity when applied to our sample. However, the overall cognitive burden of the sample was limited relative to that observed in prior large-scale MS studies. Conclusion The current study provides initial data regarding the performance of select ePVTs among an MS sample. Results indicate most previously validated cut-offs avoid excessive false positive errors in a predominantly relapsing remitting MS sample. Further validation among MS patients with more advanced disease is warranted.

A-181 Do Effortful Epilepsy Patients Fall Below Performance Validity Test Cutoffs at Higher Rates than Is Considered Acceptable?

2021 ◽  
Vol 36 (6) ◽  
pp. 1236-1236
Author(s):  
Hyun Jin Kang ◽  
Michelle Kim ◽  
Karen Torres
Keyword(s):  
False Positive ◽  
False Positive Rate ◽  
Performance Validity ◽  
Test Of Memory Malingering ◽  
Positive Rate ◽  
History Of ◽  
Warrington Recognition Memory Test ◽  
Validity Tests ◽  
The University ◽  
Performance Validity Tests

Abstract Objective Factors specific to an epilepsy population (e.g., medications, psychiatric comorbidities, localization) may result in higher false positive rates on performance validity tests (PVT), rendering the results more equivocal. This study examined whether specificity is reduced in effortful epilepsy patients on the Warrington Recognition Memory Test - Words (WRMT-W) and Test of Memory Malingering (TOMM). Method 53 epilepsy patients referred for neuropsychological evaluation through the University of Washington Regional Epilepsy Center were examined. Patients were majority male (56.6%) and Caucasian (79.2%). Average age and education were 36.1 (SD = 13.03) and 13.4 years (SD = 2.39), respectively. Patients with an intelligence quotient of <70, history of brain surgery, and those who seized during testing were excluded. Patients clinically observed to have reduced effort with 2+ PVT failures were excluded (n = 3). Frequency tables for WRMT-W and TOMM performances were utilized to examine specificity based on prior cutoffs identified for these measures. Results The WRMT-W cutoff of ≤42 was associated with 88.7% specificity. TOMM Trial 2 and Retention cutoffs of <45 were associated with 98.1% and 100% specificity, respectively. The WRMT-W cutoff was associated with 91.7% specificity in language dominant hemisphere onset epilepsy patients (n = 16). None performed below cutoffs on the TOMM. All nondominant hemisphere onset patients (n = 8) performed above WRMT-W and TOMM cutoffs. Conclusions Use of the WRMT-W and TOMM in an epilepsy population is associated with an acceptable false positive rate (specificity around 90%). However, future studies examining the sensitivity of these measures in epilepsy patients should be performed.

A Bayesian Approach to Mixed Group Validation of Performance Validity Tests

2014 ◽  
Author(s):  
Douglas Mossman ◽  
William Miller ◽  
Elliot Lee ◽  
Roger Gervais ◽  
Kathleen Hart ◽  
...  
Keyword(s):  
Bayesian Approach ◽  
Performance Validity ◽  
Mixed Group ◽  
Validity Tests ◽  
Performance Validity Tests

The Rey Dot Counting Test as a Tool for Detecting Suboptimal Performance in Athlete Baseline Testing

2020 ◽  
Author(s):  
Andrew DaCosta ◽  
Frank Webbe ◽  
Anthony LoGalbo
Keyword(s):  
Cognitive Testing ◽  
Performance Validity ◽  
Baseline Assessment ◽  
Baseline Testing ◽  
Validity Measures ◽  
Significant Disagreement ◽  
Two Measures ◽  
Validity Tests ◽  
Performance Validity Tests ◽  
Suboptimal Effort

Abstract Objective The limitations of Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT)’s embedded validity measures (EVMs) are well-documented, as estimates suggest up to 35% of invalid baseline performances go undetected. Few studies have examined standalone performance validity tests (PVT) as a supplement to ImPACT’s EVMs. Method College athletes (n = 1,213) were administered a preseason baseline assessment that included ImPACT and the Rey Dot Counting Test (DCT), a standalone PVT, among other measures. Results Sixty-nine athletes (5.69%) met criteria for suboptimal effort on either ImPACT or the DCT. The DCT detected more cases of suboptimal effort (n = 50) than ImPACT (n = 21). A χ2 test of independence detected significant disagreement between the two measures, as only two individuals produced suboptimal effort on both (χ2(2) = 1.568, p = .210). Despite this disagreement, there were significant differences between the suboptimal effort DCT group and the adequate effort DCT group across all four ImPACT neurocognitive domains (U = 19,225.000, p < .001; U = 17,859.000, p < .001; U = 13,854.000, p < .001; U = 17,850.500, p < .001). Conclusions The DCT appears to detect suboptimal effort otherwise undetected by ImPACT’s EVMs.

scholarly journals A-225 The Rey Dot Counting Test as a Tool for Detecting Suboptimal Performance in Athlete Baseline Testing

2020 ◽  
Vol 35 (6) ◽  
pp. 1020-1020
Author(s):  
Dacosta A ◽  
Roccaforte A ◽  
Sohoni R ◽  
Crane A ◽  
Webbe F ◽  
...  
Keyword(s):  
Cognitive Testing ◽  
Performance Validity ◽  
Baseline Assessment ◽  
Baseline Testing ◽  
Validity Measures ◽  
Significant Disagreement ◽  
Two Measures ◽  
Validity Tests ◽  
Performance Validity Tests ◽  
Suboptimal Effort

Abstract Objective The limitations of Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT)‘s embedded validity measures (EVMs) are well-documented, as estimates suggest up to 35% of invalid baseline performances go undetected (Gaudet & Weyandt, 2017). Few studies have examined standalone performance validity tests (PVT) as a supplement to ImPACT’s EVMs (Gaudet & Weyandt, 2017). Method College athletes (n = 1,213) were administered a preseason baseline assessment that included ImPACT and the Rey Dot Counting Test (DCT), a standalone PVT, among other measures. Results Sixty-nine athletes (5.69%) met criteria for suboptimal effort on either ImPACT or the DCT. The DCT detected more cases of suboptimal effort (n = 50) than ImPACT (n = 21). A χ2 test of independence detected significant disagreement between the two measures, as only 2 individuals produced suboptimal effort on both (χ2(2) = 1.568, p = .210). Despite this disagreement, there were significant differences between the suboptimal effort DCT group and the adequate effort DCT group across all four ImPACT neurocognitive domains (U = 19225.000, p < .001; U = 17859.000, p < .001; U = 13854.000, p < .001; U = 17850.500, p < .001). Conclusions The DCT appears to detect suboptimal effort otherwise undetected by ImPACT’s EVMs.

Below Chance Performance of p < .05 or p < .20: Frequency of Statistically Below Chance Scores in Dementia

2019 ◽  
Vol 34 (6) ◽  
pp. 835-835
Author(s):  
D Olsen ◽  
R Schroeder ◽  
P Martin
Keyword(s):  
Empirical Literature ◽  
P Value ◽  
Cognitively Impaired ◽  
Performance Validity ◽  
Chance Performance ◽  
Choice Performance ◽  
Hand Test ◽  
Validity Tests ◽  
External Incentives ◽  
Performance Validity Tests

Abstract Objective A p-value of < .05 has traditionally been utilized to determine below chance performance on forced-choice performance validity tests (PVT). Recently, Binder and colleagues (2014 & 2018) proposed that the p-value cutoff increase to < .20. To ensure this does not result in frequent false-positive errors in patients who are likely to have significant cognitive impairment, frequency of below chance scores at both p-values were examined within the context of possible dementia. Method Archival data of cognitively impaired inpatient (n = 55; mean RBANS Total Score = 64.67) and outpatient (n = 203; mean RBANS Total Score = 74.15) older adults without external incentives were examined to determine frequency of below chance performances on the Coin-in-the-Hand Test. To supplement this data and examine below chance performance on a second PVT, the authors reviewed empirical literature and extracted data on TOMM performance in individuals with dementia. Four studies (n = 269 patients) provided data that could be extracted. Results No patient produced a Coin-in-the-Hand Test score (0/258 individuals) reaching either p value cutoff. Similarly, no patient produced a TOMM Trial 2 (0/121 individuals) or Retention score (0/84 individuals) reaching either p value cutoff. For TOMM Trial 1, no patient (0/44) scored at p < .05 but two patients (2/64) scored at p < .20. Conclusions No individual in this study produced scores on either PVT reaching the p < .05 cutoff. At the p < .20 cutoff, there were only 2 out of 527 performances (0.4%) that reached this threshold; both of which were observed on TOMM Trial 1. This data supports the recommendation that p < .20 be used when determining below chance performance.

Sign in / Sign up

Export Citation Format

Share Document