Minimizing False Positive Error With Multiple Performance Validity Tests: Response to Bilder, Sugar, and Hellemann (2014 this issue)

2014 ◽  
Vol 28 (8) ◽  
pp. 1230-1242 ◽  
Author(s):  
Glenn J. Larrabee
Keyword(s):  
False Positive ◽  
Performance Validity ◽  
Positive Error ◽  
False Positive Error ◽  
Validity Tests ◽  
Performance Validity Tests

scholarly journals Are Established Embedded Performance Validity Test Cut-Offs Generalizable to Patients With Multiple Sclerosis?

2020 ◽  
Vol 35 (5) ◽  
pp. 511-516
Author(s):  
Christopher H Domen ◽  
Michael R Greher ◽  
Patrick W Hosokawa ◽  
Sierra L Barnes ◽  
Brian D Hoyt ◽  
...  
Keyword(s):  
Multiple Sclerosis ◽  
Retrospective Study ◽  
Initial Data ◽  
False Positive ◽  
Large Scale ◽  
Advanced Disease ◽  
Performance Validity ◽  
Relapsing Remitting ◽  
Validity Tests ◽  
Performance Validity Tests

Abstract Objective Data for the use of embedded performance validity tests (ePVTs) with multiple sclerosis (MS) patients are limited. The purpose of the current study was to determine whether ePVTs previously validated in other neurological samples perform similarly in an MS sample. Methods In this retrospective study, the prevalence of below-criterion responding at different cut-off scores was calculated for each ePVT of interest among patients with MS who passed a stand-alone PVT. Results Previously established PVT cut-offs generally demonstrated acceptable specificity when applied to our sample. However, the overall cognitive burden of the sample was limited relative to that observed in prior large-scale MS studies. Conclusion The current study provides initial data regarding the performance of select ePVTs among an MS sample. Results indicate most previously validated cut-offs avoid excessive false positive errors in a predominantly relapsing remitting MS sample. Further validation among MS patients with more advanced disease is warranted.

A-181 Do Effortful Epilepsy Patients Fall Below Performance Validity Test Cutoffs at Higher Rates than Is Considered Acceptable?

2021 ◽  
Vol 36 (6) ◽  
pp. 1236-1236
Author(s):  
Hyun Jin Kang ◽  
Michelle Kim ◽  
Karen Torres
Keyword(s):  
False Positive ◽  
False Positive Rate ◽  
Performance Validity ◽  
Test Of Memory Malingering ◽  
Positive Rate ◽  
History Of ◽  
Warrington Recognition Memory Test ◽  
Validity Tests ◽  
The University ◽  
Performance Validity Tests

Abstract Objective Factors specific to an epilepsy population (e.g., medications, psychiatric comorbidities, localization) may result in higher false positive rates on performance validity tests (PVT), rendering the results more equivocal. This study examined whether specificity is reduced in effortful epilepsy patients on the Warrington Recognition Memory Test - Words (WRMT-W) and Test of Memory Malingering (TOMM). Method 53 epilepsy patients referred for neuropsychological evaluation through the University of Washington Regional Epilepsy Center were examined. Patients were majority male (56.6%) and Caucasian (79.2%). Average age and education were 36.1 (SD = 13.03) and 13.4 years (SD = 2.39), respectively. Patients with an intelligence quotient of <70, history of brain surgery, and those who seized during testing were excluded. Patients clinically observed to have reduced effort with 2+ PVT failures were excluded (n = 3). Frequency tables for WRMT-W and TOMM performances were utilized to examine specificity based on prior cutoffs identified for these measures. Results The WRMT-W cutoff of ≤42 was associated with 88.7% specificity. TOMM Trial 2 and Retention cutoffs of <45 were associated with 98.1% and 100% specificity, respectively. The WRMT-W cutoff was associated with 91.7% specificity in language dominant hemisphere onset epilepsy patients (n = 16). None performed below cutoffs on the TOMM. All nondominant hemisphere onset patients (n = 8) performed above WRMT-W and TOMM cutoffs. Conclusions Use of the WRMT-W and TOMM in an epilepsy population is associated with an acceptable false positive rate (specificity around 90%). However, future studies examining the sensitivity of these measures in epilepsy patients should be performed.

Forensic science evidence: Naive estimates of false positive error rates and reliability

2020 ◽  
Author(s):  
Kristy Martire ◽  
Agnes Bali ◽  
Kaye Ballantyne ◽  
Gary Edmond ◽  
Richard Kemp ◽  
...  
Keyword(s):  
Forensic Science ◽  
False Positive ◽  
Error Rates ◽  
Forensic Sciences ◽  
Positive Error ◽  
False Positive Error ◽  
Reliability Estimates ◽  
Positive Report ◽  
Science Evidence ◽  
Science Disciplines

We do not know how often false positive reports are made in a range of forensic science disciplines. In the absence of this information it is important to understand the naive beliefs held by potential jurors about forensic science evidence reliability. It is these beliefs that will shape evaluations at trial. This descriptive study adds to our knowledge about naive beliefs by: 1) measuring jury-eligible (lay) perceptions of reliability for the largest range of forensic science disciplines to date, over three waves of data collection between 2011 and 2016 (n = 674); 2) calibrating reliability ratings with false positive report estimates; and 3) comparing lay reliability estimates with those of an opportunity sample of forensic practitioners (n = 53). Overall the data suggest that both jury-eligible participants and practitioners consider forensic evidence highly reliable. When compared to best or plausible estimates of reliability and error in the forensic sciences these views appear to overestimate reliability and underestimate the frequency of false positive errors. This result highlights the importance of collecting and disseminating empirically derived estimates of false positive error rates to ensure that practitioners and potential jurors have a realistic impression of the value of forensic science evidence.

A Bayesian Approach to Mixed Group Validation of Performance Validity Tests

2014 ◽  
Author(s):  
Douglas Mossman ◽  
William Miller ◽  
Elliot Lee ◽  
Roger Gervais ◽  
Kathleen Hart ◽  
...  
Keyword(s):  
Bayesian Approach ◽  
Performance Validity ◽  
Mixed Group ◽  
Validity Tests ◽  
Performance Validity Tests

Error Rates in Competency Testing When Test Retaking Is Permitted

1990 ◽  
Vol 15 (1) ◽  
pp. 39-52 ◽  
Author(s):  
Huynh Huynh
Keyword(s):  
Conditional Probability ◽  
False Positive ◽  
False Negative ◽  
Error Rates ◽  
Rasch Models ◽  
Positive Error ◽  
False Positive Error ◽  
Competency Testing ◽  
Negative Error ◽  
False Negative Error

False positive and false negative error rates are studied for competency testing where examinees are permitted to retake the test if they fail to pass. Formulae are provided for the beta-binomial and Rasch models, and estimates based on these two models are compared for several typical situations. Although Rasch estimates are expected to be more accurate than beta-binomial estimates, differences among them are found not to be substantial in a number of practical situations. Under relatively general conditions and when test retaking is permitted, the probability of making a false negative error is zero. Under the same situation, and given that an examinee is a true nonmaster, the conditional probability of making a false positive error for this examinee is one.

Forensic science evidence: Naive estimates of false positive error rates and reliability

2019 ◽  
Vol 302 ◽  
pp. 109877 ◽  
Author(s):  
Kristy A. Martire ◽  
Kaye N. Ballantyne ◽  
Agnes Bali ◽  
Gary Edmond ◽  
Richard I. Kemp ◽  
...  
Keyword(s):  
Forensic Science ◽  
False Positive ◽  
Error Rates ◽  
Positive Error ◽  
False Positive Error ◽  
Science Evidence

The Rey Dot Counting Test as a Tool for Detecting Suboptimal Performance in Athlete Baseline Testing

2020 ◽  
Author(s):  
Andrew DaCosta ◽  
Frank Webbe ◽  
Anthony LoGalbo
Keyword(s):  
Cognitive Testing ◽  
Performance Validity ◽  
Baseline Assessment ◽  
Baseline Testing ◽  
Validity Measures ◽  
Significant Disagreement ◽  
Two Measures ◽  
Validity Tests ◽  
Performance Validity Tests ◽  
Suboptimal Effort

Abstract Objective The limitations of Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT)’s embedded validity measures (EVMs) are well-documented, as estimates suggest up to 35% of invalid baseline performances go undetected. Few studies have examined standalone performance validity tests (PVT) as a supplement to ImPACT’s EVMs. Method College athletes (n = 1,213) were administered a preseason baseline assessment that included ImPACT and the Rey Dot Counting Test (DCT), a standalone PVT, among other measures. Results Sixty-nine athletes (5.69%) met criteria for suboptimal effort on either ImPACT or the DCT. The DCT detected more cases of suboptimal effort (n = 50) than ImPACT (n = 21). A χ2 test of independence detected significant disagreement between the two measures, as only two individuals produced suboptimal effort on both (χ2(2) = 1.568, p = .210). Despite this disagreement, there were significant differences between the suboptimal effort DCT group and the adequate effort DCT group across all four ImPACT neurocognitive domains (U = 19,225.000, p < .001; U = 17,859.000, p < .001; U = 13,854.000, p < .001; U = 17,850.500, p < .001). Conclusions The DCT appears to detect suboptimal effort otherwise undetected by ImPACT’s EVMs.

Sign in / Sign up

Export Citation Format

Share Document