A-187 Lack of Agreement between Reliable Digit Span and Rey-15 Item Test as Validity Indicators for Test of Memory Malingering Performance in Youth Athletes

Objective In neuropsychological assessments, it is critical to evaluate examinee performance validity. Reliable Digit Span (RDS) and the Rey-15 Item Test (FIT) are performance validity tests (PVTs) used to assess the accuracy of results during neuropsychological assessment (NA; Lichtenstein et al., 2017). While many studies have examined adult populations, few examined children to determine the base rate level of PVT failure even in a healthy population. The present study assessed RDS and FIT as PVTs in youth athletes, with valid performance operationalized by Test of Memory Malingering (TOMM) performance. Method Youth athletes (n = 109, 79% male, Mage = 11.9) completed a NA including RDS (cutoff <7), TOMM Trial 2 (cutoff <50), and FIT (cutoff <26) during baseline evaluation for sport participation. A stringent TOMM cutoff was used to improve sensitivity (Schroeder et al., 2012). Receiver operator characteristic (ROC) curve analysis determined whether RDS or FIT performance accurately categorized participants’ validity test performance, based upon TOMM Pass/Fail. Results Both RDS and FIT produced minimal accuracy in categorizing participants performance on TOMM; areas under the curve ranged from 0.41 to 0.60 and did not significantly differ from chance (0.50). Additionally, there was a lack of agreement across PVTs, as no participant failed all three PVTs. Conclusion RDS and FIT did not adequately predict TOMM performance in healthy youth. These findings illustrate that even in a brief NA, healthy children demonstrated variable performance across validity measures, with low consistency across three commonly used PVTs. Findings highlight the importance of clinical judgment in evaluating discrepant PVT performance in children.

A-181 Do Effortful Epilepsy Patients Fall Below Performance Validity Test Cutoffs at Higher Rates than Is Considered Acceptable?

Objective Factors specific to an epilepsy population (e.g., medications, psychiatric comorbidities, localization) may result in higher false positive rates on performance validity tests (PVT), rendering the results more equivocal. This study examined whether specificity is reduced in effortful epilepsy patients on the Warrington Recognition Memory Test - Words (WRMT-W) and Test of Memory Malingering (TOMM). Method 53 epilepsy patients referred for neuropsychological evaluation through the University of Washington Regional Epilepsy Center were examined. Patients were majority male (56.6%) and Caucasian (79.2%). Average age and education were 36.1 (SD = 13.03) and 13.4 years (SD = 2.39), respectively. Patients with an intelligence quotient of <70, history of brain surgery, and those who seized during testing were excluded. Patients clinically observed to have reduced effort with 2+ PVT failures were excluded (n = 3). Frequency tables for WRMT-W and TOMM performances were utilized to examine specificity based on prior cutoffs identified for these measures. Results The WRMT-W cutoff of ≤42 was associated with 88.7% specificity. TOMM Trial 2 and Retention cutoffs of <45 were associated with 98.1% and 100% specificity, respectively. The WRMT-W cutoff was associated with 91.7% specificity in language dominant hemisphere onset epilepsy patients (n = 16). None performed below cutoffs on the TOMM. All nondominant hemisphere onset patients (n = 8) performed above WRMT-W and TOMM cutoffs. Conclusions Use of the WRMT-W and TOMM in an epilepsy population is associated with an acceptable false positive rate (specificity around 90%). However, future studies examining the sensitivity of these measures in epilepsy patients should be performed.

A-185 An Analysis of Rarely Missed Items on the TOMM

Objective This study examined item performance on Trial 1 of the Test of Memory Malingering (TOMM). We also identified items that were most often missed in individuals with genuine effort. Method Participants were 106 adults seen for disability claims (87.7% male; 70.5% Caucasian, 26.7% Black; age range 22–84 years, Mage = 44.42 years, SD = 13.07; Meducation = 13.58, SD = 2.05) who completed and passed the TOMM as part of a larger battery. Mean score Trial 1 was 43.08, SD = 5.49. Mean score on Trial 2 was 48.98, SD = 1.54. Results Frequency analysis indicated that >95% of the sample correctly identified six items on Trial 1: item 1-spinning wheel (97.2%), item 8-musical notes (99.1%), item 38-ice cream (98.1%), item 41-life preserver (95.3%), item 45-iron (95.3%), and item 47-dart (98.1%). Nine items were correctly identified on Trial 1 by <80% of the sample: item 2-tissue box (77.4%), item 6-suitcase (77.4%), item 20-motorcycle (77.4%), item 22-jack-in-the box (71.7%), item 26-light bulb (75.5%), item 27-maple leaf (72.6%), item 32-racket (79.2%), item 36-birdhouse (79.2%), item 44-pail & shovel (66.0%). Conclusions These findings suggest that items on Trial 1 of the TOMM differ in difficulty in a disability claims sample who performed genuinely on the TOMM. Items 1, 8, 38, 41, 45, and 47 are good candidates for a rarely missed index where failure of these items would be probabilistically unlikely. Future research should evaluate whether these items are failed at higher rates in cases of borderline TOMM performance to improve sensitivity to feigning.

RBANS Validity Measures and the TOMM in Veterans Undergoing Seizure Monitoring

Objective To assess the validity of embedded measures of performance validity, the effort index (EI) and effort scale (ES) of the repeatable battery for the assessment of neuropsychological status (RBANS), in Veterans with seizures; to compare the frequency of failure on the test of memory malingering (TOMM) in patients with epileptic versus psychogenic nonepileptic seizures (PNES). Methods Seizure diagnosis was established for 309 participants in epilepsy monitoring units using conventional diagnostic criteria who completed both the TOMM and RBANS. The criterion for performance invalidity was failure on any trial of the TOMM. We examined multiple EI and ES cutoffs to establish optimal sensitivity and specificity. Results An RBANS EI cutoff score of greater than three was optimal with specificity of .98, sensitivity of.19, and positive Likelihood Ratio of 10 but was not useful when below this cutoff. Confidence intervals indicate the need for confirmation of a failed EI with another performance validity test (PVT). No ES cutoff had sufficient specificity for clinical use. Invalid TOMM performance but not invalid RBANS performance was significantly more common in persons with PNES than in persons with epileptic seizures. Conclusions In Veterans undergoing seizure monitoring, the RBANS EI was useful as a screen when positive that requires confirmation with another PVT. The RBANS ES was not useful. Invalid performance on the TOMM was more common in persons with PNES than in persons with epileptic seizures.

A Systematic Review and Meta-Analysis of the Utility of the Test of Memory Malingering in Pediatric Examinees

Objective This is the first systematic review and meta-analysis of the Test of Memory Malingering (TOMM) in pediatric examinees. It adheres to Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Method A systematic literature search was conducted using PsycINFO and PubMed, reviewing articles from January 1997 to July 2019. Books providing data on pediatric validity testing were also reviewed for references to relevant articles. Eligibility criteria included publication in a peer-reviewed journal, utilizing a pediatric sample, providing sufficient data to calculate specificity and/or sensitivity, and providing a means for evaluating validity status external to the TOMM. After selection criteria were applied, 9 articles remained for meta-analysis. Samples included clinical patients and healthy children recruited for research purposes; ages ranged from 5 to 18. Fixed and random effects models were used to calculate classification accuracy statistics. Results Traditional adult-derived cutoffs for Trial 2 and Retention were highly specific (0.96–0.99) in pediatric examinees for both clinical and research samples. Sensitivity was relatively strong (0.68–0.70), although only two studies reported sensitivity rates. A supplemental review of the literature corroborated these findings, revealing that traditional adult-based TOMM cutoffs are supported in most pediatric settings. However, limited research exists on the impact of very young age, extremely low cognitive functioning, and varying clinical diagnoses. Conclusions The TOMM, at traditional adult cutoffs, has strong specificity as a performance validity test in pediatric neuropsychological evaluations. This meta-analysis found that specificity values in children are comparable to those of adults. Areas for further research are discussed.