A-176 Undercover Performance Validity Testing: Proposed Embedded Validity Indicators for Pediatric Neuropsychological Assessment

Objective Literature supports the use of the Delis-Kaplan Executive Function System Trail Making Test Conditions 4/2 ratio (TMT 4/2) and Stroop Color Word Test Word Reading (WR) as embedded validity indicators (EVIs) with adults (Erdodi et al., 2018; Guise et al., 2012) and the Wechsler Abbreviated Scale of Intelligence, 2nd Edition Matrix Reasoning (MR) as an EVI with children (Sussman et al., 2017). This study assessed the utility of these measures as EVIs in healthy children, compared to the Test of Memory Malingering Trial 1 (TOMM1 < 45; Perna & Loughan, 2013) and Reliable Digit Span (RDS). Method Participants (n = 99, 68.7% male, Mage = 11.9) completed baseline neuropsychological evaluations for sport participation, including the aforementioned measures. Receiver operator characteristic curve analysis was used to determine whether TMT 4/2, MR, and WR accurately categorized valid performance. Results TMT 4/2 yielded adequate sensitivity (0.83–1.00) but poor specificity (0.07–0.09) when predicting TOMM1 and RDS pass/fail performances. MR yielded adequate sensitivity (1.00) and specificity (0.92) when predicting RDS pass/fail performance and adequate specificity (0.92) and poor sensitivity (0.18) when predicting TOMM1 pass/fail performance. The only EVI that produced better than chance accuracy was MR when predicting RDS pass/fail performance (area under the curve [AUC] = 0.98). All participants failed the WR cutoff, suggesting poor specificity. Conclusion Results suggest that MR was the only EVI that achieved minimally acceptable specificity (≥0.90) in children. MR performed adequately when detecting valid performances but variably when detecting invalid performances; therefore, MR may be used alongside well-established performance validity tests with children but not independently.

A-222 Efficacy of D–KEFS Trail Making Test Conditions 1–5 Using Age-Corrected Scaled Score Cutoffs as Embedded Validity Indicators in a Sample of Youth Athletes

Objective Embedded validity indicators (EVI) assist in the evaluation of performance validity across a neuropsychological battery without increasing the time or cognitive demand on patients during testing. Erdodi et al. (2018) evaluated Delis–Kaplan Executive Function System (D–KEFS) Trail Making Test Conditions 1–5 (TMT1–5) as EVIs in adults using age-corrected scaled score (ACSS) cutoffs, demonstrating adequate specificity but inadequate sensitivity. This study assessed the TMT1–5 as EVIs in children using a specificity threshold of > .89 (Boone, 2013). Method Youth athletes (n = 134, M age = 12.2) completed a two-hour neuropsychological battery that included TMT1–5 and TOMM during a larger sport-related concussion clinical research evaluation. Specificity, sensitivity, and area under the curve (AUC) were calculated for TMT1–5 using ACSS cutoffs to predict TOMM Trial 2 (TOMM2) pass/fail performance. Results When predicting TOMM2 (cutoff.89) and inadequate sensitivity (.00), with TMT3 and TMT4 having the highest AUC (.81 and .74, respectively). The following ACSS cutoffs met the specificity threshold (>.89): TMT1 ACSS < 7, TMT2 < 9, TMT3 < 6, TMT4 < 7, and TMT5.79, we increased sensitivity for TMT1–3 (.33). Conclusion D–KEFS TMT1–5 achieved excellent specificity (>.89) when predicting TOMM2 performance but demonstrated inadequate sensitivity. Lowering the specificity threshold slightly still yielded low sensitivity. These data suggest that D–KEFS TMT3 and TMT4 were most effective at detecting adequate effort, but should be supported with other validity measures when assessing performance validity across a neuropsychological battery.