A-187 Lack of Agreement between Reliable Digit Span and Rey-15 Item Test as Validity Indicators for Test of Memory Malingering Performance in Youth Athletes

2021 ◽  
Vol 36 (6) ◽  
pp. 1242-1242
Author(s):  
Rachel M Murley ◽  
Daniel Baldini ◽  
Julius Flowers ◽  
Aiden Hall ◽  
Jamie P Kiefer ◽  
...  
Keyword(s):  
Clinical Judgment ◽  
Test Performance ◽  
Sport Participation ◽  
Digit Span ◽  
Healthy Population ◽  
Healthy Children ◽  
Performance Validity ◽  
Test Of Memory Malingering ◽  
Roc Curve Analysis ◽  
Youth Athletes

Abstract Objective In neuropsychological assessments, it is critical to evaluate examinee performance validity. Reliable Digit Span (RDS) and the Rey-15 Item Test (FIT) are performance validity tests (PVTs) used to assess the accuracy of results during neuropsychological assessment (NA; Lichtenstein et al., 2017). While many studies have examined adult populations, few examined children to determine the base rate level of PVT failure even in a healthy population. The present study assessed RDS and FIT as PVTs in youth athletes, with valid performance operationalized by Test of Memory Malingering (TOMM) performance. Method Youth athletes (n = 109, 79% male, Mage = 11.9) completed a NA including RDS (cutoff <7), TOMM Trial 2 (cutoff <50), and FIT (cutoff <26) during baseline evaluation for sport participation. A stringent TOMM cutoff was used to improve sensitivity (Schroeder et al., 2012). Receiver operator characteristic (ROC) curve analysis determined whether RDS or FIT performance accurately categorized participants’ validity test performance, based upon TOMM Pass/Fail. Results Both RDS and FIT produced minimal accuracy in categorizing participants performance on TOMM; areas under the curve ranged from 0.41 to 0.60 and did not significantly differ from chance (0.50). Additionally, there was a lack of agreement across PVTs, as no participant failed all three PVTs. Conclusion RDS and FIT did not adequately predict TOMM performance in healthy youth. These findings illustrate that even in a brief NA, healthy children demonstrated variable performance across validity measures, with low consistency across three commonly used PVTs. Findings highlight the importance of clinical judgment in evaluating discrepant PVT performance in children.

A-176 Undercover Performance Validity Testing: Proposed Embedded Validity Indicators for Pediatric Neuropsychological Assessment

2021 ◽  
Vol 36 (6) ◽  
pp. 1231-1231
Author(s):  
Naomi R Kaswan ◽  
Ryan C Thompson ◽  
Yelena Markiv ◽  
Aubrey Deenen ◽  
Haig V Pilavjian ◽  
...  
Keyword(s):  
Sport Participation ◽  
Word Reading ◽  
Digit Span ◽  
Characteristic Curve ◽  
Test Word ◽  
Healthy Children ◽  
Performance Validity ◽  
Validity Indicators ◽  
Neuropsychological Evaluations ◽  
Embedded Validity Indicators

Abstract Objective Literature supports the use of the Delis-Kaplan Executive Function System Trail Making Test Conditions 4/2 ratio (TMT 4/2) and Stroop Color Word Test Word Reading (WR) as embedded validity indicators (EVIs) with adults (Erdodi et al., 2018; Guise et al., 2012) and the Wechsler Abbreviated Scale of Intelligence, 2nd Edition Matrix Reasoning (MR) as an EVI with children (Sussman et al., 2017). This study assessed the utility of these measures as EVIs in healthy children, compared to the Test of Memory Malingering Trial 1 (TOMM1 < 45; Perna & Loughan, 2013) and Reliable Digit Span (RDS). Method Participants (n = 99, 68.7% male, Mage = 11.9) completed baseline neuropsychological evaluations for sport participation, including the aforementioned measures. Receiver operator characteristic curve analysis was used to determine whether TMT 4/2, MR, and WR accurately categorized valid performance. Results TMT 4/2 yielded adequate sensitivity (0.83–1.00) but poor specificity (0.07–0.09) when predicting TOMM1 and RDS pass/fail performances. MR yielded adequate sensitivity (1.00) and specificity (0.92) when predicting RDS pass/fail performance and adequate specificity (0.92) and poor sensitivity (0.18) when predicting TOMM1 pass/fail performance. The only EVI that produced better than chance accuracy was MR when predicting RDS pass/fail performance (area under the curve [AUC] = 0.98). All participants failed the WR cutoff, suggesting poor specificity. Conclusion Results suggest that MR was the only EVI that achieved minimally acceptable specificity (≥0.90) in children. MR performed adequately when detecting valid performances but variably when detecting invalid performances; therefore, MR may be used alongside well-established performance validity tests with children but not independently.

scholarly journals The utility of Matrix Reasoning as an embedded performance validity indicatory in youth athletes

Neurology ◽  
2019 ◽  
Vol 93 (14 Supplement 1) ◽  
pp. S16.2-S17
Author(s):  
Jennifer Adler ◽  
Ryan Thompson ◽  
Naomi Kaswan ◽  
Rayna Hirst
Keyword(s):  
Roc Curve ◽  
Recent Literature ◽  
Curve Analysis ◽  
Performance Validity ◽  
Test Of Memory Malingering ◽  
Roc Curve Analysis ◽  
Youth Athletes ◽  
T Score ◽  
Youth Athlete ◽  
Matrix Reasoning

ObjectiveThe present study assessed Matrix Reasoning (MR) as an embedded validity indicator (EVI) in youth athletes vulnerable to sport-related concussion, using performance on the Test of Memory Malingering (TOMM) to operationalize effort.BackgroundMatrices tasks have been examined as EVI for pediatric neropsychological assessment (NA; McKinsey, Prieler, & Raven, 2003), and recent literature suggests a cutoff T-score of 43 for MR in the Wechsler Abbreviated Scale of Intelligence, Second Edition (WASI-II) may demonstrate utility within youth athletes completing baseline NA.Design/Methods103 youth athletes (76% male, Mage = 12.14) completed a NA, including MR (cutoff T = 43) and TOMM (cut-offs = 45 and 49). Sensitivity and specificity for MR were calclated. Reciever operator characteristics (ROC) curve analysis determined whether MR performance accurately categorized participants' effort (represented by TOMM performance).ResultsMR (cut-off T = 43; Sussman et al., 2019) produced sensitivity of 9.09% and specificity of 91.36% in predicting TOMM Trial 1 performance (TOMM1; AUC = 0.449) and 0.00% and 91.18% in predicting TOMM Trial 2 (TOMM2; AUC = 0.074). As a TOMM2 cut-off of 49 offers greater sensitivity to inadequate effort, a further analysis showed MR yielded sensitivity of 0.00% and specificity of 91.00% (AUC = 0.330) in predicting TOMM2 performance with the more conservative cutoff.ConclusionsMR is an adequate EVI in predicting sufficient effort on TOMM, detecting true effortful performance; however, it was inadequate in detecting true non-effortful performance. A more stringent TOMMM cutoff did not improve sensitivity; thus, MR exhibited poor detection of inadequate effort. Overall, MR has utility as an EVI to support adequate effort in youth athlete populations but should not be used independently. This finding is clinically important because adequate effort at baseline is imperative in determining recovery from concussion.

scholarly journals A-69 A Follow Up Study of Reliable Digit Span Performance in Children with Academic Difficulties

2019 ◽  
Vol 34 (6) ◽  
pp. 929-929
Author(s):  
L Pease ◽  
W Ernst ◽  
M Kneavel ◽  
A Brosof
Keyword(s):  
Digit Span ◽  
Psychological Services ◽  
Pass Rate ◽  
Performance Validity ◽  
Test Of Memory Malingering ◽  
Academic Difficulties ◽  
Prior Literature ◽  
Key Variables ◽  
Cutoff Scores

Abstract Objectives This study sought to describe reliable digit span (RDS) performance in a sample of children referred for evaluation due to academic difficulties and provide preliminary information regarding its utility as a performance validity test (PVT) in this population. Methods The sample included 110 children (M = 11.91, SD = 3.34, Range = 6-16) evaluated in a college affiliated psychological services clinic. Measures included the WISC-IV or WISC-V and the WIAT-III. A subset of the sample (n = 30) was administered the Test of Memory Malingering (TOMM). Results The average RDS performance was 8.2 (SD = 2.0, Range 4-15). Using the previously established adult cutoff of ≤7, 55.5% of the sample passed. At a cutoff of ≤6, 86.4% of the sample passed, which approaches a 90% pass rate recommended in prior literature. RDS was correlated with age r(n = 110) = .53, p < .001, education r(n = 110) = .50, p < .001, FSIQ r(n = 110) = .31, p < .01, the WIAT Reading Composite r(n = 110) = .34, p < .001, and the WIAT Mathematics Composite r(n = 110) = .29, p < .01. The TOMM pass rate was 100% and all children in that subset scored above the RDS cutoff of ≤6. Conclusions RDS is potentially useful as an embedded PVT for children referred for evaluation due to academic difficulties. Key variables including age, FSIQ, and academic performance correlated with RDS, which requires consideration of those variables when interpreting RDS performance. Additional research is needed to determine appropriate cutoff scores given the potential influence of variables such as FSIQ and age on RDS performance for this population.

Convergent, Discriminant, and Concurrent Validity of Nonmemory-Based Performance Validity Tests

Assessment ◽  
2018 ◽  
Vol 27 (7) ◽  
pp. 1399-1415 ◽  
Author(s):  
Troy A. Webber ◽  
Edan A. Critchfield ◽  
Jason R. Soble
Keyword(s):  
Test Performance ◽  
Digit Span ◽  
Neuropsychological Test ◽  
Test Validity ◽  
Choice Test ◽  
Word Choice ◽  
Performance Validity ◽  
Adult Intelligence Scale ◽  
Validity Tests ◽  
Performance Validity Tests

To supplement memory-based Performance Validity Tests (PVTs) in identifying noncredible performance, we examined the validity of the two most commonly used nonmemory-based PVTs—Dot Counting Test (DCT) and Wechsler Adult Intelligence Scale–Fourth edition (WAIS-IV) Reliable Digit Span (RDS)—as well as two alternative WAIS-IV Digit Span (DS) subtest PVTs. Examinees completed DCT, WAIS-IV DS, and the following criterion PVTs: Test of Memory Malingering, Word Memory Test, and Word Choice Test. Validity groups were determined by passing 3 (valid; n = 69) or failing ⩾2 (noncredible; n = 30) criterion PVTs. DCT, RDS, RDS–Revised (RDS-R), and WAIS-IV DS Age-Corrected Scaled Score (ACSS) were significantly correlated (but uncorrelated with memory-based PVTs). Combining RDS, RDS-R, and ACSS with DCT improved classification accuracy (particularly for DCT/ACSS) for detecting noncredible performance among valid-unimpaired, but largely not valid-impaired examinees. Combining DCT with ACSS may uniquely assess and best supplement memory-based PVTs to identify noncredible neuropsychological test performance in cognitively unimpaired examinees.

A-178 Might Tip-of-the-Tongue Responses on the Auditory and Visual Naming Test be a Useful Embedded Performance Validity Measure in the Language Domain?

2021 ◽  
Vol 36 (6) ◽  
pp. 1233-1233
Author(s):  
Janice C Marceaux ◽  
Karin J M McCoy
Keyword(s):  
Test Performance ◽  
Small Sample Size ◽  
Small Sample ◽  
Performance Validity ◽  
Roc Curve Analysis ◽  
Validity Measures ◽  
Valid Test ◽  
Validity Measure ◽  
Tip Of The Tongue ◽  
Naming Test

Abstract Objective This study examined whether responses on the Auditory and Visual Naming Test (ANT/VNT) would discriminate between veterans with invalid and valid test performance, thus suggesting a novel embedded performance validity test (PVT) for a language task. Method Veterans who completed the ANT/VNT and failed two or more PVTs were selected for inclusion, resulting in a sample of 9 veterans (8 males; Mage = 55.8, SD = 11.5; Medu = 12.7, SD = 2.1). An age-and education-matched group of 9 cognitively unimpaired veterans who passed PVTs (7 males; Mage = 56.1, SD = 11.5; Medu = 14.0, SD = 2.4) were selected from a database of patients from the same clinical setting. Variables of interest included tip-of-the-tongue (TOT) responses (responses with a latency between 2 and 20 seconds) and Total Correct responses for the ANT/VNT. Results Groups were comparable on demographic characteristics and there were no statistical differences in Total Correct responses on ANT/VNT. The invalid group had more TOT on both the visual (M = 4.2, SD = 3.8) and auditory (M = 14.4,SD = 9.7) tasks, as compared to the valid group (Mvisual = 0.7, SD = 0.7; Mauditory = 4.9,SD = 2.7). ROC curve analysis for detecting invalid performance was significant for both VNT-TOT (AUC = 0.79, p = 0.038) and ANT-TOT (AUC = 0.88, p = 0.006). A cutoff of 1 was optimal for VNT-TOT (sensitivity = 0.67, specificity = 0.89), while 6 was optimal for ANT-TOT (sensitivity = 0.89, specificity = 0.89). Conclusions Despite the small sample size, findings are promising regarding the utility of ANT/VNT TOT responses as embedded PVTs. These should be further explored since the field is in need of validity measures embedded within language/naming tasks. Visual Naming Test may be particularly useful given that it can take less than 5 minutes to administer.

A Systematic Review and Meta-Analysis of the Utility of the Test of Memory Malingering in Pediatric Examinees

2020 ◽  
Vol 35 (8) ◽  
pp. 1312-1322
Author(s):  
Hilary A Clark ◽  
Phillip K Martin ◽  
Hayrettin Okut ◽  
Ryan W Schroeder
Keyword(s):  
Systematic Review ◽  
Meta Analysis ◽  
Healthy Children ◽  
Performance Validity ◽  
Test Of Memory Malingering ◽  
Eligibility Criteria ◽  
Random Effects Models ◽  
Neuropsychological Evaluations ◽  
Meta Analyses ◽  
The Impact

Abstract Objective This is the first systematic review and meta-analysis of the Test of Memory Malingering (TOMM) in pediatric examinees. It adheres to Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Method A systematic literature search was conducted using PsycINFO and PubMed, reviewing articles from January 1997 to July 2019. Books providing data on pediatric validity testing were also reviewed for references to relevant articles. Eligibility criteria included publication in a peer-reviewed journal, utilizing a pediatric sample, providing sufficient data to calculate specificity and/or sensitivity, and providing a means for evaluating validity status external to the TOMM. After selection criteria were applied, 9 articles remained for meta-analysis. Samples included clinical patients and healthy children recruited for research purposes; ages ranged from 5 to 18. Fixed and random effects models were used to calculate classification accuracy statistics. Results Traditional adult-derived cutoffs for Trial 2 and Retention were highly specific (0.96–0.99) in pediatric examinees for both clinical and research samples. Sensitivity was relatively strong (0.68–0.70), although only two studies reported sensitivity rates. A supplemental review of the literature corroborated these findings, revealing that traditional adult-based TOMM cutoffs are supported in most pediatric settings. However, limited research exists on the impact of very young age, extremely low cognitive functioning, and varying clinical diagnoses. Conclusions The TOMM, at traditional adult cutoffs, has strong specificity as a performance validity test in pediatric neuropsychological evaluations. This meta-analysis found that specificity values in children are comparable to those of adults. Areas for further research are discussed.

Coaches’ impact on youth athletes’ intentions to continue sport participation: The mediational influence of the coach–athlete relationship

2021 ◽  
pp. 174795412199181
Author(s):  
Meredith M Wekesser ◽  
Brandonn S Harris ◽  
Jody Langdon ◽  
Charles H Wilson
Keyword(s):  
Indirect Effects ◽  
Sport Participation ◽  
Self Determination Theory ◽  
Drop Out ◽  
Self Determination ◽  
Need Fulfillment ◽  
Sports Teams ◽  
Youth Athletes ◽  
The Relationship ◽  
Negative Relationships

About 70% of youth athletes drop out of sport by age 13. Self-determination theory has been utilised to investigate athletes’ motivations for behaviours including sport persistence (i.e. continuation) and suggests that the coach can be an influence on such motivations. Basic need fulfillment via interpersonal coaching behaviours, the coach-athlete relationship (CAR), and intentions to continue sport participation have been examined independently and in various combinations and directions, but these variables have not been examined collectively in this manner. The purpose of this study was to determine if CAR quality mediates the relationship between interpersonal coaching behaviours and intentions to continue sport participation. Surveys were administered to 148 athletes ages 11 to 16 from organised sports teams. No significant indirect effects of mediation could be established. However, there was a significant and direct effect of competence-supportive behaviours on intentions ( β = .341, p < .001). Overall, the total effects model was significant ( F(1,146) = 18.762, p < .001, adjusted R2 = .114). Significant positive relationships were shown among supportive coach behaviours and CAR quality in addition to CAR quality and intentions. Negative relationships were demonstrated among thwarting coach behaviours and CAR quality. Results support that coaches’ competence-supportive behaviours can positively impact CAR quality and intentions to continue sport participation in youth athletes.

Sign in / Sign up

Export Citation Format

Share Document