Critical Item Analysis Enhances the Classification Accuracy of the Logical Memory Recognition Trial as a Performance Validity Indicator

2021 ◽  
Vol 46 (5) ◽  
pp. 327-346
Author(s):  
Alexa Dunn ◽  
Sadie Pyne ◽  
Brad Tyson ◽  
Robert Roth ◽  
Ayman Shahein ◽  
...  
Keyword(s):  
Classification Accuracy ◽  
Item Analysis ◽  
Critical Item ◽  
Performance Validity ◽  
Logical Memory ◽  
Memory Recognition ◽  
A Performance

Channel Selection for Optimal EEG Measurement in Motor Imagery-Based Brain-Computer Interfaces

2020 ◽  
pp. 2150003
Author(s):  
Pasquale Arpaia ◽  
Francesco Donnarumma ◽  
Antonio Esposito ◽  
Marco Parvis
Keyword(s):  
Motor Imagery ◽  
Classification Accuracy ◽  
State Of The Art ◽  
Binary Classification ◽  
Brain Computer Interfaces ◽  
Eeg Signals ◽  
Non Invasive ◽  
Computer Interfaces ◽  
Selection For ◽  
A Performance

A method for selecting electroencephalographic (EEG) signals in motor imagery-based brain-computer interfaces (MI-BCI) is proposed for enhancing the online interoperability and portability of BCI systems, as well as user comfort. The attempt is also to reduce variability and noise of MI-BCI, which could be affected by a large number of EEG channels. The relation between selected channels and MI-BCI performance is therefore analyzed. The proposed method is able to select acquisition channels common to all subjects, while achieving a performance compatible with the use of all the channels. Results are reported with reference to a standard benchmark dataset, the BCI competition IV dataset 2a. They prove that a performance compatible with the best state-of-the-art approaches can be achieved, while adopting a significantly smaller number of channels, both in two and in four tasks classification. In particular, classification accuracy is about 77–83% in binary classification with down to 6 EEG channels, and above 60% for the four-classes case when 10 channels are employed. This gives a contribution in optimizing the EEG measurement while developing non-invasive and wearable MI-based brain-computer interfaces.

scholarly journals Detection of inadequate effort on the California Verbal Learning Test-Second edition: Forced choice recognition and critical item analysis

2006 ◽  
Vol 12 (5) ◽  
pp. 688-696 ◽  
Author(s):  
JAMES C. ROOT ◽  
REUBEN N. ROBBINS ◽  
LUKE CHANG ◽  
WILFRED G. VAN GORP
Keyword(s):  
Predictive Value ◽  
Memory Performance ◽  
Item Analysis ◽  
Verbal Learning ◽  
Critical Item ◽  
Forced Choice ◽  
Incremental Validity ◽  
California Verbal Learning Test ◽  
Lower Sensitivity ◽  
Learning Test

The Forced Choice Recognition (FCR) and the Critical Item Analysis (CIA) indices of the California Verbal Learning Test-II (CVLT-II) have been identified by the CVLT-II test developers as potentially useful, brief screening indicators of effort in neuropsychological assessment. This retrospective study analyzes performance on these measures in three groups: (1) clinically referred individuals; (2) forensically referred individuals not suspected of inadequate effort; and (3) forensically referred individuals whose performance on freestanding tests of effort suggested inadequate effort. Performances on FCR were analyzed for their relation to actual memory impairment and with regard to concrete and abstract distractor endorsement. FCR and CIA performances were analyzed for agreement with formal tests of inadequate effort and their test characteristics. Incremental validity was assessed by hierarchical logistic regression with previously identified indices for detection of inadequate effort on the CVLT. Results indicate that (1) FCR and CIA performances are not related to decreased memory performance; (2) FCR and CIA indices exhibit higher specificity and lower sensitivity, with higher positive predictive value than negative predictive value; and (3) FCR and CIA indices exhibit modest incremental validity with previously identified indices. Implications for use of FCR and CIA indices in inadequate effort detection are discussed (JINS, 2006, 12, 688–696.)

When Time is of the Essence: Preliminary Findings for a Quick Administration of the Dot Counting Test

2019 ◽  
Author(s):  
K Chase Bailey ◽  
Troy A Webber ◽  
Jacob I Phillips ◽  
Lindsay D R Kraemer ◽  
Janice C Marceaux ◽  
...  
Keyword(s):  
Classification Accuracy ◽  
Clinical Sample ◽  
Characteristic Curve ◽  
Cross Sectional Study ◽  
Receiver Operator Characteristic Curve ◽  
Performance Validity ◽  
Free Standing ◽  
Cross Sectional ◽  
Validity Measures ◽  
Sensitivity Specificity

Abstract Objective Performance validity research has emphasized the need for briefer measures and, more recently, abbreviated versions of established free-standing tests to minimize neuropsychological evaluation costs/time burden. This study examined the accuracy of multiple abbreviated versions of the Dot Counting Test (“quick” DCT) for detecting invalid performance in isolation and in combination with the Test of Memory Malingering Trial 1 (TOMMT1). Method Data from a mixed clinical sample of 107 veterans (80 valid/27 invalid per independent validity measures and structured criteria) were included in this cross-sectional study; 47% of valid participants were cognitively impaired. Sensitivities/specificities of various 6- and 4-card DCT combinations were calculated and compared to the full, 12-card DCT. Combined models with the most accurate 6- and 4-card combinations and TOMMT1 were then examined. Results Receiver operator characteristic curve analyses were significant for all 6- and 4-card DCT combinations with areas under the curve of .868–.897. The best 6-card combination (cards, 1-3-5-8-11-12) had 56% sensitivity/90% specificity (E-score cut-off, ≥14.5), and the best 4-card combination (cards, 3-4-8-11) had 63% sensitivity/94% specificity (cut-off, ≥16.75). The full DCT had 70% sensitivity/90% specificity (cut-off, ≥16.00). Logistic regression revealed 95% classification accuracy when 6-card or 4-card “quick” combinations were combined with TOMMT1, with the DCT combinations and TOMMT1 both emerging as significant predictors. Conclusions Abbreviated DCT versions utilizing 6- and 4-card combinations yielded comparable sensitivity/specificity as the full DCT. When these “quick” DCT combinations were further combined with an abbreviated memory-based performance validity test (i.e., TOMMT1), overall classification accuracy for identifying invalid performance was 95%.

A-112 Examining the Utility of the New Dot Counting Test Cut-off Score in Latinx and Traumatic Brain Injury

2021 ◽  
Vol 36 (6) ◽  
pp. 1161-1161
Author(s):  
Sarah Saravia ◽  
Daniel W Lopez-Hernandez ◽  
Abril J Baez ◽  
Isabel Muñoz ◽  
Winter Olmos ◽  
...  
Keyword(s):  
Traumatic Brain Injury ◽  
Brain Injury ◽  
Ethnic Groups ◽  
Group Differences ◽  
Failure Rates ◽  
Performance Validity ◽  
Validity Test ◽  
Chi Squared ◽  
Future Work ◽  
A Performance

Abstract Objective The Dot Counting Test (DCT) is a performance validity test. McCaul et al. (2018) recently revised the DCT cut-off score from ≥17 to 13.80; we evaluated the new cut-off in non-Latinx Caucasian and Caucasian Latinx traumatic brain injury (TBI) survivors and healthy comparison (HC) participants. Method The sample consisted of 37 acute TBI (ATBI; 11 Caucasian Latinx; 26 non-Latinx Caucasian), 27 chronic TBI (CTBI; 10 Caucasian Latinx; 17 non-Latinx Caucasian), and 55 HC (29 Caucasian Latinx; 26 non-Latinx Caucasian) participants. Results An ANCOVA, controlling for age, revealed no DCT E-scores differences between groups. Both the conventional and the new cut-off scores had different failure rates in ATBI (conventional cut-off: 0%; PNC: 16%), CTBI (conventional cut-off: 7%; PNC: 15%), and HC (conventional cut-off: 10%; PNC: 11%) participants. For the Caucasian Latinx group (conventional cut-off: 6%; PNC: 12%) and the non-Latinx Caucasian group (conventional cut-off: 6%; PNC: 14%), demonstrated different failure rates across cut-off scores. Group differences were found with the McCaul et al. (2018) cut-off and the conventional cut-off. Also, chi-squared analysis revealed non-Latinx Caucasian participants with ATBI had greater failure rates than Caucasian Latinx participants with ATBI. Conclusion The new DCT cut-off score resulted in greater failure rates in TBI survivors. Also, this effect appears to be most pronounced in non-Latinx Caucasian persons with ATBI. Future work should investigate possible reasons for these differences so that more stringent DCT can be utilized in a way that provides less biased results for brain injury survivors across racial and ethnic groups.

TOMM Trial 1 as a performance validity indicator in a criminal forensic sample

2016 ◽  
Vol 31 (1) ◽  
pp. 251-267 ◽  
Author(s):  
Rachel L. Fazio ◽  
John H. Denning ◽  
Robert L. Denney
Keyword(s):  
Performance Validity ◽  
Forensic Sample ◽  
A Performance

scholarly journals A-186 An Enhanced Delayed Recognition Measure for the WMS-IV Logical Memory Subtest

2020 ◽  
Vol 35 (6) ◽  
pp. 981-981
Author(s):  
Gass C ◽  
Patten B ◽  
Penate A ◽  
Rhodes A
Keyword(s):  
Free Recall ◽  
Recognition Test ◽  
Recall Performance ◽  
Memory Storage ◽  
Performance Validity ◽  
Logical Memory ◽  
Storage And Retrieval ◽  
New Information ◽  
Delayed Recognition ◽  
Neurologically Normal

Abstract Objective We introduce a supplemental measure based on the Logical Memory (LM) subtest of the Wechsler Memory Scale – IV (Wechsler, 2008) to assist in distinguishing deficient memory storage from compromised retrieval operations. A 20-item five-option multiple choice delayed recognition test for the LM stories is described, followed by descriptive data based on a normative sample of 168 female and 105 male neurologically normal outpatient referrals to a neuropsychology clinic. Method Ten 5-option items were developed for each of the three LM stories (A, B, and C) reflecting the stories’ narrative content, yielding 20 items for administration to examinees under 70 (Stories B and C) and older than 69 (Stories A and B). Four distractor options for each item were designed to be “plausible” alternatives for the examinee who had little or no recollection of the narrative material. Examinees (N=360), screened for performance validity (effort), completed the LM Recognition test immediately after delayed LM free recall. Results Internal consistency (Cronbach alpha) was acceptable for both 20-item LM Recognition tests. Individuals’ scaled scores on LM-Recognition were compared with LM-II (free recall). Recognition was superior (> one SD) to free recall performance in 43% (age 18 to 69) and 48.2% (age 70+). Conclusion Delayed free recall scores often underestimate a person’s ability to encode and store new information. For clinical application, separate LM Recognition norms were derived for older (Stories A and B) and younger examinees (stories B and C). This 20-item measure assists clinicians in differentiating between storage and retrieval deficits on the Logical Memory subtest.

Sign in / Sign up

Export Citation Format

Share Document