The test–retest reliability of the latent construct of executive function depends on whether tasks are represented as formative or reflective indicators

ABSTRACTBackground:Most neuropsychological tests of executive function are time-consuming and otherwise unsuitable for routine bedside assessment, particularly in older people admitted to a general hospital. After introducing the Executive Interview (EXIT25) in our routine clinical practice, we hypothesized that it would be possible to shorten it for easier administration in our group of patients while maintaining or improving properties of the scale.Methods:The EXIT25 was applied in 122 clinically stable medical and surgical inpatients aged 65 years and over referred to the Liaison Psychiatry Service for Older People. Individual items were initially tested for floor and ceiling effects, inter-rater and test-retest reliability, and item-total correlations. Items were then selected for retention in the brief scale on the basis of adequate item-total correlation and inter-rater and test-retest reliability. The construct validity of the original and brief versions of the EXIT25 scale was assessed.Results:The original EXIT25 scale was found to lack desirable scaling properties either as a classical or hierarchical scale. The study confirmed a possibility to reduce the number of items to nine out of the original 25 while improving internal consistency, test-retest and inter-rater reliability, and maintaining high correlation with the original EXIT25 score, and moderate inverse correlation with the Mini-Mental State Examination score.Conclusions:There is potential to abbreviate the original EXIT25, and improve internal consistency and hierarchical scaling properties. Future research is necessary to focus on piloting these brief measures of executive function in relevant clinical settings, when administered en bloc, rather than with items interspersed in the original longer version.

Download Full-text

Test–Retest Reliability and Measurement Invariance of Executive Function Tasks in Young Children With and Without ADHD

Journal of Attention Disorders ◽

10.1177/1087054715627488 ◽

2016 ◽

Vol 24 (13) ◽

pp. 1891-1904 ◽

Cited By ~ 5

Author(s):

Sarah L. Karalunas ◽

Karen L. Bierman ◽

Cynthia L. Huang-Pollock

Keyword(s):

Executive Function ◽

Young Children ◽

Measurement Invariance ◽

Latent Variable ◽

Retest Reliability ◽

Mimic Models ◽

Multiple Indicators ◽

Confirmatory Factor ◽

Two Factors ◽

Test Retest Reliability

Objective: Measurement reliability is assumed when executive function (EF) tasks are used to compare groups or to examine relationships between cognition and etiologic and maintaining factors of psychiatric disorders. However, the test–retest reliabilities of many commonly used EF tasks have rarely been examined in young children. Furthermore, measurement invariance between typically developing and psychiatric populations has not been examined. Method: Test–retest reliability of a battery of commonly used EF tasks was assessed in a group of children between the ages of 5 and 6 years old with ( n = 63) and without ( n = 44) ADHD. Results: Few individual tasks achieved adequate reliability. However, confirmatory factor analysis (CFA) models identified two factors, working memory and inhibition, with test–retest correlations approaching 1.0. Multiple indicators multiple causes (MIMIC) models confirmed configural measurement invariance between the groups. Conclusion: Problems created by poor reliability, including reduced power to detect group differences, index change over time, or to identify relationships with other measures, may be mitigated using latent variable approaches.

Download Full-text

Multimodal immersive trail making – virtual reality paradigm to study cognitive-motor interactions

10.1101/2020.05.27.118760 ◽

2020 ◽

Author(s):

Meir Plotnik ◽

Oran Ben-Gal ◽

Glen M. Doniger ◽

Amihai Gottlieb ◽

Yotam Bahat ◽

...

Keyword(s):

Virtual Reality ◽

Executive Function ◽

Construct Validity ◽

Repeated Measures ◽

Discriminant Validity ◽

Middle Aged ◽

Retest Reliability ◽

Pencil And Paper ◽

Test Retest Reliability ◽

Completion Times

ABSTRACTBackgroundNeuropsychological tests of executive function have limited real-world predictive and functional relevance. An emerging solution for this limitation is to adapt the tests for implementation in virtual reality (VR). We thus developed two VR-based versions of the classic Color-Trails Test (CTT), a well-validated pencil-and-paper executive function test assessing sustained (Trails A) and divided (Trails B) attention - one for a large-scale VR system (DOME-CTT) and the other for a portable head-mount display VR system (HMD-CTT). We then evaluated construct validity, test-retest reliability, and age-related discriminant validity of the VR-based versions and explored effects on motor function.MethodsHealthy adults (n=147) in three age groups (young: n=50; middle-aged: n=80; older: n=17) participated. All participants were administered the original CTT, some completing the DOME-CTT (14 young, 29 middle-aged) and the rest completing the HMD-CTT. Primary outcomes were Trails A and B completion times (tA, tB). Spatiotemporal characteristics of upper-limb reaching movements during VR test performance were reconstructed from motion capture data. Statistics included correlations and repeated measures analysis of variance.ResultsConstruct validity was substantiated by moderate correlations between the ‘gold standard’ pencil-and-paper CTT and the VR adaptations (DOME-CTT: tA 0.58, tB 0.71; HMD-CTT: tA 0.62, tB 0.69). VR versions showed relatively high test-retest reliability (intraclass correlation; VR: tA 0.60-0.75, tB 0.59-0.89; original: tA 0.75-0.85, tB 0.77-0.80) and discriminant validity (area under the curve; VR: tA 0.70-0.92, tB 0.71-0.92; original: tA 0.73-0.95, tB 0.77-0.95). VR completion times were longer than for the original pencil-and-paper test; completion times were longer with advanced age. Compared with Trails A, Trails B target-to-target VR hand trajectories were characterized by delayed, more erratic acceleration and deceleration, consistent with the greater executive function demands of divided vs. sustained attention; acceleration onset later for older participants.ConclusionsThe present study demonstrates the feasibility and validity of converting a neuropsychological test from two-dimensional pencil-and-paper to three-dimensional VR-based format while preserving core neuropsychological task features. Findings on the spatiotemporal morphology of motor planning/execution during the cognitive tasks may lead to multimodal analysis methods that enrich the ecological validity of VR-based neuropsychological testing, representing a novel paradigm for studying cognitive-motor interactions.

Download Full-text

Catch the ruler: concurrent validity and test–retest reliability of the ReacStick measures of reaction time and inhibitory executive function in older people

Aging Clinical and Experimental Research ◽

10.1007/s40520-018-1050-6 ◽

2018 ◽

Vol 31 (8) ◽

pp. 1147-1154 ◽

Cited By ~ 4

Author(s):

Kimberley S. van Schooten ◽

Lionne Duran ◽

Maike Visschedijk ◽

Mirjam Pijnappels ◽

Stephen R. Lord ◽

...

Keyword(s):

Executive Function ◽

Reaction Time ◽

Older People ◽

Concurrent Validity ◽

Retest Reliability ◽

Test Retest Reliability

Download Full-text

Test-retest reliability of a new executive function battery for use in early childhood

Child Neuropsychology ◽

10.1080/09297049.2011.554390 ◽

2011 ◽

Vol 17 (6) ◽

pp. 564-579 ◽

Cited By ~ 45

Author(s):

Michael Willoughby ◽

Clancy Blair

Keyword(s):

Early Childhood ◽

Executive Function ◽

Retest Reliability ◽

Test Retest Reliability

Download Full-text

Test–Retest Reliability and Practice Effects of Executive Function Tasks in Preschool Children

The Clinical Neuropsychologist ◽

10.1080/13854046.2011.645558 ◽

2012 ◽

Vol 26 (2) ◽

pp. 271-287 ◽

Cited By ~ 34

Author(s):

Ulrich Müller ◽

Kimberly A. Kerns ◽

Katelyn Konkin

Keyword(s):

Executive Function ◽

Preschool Children ◽

Practice Effects ◽

Retest Reliability ◽

Test Retest Reliability

Download Full-text

Development and Evaluation of an Ecological Task to Assess Executive Functioning Post Childhood TBI: The Children's Cooking Task

Brain Impairment ◽

10.1375/brim.11.2.125 ◽

2010 ◽

Vol 11 (2) ◽

pp. 125-143 ◽

Cited By ~ 28

Author(s):

Mathilde P. Chevignard ◽

Cathy Catroppa ◽

Jane Galvin ◽

Vicki Anderson

Keyword(s):

Executive Function ◽

Executive Functioning ◽

Executive Functions ◽

Concurrent Validity ◽

Discriminant Validity ◽

Ecological Validity ◽

Retest Reliability ◽

Behavior Rating Inventory ◽

Severe Tbi ◽

Test Retest Reliability

AbstractPurpose:Traumatic brain injury (TBI) often leads to executive functions deficits, which may be responsible for severe and longstanding disabilities in everyday activities. Sensitivity and ecological validity of neuropsychological tests of executive functions have been questioned. The aims of this study were to pilot an ecological open-ended assessment of executive functions in children, the ‘Children's Cooking Task’ (CCT), specifically to report its reliability, discriminant validity and concurrent validity.Methods:Twenty-five children with mild (n= 10) or moderate-to-severe TBI (n= 15), and 21 matched controls (aged 8 to 20 years) participated in the study. An open-ended cooking task was designed to test multi-tasking abilities. It required the preparation of two simple recipes using specific instructions. Outcome measures included the number of errors and an overall qualitative analysis of the task. Validating measures of executive functions included the Delis Kaplan Executive Function System, the Six-Part Test and two questionnaires completed by the child's primary care-giver: the Behavior Rating Inventory of Executive Function and the Dysexecutive Questionnaire for Children.Results:Internal consistency of the Children's Cooking Task was high (Cronbach's alpha = .86), as was test–retest reliability (ICC = .89). Children with moderate-to-severe TBI, as well as children with mild TBI made significantly more errors in the Children's Cooking Task in comparison to controls (p< .001). The CCT was correlated with several tests and one questionnaire of executive functioning (Trails, verbal fluency, sorting, 20 questions, Dysexecutive Questionnaire).Discussion and Conclusion:The Children's Cooking Task has good interrater and test–retest reliability, as well as good discriminant and concurrent validity.

Download Full-text