Detecting significant change in neuropsychological test performance: A comparison of four models

1999 ◽  
Vol 5 (4) ◽  
pp. 357-369 ◽  
Author(s):  
NANCY R. TEMKIN ◽  
ROBERT K. HEATON ◽  
IGOR GRANT ◽  
SUREYYA S. DIKMEN
Keyword(s):  
Multiple Regression ◽  
Confidence Intervals ◽  
Test Performance ◽  
Neuropsychological Test ◽  
Error Rates ◽  
Practice Effect ◽  
Reliable Change Index ◽  
Reliable Change ◽  
The Individual

A major use of neuropsychological assessment is to measure changes in functioning over time; that is, to determine whether a difference in test performance indicates a real change in the individual or just chance variation. Using 7 illustrative test measures and retest data from 384 neurologically stable adults, this paper compares different methods of predicting retest scores, and of determining whether observed changes in performance are unusual. The methods include the Reliable Change Index, with and without correction for practice effect, and models based upon simple and multiple regression. For all test variables, the most powerful predictor of follow-up performance was initial performance. Adding demographic variables and overall neuropsychological competence at baseline significantly but slightly improved prediction of all follow-up scores. The simple Reliable Change Index without correction for practice performed least well, with high error rates and large prediction intervals (confidence intervals). Overall prediction accuracy was similar for the other three methods; however, different models produce large differences in predicted scores for some individuals, especially those with extremes of initial test performance, overall competency, or demographics. All 5 measures from the Halstead–Reitan Battery had residual (observed − predicted score) variability that increased with poorer initial performance. Two variables showed significant nonnormality in the distribution of residuals. For accurate prediction with smallest prediction–confidence intervals, we recommend multiple regression models with attention to differential variability and nonnormality of residuals. (JINS, 1999, 5, 357–369.)

Die Wirksamkeit eines Frühinterventionsprogramms für Jugendliche mit Computerspiel- und Internetabhängigkeit: Mittelfristige Effekte der PROTECT+ Studie

2020 ◽  
Vol 48 (1) ◽  
pp. 3-14 ◽  
Author(s):  
Carolin Szász-Janocha ◽  
Eva Vonderlin ◽  
Katajun Lindenberg
Keyword(s):  
Internet Use ◽  
Health Problems ◽  
Reliable Change Index ◽  
Gaming Disorder ◽  
Statistical Classification ◽  
Reliable Change ◽  
Classification Of Diseases ◽  
International Statistical Classification

Zusammenfassung. Fragestellung: Das junge Störungsbild der Computerspiel- und Internetabhängigkeit hat in den vergangenen Jahren in der Forschung zunehmend an Aufmerksamkeit gewonnen. Durch die Aufnahme der „Gaming Disorder“ in die ICD-11 (International Statistical Classification of Diseases and Related Health Problems) wurde die Notwendigkeit von evidenzbasierten und wirksamen Interventionen avanciert. PROTECT+ ist ein kognitiv-verhaltenstherapeutisches Gruppentherapieprogramm für Jugendliche mit Symptomen der Computerspiel- und Internetabhängigkeit. Die vorliegende Studie zielt auf die Evaluation der mittelfristigen Effekte nach 4 Monaten ab. Methodik: N = 54 Patientinnen und Patienten im Alter von 9 bis 19 Jahren (M = 13.48; SD = 1.72) nahmen an der Frühinterventionsstudie zwischen April 2016 und Dezember 2017 in Heidelberg teil. Die Symptomschwere wurde zu Beginn, zum Abschluss der Gruppentherapie sowie nach 4 Monaten anhand von standardisierten Diagnostikinstrumenten erfasst. Ergebnisse: Mehrebenenanalysen zeigten eine signifikante Reduktion der Symptomschwere anhand der Computerspielabhängigkeitsskala (CSAS) nach 4 Monaten. Im Selbstbeurteilungsbogen zeigte sich ein kleiner Effekt (d = 0.35), im Elternurteil ein mittlerer Effekt (d = 0.77). Der Reliable Change Index, der anhand der Compulsive Internet Use Scale (CIUS) berechnet wurde, deutete auf eine starke Heterogenität im individuellen Symptomverlauf hin. Die Patientinnen und Patienten bewerteten das Programm zu beiden Follow-Up-Messzeitpunkten mit einer hohen Zufriedenheit. Schlussfolgerungen: Die vorliegende Arbeit stellt international eine der wenigen Studien dar, die eine Reduktion der Symptome von Computerspiel- und Internetabhängigkeit im Jugendalter über 4 Monate belegen konnte.

scholarly journals Validating the Reliable Change Index with Tensor-Based Morphometry: the ADNI-MEM and the ADNI-EF

2019 ◽  
Vol 34 (6) ◽  
pp. 847-847
Author(s):  
S Lin ◽  
E Fletcher ◽  
B Gavett
Keyword(s):  
Neuropsychological Test ◽  
Regional Analysis ◽  
Brain Regions ◽  
Reliable Change Index ◽  
Cognitive Test ◽  
Parahippocampal Gyrus ◽  
Reliable Change ◽  
Confounding Variables ◽  
One Year ◽  
Neuroanatomical Data

Abstract Objective The Reliable Change Index (RCI) is a commonly used method for interpreting change in neuropsychological test scores over time. However, the RCI is a psychometric phenomenon that, to date, has not been validated by neuroanatomical evidence. Method Longitudinal neuroimaging and neuropsychological data from baseline and one-year follow-up visits were retrieved from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database. The RCI was used to identify participants showing reliable decline on ADNI-MEM and ADNI-EF factor scores, which provide composite measures of memory and executive functioning, respectively. For each cognitive test score, two groups (reliable change vs. no reliable change) were matched on potential confounding variables using a genetic algorithm. Longitudinal neuroanatomical data were analyzed using tensor-based morphometry. Results Whole brain analysis revealed that reliable change on ADNI-MEM was associated with extended atrophy of the temporal lobe, the parahippocampal gyrus, the entorhinal cortex, and the posterior cingulate gyrus (Left Figure). Similar extended atrophy patterns were found for reliable change on ADNI-EF, except that the atrophy was more extensive and of higher magnitude (Right Figure). Regional analysis further confirmed that in such brain regions, the reliable change group manifested higher grey matter loss than the no change group. Conclusion The current study not only validated clinical usage of the RCI with neuroanatomical evidence but also practically suggested patterns of likely brain atrophy when reliable cognitive decline is detected.

scholarly journals Predictors of Neuropsychological Improvement Following Cognitive Rehabilitation in Patients with Gliomas

2010 ◽  
Vol 17 (2) ◽  
pp. 256-266 ◽  
Author(s):  
Karin Gehring ◽  
Neil K. Aaronson ◽  
Chad M. Gundy ◽  
Martin J.B. Taphoorn ◽  
Margriet M. Sitskoorn
Keyword(s):  
Test Performance ◽  
Cognitive Rehabilitation ◽  
Practical Implication ◽  
Neuropsychological Test ◽  
Patient Characteristics ◽  
Rehabilitation Program ◽  
Control Group ◽  
Specific Patient ◽  
Individual Level ◽  
The Individual

AbstractThis study investigated the specific patient factors that predict responsiveness to a cognitive rehabilitation program. The program has previously been demonstrated to be successful at the group level in patients with gliomas, but it is unclear which patient characteristics optimized the effect of the intervention at the individual level. Four categories of possible predictors of improvement were selected for evaluation: sociodemographic and clinical variables, self-reported cognitive symptoms, and objective neuropsychological test performance. Hierarchical logistic regression analyses were conducted, beginning with the most accessible (sociodemographic) variables and ending with the most difficult (baseline neuropsychological) to identify in clinical practice. Nearly 60% of the participants of the intervention were classified as reliably improved. Reliable improvement was predicted by age (p = .003) and education (p = .011). Additional results suggested that younger patients were more likely to benefit specifically from the cognitive rehabilitation program (p = .001), and that higher education was also associated with improvement in the control group (p = .024). The findings are discussed in light of brain reserve theory. A practical implication is that cognitive rehabilitation programs should take the patients’ age into account and, if possible, adapt programs to increase the likelihood of improvement among older participants. (JINS, 2011, 17, 256–266)

scholarly journals Standard error in the Jacobson and Truax Reliable Change Index: The “classical approach” leads to poor estimates

2004 ◽  
Vol 10 (6) ◽  
pp. 899-901 ◽  
Author(s):  
NANCY R. TEMKIN
Keyword(s):  
Measurement Error ◽  
Classical Theory ◽  
Error Rate ◽  
Standard Error ◽  
Practice Effect ◽  
Practice Effects ◽  
Reliable Change Index ◽  
Classical Approach ◽  
Reliable Change

Different authors have used different estimates of variability in the denominator of the Reliable Change Index (RCI). Maassen attempts to clarify some of the differences and the assumptions underlying them. In particular he compares the ‘classical’ approach using an estimate SEd supposedly based on measurement error alone with an estimate SDiff based on the variability of observed differences in a population that should have no true change. Maassen concludes that not only is SEd based on classical theory, but it properly estimates variability due to measurement error and practice effect while SDiff overestimates variability by accounting twice for the variability due to practice. Simulations show Maassen to be wrong on both accounts. With an error rate nominally set to 10%, RCI estimates using SDiff wrongly declare change in 10.4% and 9.4% of simulated cases without true change while estimates using SEd wrongly declare change in 17.5% and 12.3% of the simulated cases (p < .000000001 and p < .008, respectively). In the simulation that separates measurement error and practice effects, SEd estimates the variability of change due to measurement error to be .34, when the true variability due to measurement error was .014. Neuropsychologists should not use SEd in the denominator of the RCI. (JINS, 2004, 10, 899–901.)

scholarly journals Determining the Presence of Reliable Change over Time in Multiple Sclerosis

2013 ◽  
Vol 15 (4) ◽  
pp. 170-178 ◽  
Author(s):  
Suzanne L. Barker-Collo ◽  
Suzanne C. Purdy
Keyword(s):  
Multiple Sclerosis ◽  
Reliable Change Index ◽  
Test Results ◽  
Reliable Change ◽  
Change Over Time ◽  
Method Of Calculation ◽  
Serial Addition ◽  
Number Of Individuals ◽  
Over Time

People with multiple sclerosis (MS) often undergo repeated assessments. Methods for determining whether an individual's change in test results over time is reliable require further study. A sample of individuals with MS (N = 52) was assessed at baseline and at 6-month follow-up using the Paced Auditory Serial Addition Test (PASAT), Simple Adjusting–Paced Serial Addition Test (A-PSAT), and Victoria Stroop test. Two methods for determining the reliability of an individual's change over time were examined. The Reliable Change Index (RCI) identified few individuals who declined reliably between baseline and 6-month follow-up. The standard deviation (SD) method of calculation resulted in reliable declines for a small number of individuals on most measures. Use of the SD method resulted in a larger number of individuals who improved reliably. Responsiveness of individuals to treatment effects can be lost in group analyses. The data presented here provide clinicians with an approach for determining whether an individual's change over time on commonly used neuropsychological tests reflects reliable improvement or decline.

Performance Validity and Symptom Validity in Neuropsychological Assessment

2012 ◽  
Vol 18 (4) ◽  
pp. 625-630 ◽  
Author(s):  
Glenn J. Larrabee
Keyword(s):  
Test Performance ◽  
False Positive ◽  
Neuropsychological Test ◽  
Diagnostic Procedures ◽  
Individual Case ◽  
Effect Sizes ◽  
Symptom Validity ◽  
Performance Validity ◽  
Symptom Report ◽  
The Individual

AbstractFailure to evaluate the validity of an examinee's neuropsychological test performance can alter prediction of external criteria in research investigations, and in the individual case, result in inaccurate conclusions about the degree of impairment resulting from neurological disease or injury. The terms performance validity referring to validity of test performance (PVT), and symptom validity referring to validity of symptom report (SVT), are suggested to replace less descriptive terms such as effort or response bias. Research is reviewed demonstrating strong diagnostic discrimination for PVTs and SVTs, with a particular emphasis on minimizing false positive errors, facilitated by identifying performance patterns or levels of performance that are atypical for bona fide neurologic disorder. It is further shown that false positive errors decrease, with a corresponding increase in the positive probability of malingering, when multiple independent indicators are required for diagnosis. The rigor of PVT and SVT research design is related to a high degree of reproducibility of results, and large effect sizes of d=1.0 or greater, exceeding effect sizes reported for several psychological and medical diagnostic procedures. (JINS, 2012, 18, 1–7)

scholarly journals Searching in the dark: Shining a light on some predictors of non-response to psychotherapy for borderline personality disorder

PLoS ONE ◽  
2021 ◽  
Vol 16 (7) ◽  
pp. e0255055
Author(s):  
Jane Woodbridge ◽  
Samantha Reis ◽  
Michelle L. Townsend ◽  
Lucy Hobby ◽  
Brin F. S. Grenyer
Keyword(s):  
Borderline Personality Disorder ◽  
Personality Disorder ◽  
Borderline Personality ◽  
Community Treatment ◽  
Reliable Change Index ◽  
Global Functioning ◽  
Reliable Change ◽  
Relationship Style ◽  
Preoccupied Attachment

Background Borderline Personality Disorder (BPD) is a prevalent and serious mental health condition. People can experience recovery or remission after receiving psychotherapy for BPD; however, it is estimated that about 45% of people in well conducted treatment trials do not respond adequately to current psychological treatments. Aim To further advance psychotherapies for BPD by identifying the factors that contribute to the problem of non-response. Method 184 consecutive participants with BPD in community treatment were naturalistically followed up over 12 months and measures of personality and social functioning were examined. Logistic regressions were used to determine which baseline factors were associated with the likelihood of being a non-responder after 12 months of psychotherapy. After 12 months, 48.4% of participants were classed as non-responders due to a lack of reduction in BPD symptoms according to the Reliable Change Index (RCI) method. Results At baseline intake, patients who endorsed an adult preoccupied attachment relationship style and increased anger were more likely to be a non-responder regarding BPD symptoms at 12 months. In addition, those with preoccupied attachment patterns in their adult relationships were more likely to be non-responders regarding general psychological distress at follow up. Higher baseline levels of paranoia and endorsement of a dismissive adult relationship style was associated with being a non-responder in regard to global functioning. Conclusions Consistent with previous research, almost half of the sample did not achieve reliable change at 12-month follow up. A relationship style characterised by preoccupied insecurity and high anger seemed to be particularly challenging in being able to benefit from psychotherapy. This style may have affected both relationships outside, but also inside therapy, complicating treatment engagement and alliance with the therapist. Early identification and modification of treatment based on challenges from these relationship styles may be one way to improve psychotherapy outcomes for BPD.

Predicting change with the RBANS in a community dwelling elderly sample

2004 ◽  
Vol 10 (6) ◽  
pp. 828-834 ◽  
Author(s):  
KEVIN DUFF ◽  
MIKE R. SCHOENBERG ◽  
DOYLE PATTON ◽  
JAMES MOLD ◽  
JAMES G. SCOTT ◽  
...  
Keyword(s):  
Older Adults ◽  
Test Performance ◽  
Community Dwelling ◽  
Healthy Control ◽  
Neuropsychological Assessments ◽  
Clinically Significant ◽  
The Individual ◽  
Community Dwelling Older Adults

Repeated neuropsychological assessments are common with older adults, and the determination of clinically significant change across time is an important issue. Regression-based prediction formulas have been utilized with other patient and healthy control samples to predict follow-up test performance based on initial performance and demographic variables. Comparisons between predicted and observed follow-up performances can assist clinicians in determining the significance of change in the individual patient. In the current study, multiple regression-based prediction equations for the 5 Indexes and Total Score of the RBANS were developed for a sample of 223 community dwelling older adults. These algorithms were then validated on a separate elderly sample (N = 222). Minimal differences were present between observed and predicted follow-up scores in the validation sample, suggesting that the prediction formulas are clinically useful for practitioners who assess older adults. A case example is presented that illustrates how the algorithms can be used clinically. (JINS, 2004, 10, 828–834.)

scholarly journals Dutch multicentre, prospective follow-up, cohort study comparing the neurological and neuropsychological sequelae of hospitalised non-ICU- and ICU-treated COVID-19 survivors: a study protocol

BMJ Open ◽  
2021 ◽  
Vol 11 (10) ◽  
pp. e054901
Author(s):  
Simona Klinkhammer ◽  
Janneke Horn ◽  
Johanna M A Visser-Meilij ◽  
Esmée Verwijk ◽  
Annelien Duits ◽  
...  
Keyword(s):  
Brain Damage ◽  
Test Performance ◽  
Large Scale ◽  
Neuropsychological Test ◽  
Well Being ◽  
Icu Patients ◽  
Neuropsychological Dysfunction ◽  
Long Term Impact ◽  
The Brain

IntroductionOwing to the novelty of COVID-19, there are still large knowledge gaps concerning its effect on the brain and the resulting impact on peoples’ lives. This large-scale prospective follow-up study investigates COVID-19-associated brain damage, neuropsychological dysfunction and long-term impact on the well-being of patients and their close ones. It is hypothesised that structural brain damage and cognitive dysfunction primarily occur in severely ill patients, as compared with moderately ill patients. Cognitive complaints, emotional distress and impact on well-being are hypothesised to be less dependent on illness severity.Methods and analysisFor this multicentre study, 200 patients with COVID-19 (100 intensive care unit (ICU) patients and 100 non-ICU patients) formerly hospitalised in one of the six recruiting hospitals during the first European infection wave (ie, March to June 2020) and their close ones will be recruited. At minimally 6 months posthospital discharge, patients will perform a set of neuropsychological tests and are subjected to a 3T MRI scan. Patients and close ones will fill out a set of questionnaires, also at minimally 6 months posthospital discharge and again another 6 months thereafter. Data related to COVID-19 hospitalisation will be extracted from the patients’ medical records. MRI abnormalities will ultimately be related to neuropsychological test performance and questionnaire outcomes.Ethics and disseminationEthics approval was granted by the medical research ethics committee of Maastricht University Medical Centre and Maastricht University (NL75102.068.20). The project is sponsored by The Brain Foundation Netherlands. Findings will be presented at national and international conferences, as well as published in peer-reviewed scientific journals.Trial registration numberNCT04745611.

Deleterious Impact of Expressive Suppression on Test Performance Persists at One-Year Follow-Up in Community-Dwelling Older Adults

2018 ◽  
Vol 25 (1) ◽  
pp. 29-38 ◽  
Author(s):  
Yana Suchy ◽  
Madison Amelia Niermeyer ◽  
Emilie Irene Franchow ◽  
Rosemary Ziemnik
Keyword(s):  
Older Adults ◽  
Lower Order ◽  
Test Performance ◽  
Experimental Manipulation ◽  
Practice Effect ◽  
Community Dwelling ◽  
Control Group ◽  
Expressive Suppression ◽  
Community Dwelling Older Adults

AbstractObjectives: Expressive suppression (ES) is an emotion-regulation strategy that is associated with poorer performance on subsequently administered tests of executive functioning (EF). It is not known, however, how far into the future ES interferes with EF. This study examined whether (a) ES negatively affects performance on EF tests repeated 1 year after the initial administration (presumably through interference with learning, leading to a reduced practice effect), and (b) whether such an effect, if seen, is unique to EF or whether it also affects lower-order cognitive processes needed for EF test performance. Methods: Sixty-six non-demented community-dwelling older adults were randomly assigned to either an ES group or control group. Executive and non-executive tests were administered before and immediately following the exposure to an emotionally evocative video, and then again at 1-year follow-up. Groups were compared at 1-year follow-up on tests of EF and lower-order processes, to examine whether the previously demonstrated impact of ES on EF is evident only immediately following the experimental manipulation (Franchow & Suchy, 2017), or also at 1-year follow-up. Results: The results showed that participants who engaged in ES continued to exhibit poorer performance on EF tests 1 year later. This effect was not present for performance on tests of lower-order processes. Conclusions: These results suggest that the use of ES before an EF task can interfere with the ability to benefit from exposure to that task, thereby negatively affecting future performance. (JINS, 2019, 25, 29–38)

Sign in / Sign up

Export Citation Format

Share Document