Inter-rater variability in scoring of Addenbrooke’s Cognitive Examination – third edition (ACE-III)

Background: Despite its wide use in dementia diagnosis on the basis of cut-off points, the inter-rater variability of the ACE-III has been poorly studied.Methods: 31 healthcare professionals from an older adults’ mental health team scored two ACE-III protocols based on mock patients in a computerised form. Scoring accuracy, as well as total and domain-specific scoring variability, were calculated; factors relevant to participants were obtained, including their level of experience and self-rated confidence administering the ACE-III.Results: There was considerable inter-rater variability (up to 18 points for one of the cases), and one case’s mean score was significantly higher (by four points) than the true score. The Fluency, Visuospatial and Attention domains had greater levels of variability than Language and Memory. Higher levels of scoring accuracy were not associated with either greater levels of experience not higher self-confidence in administering the ACE-III.Conclusions: The results suggest that the ACE-III is susceptible to scoring error and considerable inter-rater variability, which highlights the critical importance of initial, and continued, administration and scoring training.

Receiver operating characteristic plot and area under the curve with binary classifiers: pragmatic analysis of cognitive screening instruments

Aim: To examine whether receiver operating characteristic plots and area under the curve (AUC) values may be potentially misleading when assessing cognitive screening instruments as binary predictors rather than as categorical or continuous scales. Materials & methods: AUC was calculated using different methods (rank-sum, diagnostic odds ratio) using data from test accuracy studies of two binary classifiers of cognitive status (applause sign, attended with sign), a screener producing categorical data (Codex), and a continuous scale screening test (Mini-Addenbrooke’s Cognitive Examination). Results: For all screeners, AUC calculated using diagnostic odds ratio method was greater than using rank-sum method. When Codex and Mini-Addenbrooke’s Cognitive Examination were analyzed as binary (single fixed threshold) tests, AUC using rank-sum method was lower than when screeners were analyzed as categorical or continuous scales, respectively. Conclusion: If cognitive screeners producing categorical or continuous measures are dichotomized, calculated AUC may be an underestimate, thus affecting screening test accuracy.

Reliability of Addenbrooke's Cognitive Examination III in differentiating between dementia, mild cognitive impairment and older adults who have not reported cognitive problems

Diagnosing dementia can be challenging for clinicians, given the array of factors that contribute to changes in cognitive function. The Addenbrooke’s Cognitive Examination III (ACE-III) is commonly used in dementia assessments, covering the domains of attention, memory, fluency, visuospatial and language. This study aims to (1) assess the reliability of ACE-III to differentiate between dementia, mild cognitive impairment (MCI) and controls and (2) establish whether the ACE-III is useful for diagnosing dementia subtypes. Client records from the Northern Health and Social Care Trust (NHSCT) Memory Service (n = 2,331, 2013–2019) were used in the analysis including people diagnosed with Alzheimer’s disease (n = 637), vascular dementia (n = 252), mixed dementia (n = 490), MCI (n = 920) and controls (n = 32). There were significant differences in total ACE-III and subdomain scores between people with dementia, MCI and controls (p < 0.05 for all), with little overlap between distribution of total ACE-III scores (< 39%) between groups. The distribution of total ACE-III and subdomain scores across all dementias were similar. There were significant differences in scores for attention, memory and fluency between Alzheimer’s disease and mixed dementia, and for visuospatial and language between Alzheimer’s disease–vascular dementia (p < 0.05 for all). However, despite the significant differences across these subdomains, there was a high degree of overlap between these scores (> 73%) and thus the differences are not clinically relevant. The results suggest that ACE-III is a useful tool for discriminating between dementia, MCI and controls, but it is not reliable for discriminating between dementia subtypes. Nonetheless, the ACE-III is still a reliable tool for clinicians that can assist in making a dementia diagnosis in combination with other factors at assessment.

Brazilian Version of Addenbrooke’s Cognitive Examination—Revised in the Differential Diagnosis of Alzheimer’S Disease and Behavioral Variant Frontotemporal Dementia

Introduction Alzheimer’s disease (AD) and behavioral variant frontotemporal dementia (bvFTD) are frequent causes of dementia and, therefore, instruments for differential diagnosis between these two conditions are of great relevance. Objective To investigate the diagnostic accuracy of Addenbrooke’s Cognitive Examination-Revised (ACE-R) for differentiating AD from bvFTD in a Brazilian sample. Methods The ACE-R was administered to 102 patients who had been diagnosed with mild dementia due to probable AD, 37 with mild bvFTD and 161 cognitively healthy controls, matched according to age and education. Additionally, all subjects were assessed using the Mattis Dementia Rating Scale and the Neuropsychiatric Inventory. The performance of patients and controls was compared by using univariate analysis, and ROC curves were calculated to investigate the accuracy of ACE-R for differentiating AD from bvFTD and for differentiating AD and bvFTD from controls. The verbal fluency plus language to orientation plus name and address delayed recall memory (VLOM) ratio was also calculated. Results The optimum cutoff scores for ACE-R were &lt;80 for AD, &lt;79 for bvFTD, and &lt;80 for dementia (AD + bvFTD), with area under the receiver operating characteristic curves (ROC) (AUC) &gt;0.85. For the differential diagnosis between AD and bvFTD, a VLOM ratio of 3.05 showed an AUC of 0.816 (Cohen’s d = 1.151; p &lt; .001), with 86.5% sensitivity, 71.4% specificity, 72.7% positive predictive value, and 85.7% negative predictive value. Conclusions The Brazilian ACE-R achieved a good diagnostic accuracy for differentiating AD from bvFTD patients and for differentiating AD and bvFTD from the controls in the present sample.

Clinical progression of progressive supranuclear palsy: impact of trials bias and phenotype variants

Progressive supranuclear palsy causes diverse clinical presentations, including classical Richardson’s syndrome and several variant phenotypes. Clinical trials of disease-modifying therapies have recently been completed, with more planned for the next two years. However, many people with progressive supranuclear palsy do not meet eligibility criteria for these clinical trials. Understanding clinical progression with different phenotypes would improve trial design and enhance the accuracy of risk-benefit and cost-benefit assessments of new treatments for progressive supranuclear palsy. We set out to determine rates of motor and cognitive progression of possible, probable and definite progressive supranuclear palsy, with different phenotypes, from a representative cohort in a regional United Kingdom healthcare service. Longitudinal clinical data from people with Richardson’s syndrome and variant phenotypes were analysed using linear mixed-modelling, using both the full and modified versions of the Progressive Supranuclear Palsy Rating Scale, Mini-Mental State Examination, and the revised Addenbrooke’s Cognitive Examination. Subgroup analyses considered patients meeting recent phase II trial entry criteria and patients with neuropathological confirmation. 227 patients (male = 59%, mean age [±Standard Deviation], 71.8[±7.0] years) were followed for a mean 21.6[±15.6] months. One hundred and seventy-four (77%) had Richardson’s syndrome at the outset, twenty-five had cortical variant presentations (13%, frontal, corticobasal, speech and language variants) and twenty-eight had subcortical variant presentations (14%, parkinsonism, postural instability and gait freezing variants). Across all participants, annual progression in Richardson’s syndrome was faster than variant phenotypes on the Mini-Mental State Examination (-1.8 vs -0.9/year, p = 0.005) and revised Addenbrooke’s Cognitive Examination (-5.3 vs -3.0/year, p = 0.01) but not the Progressive Supranuclear Palsy Rating Scale (9.0 vs 7.1/year, p = 0.2) nor the modified Progressive Supranuclear Palsy Rating Scale (2.7 vs 2.3/year, p = 0.4). However, for those with more than one years’ follow-up, a significant difference was observed between Richardson’s syndrome and variant phenotypes in Progressive Supranuclear Palsy Rating Scale (8.7 vs 6.3/year, p = 0.04). Survival was longer in variant phenotypes than Richardson’s syndrome (7.3[±3.9] vs 5.6[±2.0] years, p = 0.02). Pathologically confirmed cases (n = 49) supported these findings. Patients meeting basic trial-eligibility criteria (n = 129) progressed faster on the Progressive Supranuclear Palsy Rating Scale than trial-not-eligible patients (10.1 vs 6.1/year, p = 0.001). In conclusion, phenotypes other than Richardson’s syndrome show slower progression and longer survival. Trial criteria do not select representative progressive supranuclear palsy cases. This has implications for trial design, and application of trial results to clinically more diverse patient populations.