Correlation or Limits of Agreement? Applying the Bland-Altman Approach to the Comparison of Cognitive Screening Instruments

2016 ◽  
Vol 42 (5-6) ◽  
pp. 247-254 ◽  
Author(s):  
A.J. Larner
Keyword(s):  
Diagnostic Accuracy ◽  
Test Scores ◽  
Correlation Coefficients ◽  
Screening Instruments ◽  
Cognitive Screening ◽  
Limits Of Agreement ◽  
Cognitive Domains ◽  
Highly Correlated ◽  
State Examination ◽  
Addenbrooke’S Cognitive Examination

Background/Aims: Calculation of correlation coefficients is often undertaken as a way of comparing different cognitive screening instruments (CSIs). However, test scores may correlate but not agree, and high correlation may mask lack of agreement between scores. The aim of this study was to use the methodology of Bland and Altman to calculate limits of agreement between the scores of selected CSIs and contrast the findings with Pearson's product moment correlation coefficients between the test scores of the same instruments. Methods: Datasets from three pragmatic diagnostic accuracy studies which examined the Mini-Mental State Examination (MMSE) vs. the Montreal Cognitive Assessment (MoCA), the MMSE vs. the Mini-Addenbrooke's Cognitive Examination (M-ACE), and the M-ACE vs. the MoCA were analysed to calculate correlation coefficients and limits of agreement between test scores. Results: Although test scores were highly correlated (all >0.8), calculated limits of agreement were broad (all >10 points), and in one case, MMSE vs. M-ACE, was >15 points. Conclusion: Correlation is not agreement. Highly correlated test scores may conceal broad limits of agreement, consistent with the different emphases of different tests with respect to the cognitive domains examined. Routine incorporation of limits of agreement into diagnostic accuracy studies which compare different tests merits consideration, to enable clinicians to judge whether or not their agreement is close.

Optimising the Cutoffs of Cognitive Screening Instruments in Pragmatic Diagnostic Accuracy Studies: Maximising Accuracy or the Youden Index?

2015 ◽  
Vol 39 (3-4) ◽  
pp. 167-175 ◽  
Author(s):  
Andrew J. Larner
Keyword(s):  
Diagnostic Accuracy ◽  
Alternative Methods ◽  
Youden Index ◽  
Test Accuracy ◽  
Screening Instruments ◽  
Cognitive Screening ◽  
Optimal Method ◽  
Index Method ◽  
State Examination ◽  
Addenbrooke’S Cognitive Examination

Background/Aims: The optimal method of establishing test cutoffs or cutpoints for cognitive screening instruments (CSIs) is uncertain. Of the available methods, two base cutoffs on either the maximal test accuracy or the maximal Youden index. The aim of this study was to compare the effects of using these alternative methods of establishing cutoffs. Methods: Datasets from three pragmatic diagnostic accuracy studies which examined the Mini-Mental State Examination (MMSE), the Addenbrooke's Cognitive Examination-Revised (ACE-R), the Montreal Cognitive Assessment (MoCA), and the Test Your Memory (TYM) test were analysed to calculate test sensitivity and specificity using cutoffs based on either maximal test accuracy or the maximal Youden index. Results: For ACE-R, MoCA, and TYM, optimal cutoffs for dementia diagnosis differed from those in index studies when defined using either the maximal accuracy or the maximal Youden index method. Optimal cutoffs were higher for MMSE, MoCA, and TYM when using the maximal Youden index method and consequently more sensitive. Conclusion: Revision of the cutoffs for CSIs established in index studies may be required to optimise performance in pragmatic diagnostic test accuracy studies which more closely resemble clinical practice.

scholarly journals Comparing the Diagnostic Accuracy of Two Cognitive Screening Instruments in Different Dementia Subtypes and Clinical Depression

Diagnostics ◽  
2019 ◽  
Vol 9 (3) ◽  
pp. 93 ◽  
Author(s):  
Rónán O’Caoimh ◽  
D. William Molloy
Keyword(s):  
Cognitive Impairment ◽  
Mild Cognitive Impairment ◽  
Diagnostic Accuracy ◽  
Lewy Body ◽  
Mini Mental State Examination ◽  
Lewy Body Dementia ◽  
Screening Instruments ◽  
Cognitive Screening ◽  
State Examination ◽  
Normal Controls

Short but accurate cognitive screening instruments are required in busy clinical practice. Although widely-used, the diagnostic accuracy of the standardised Mini-Mental State Examination (SMMSE) in different dementia subtypes remains poorly characterised. We compared the SMMSE to the Quick Mild Cognitive Impairment (Qmci) screen in patients (n = 3020) pooled from three memory clinic databases in Canada including those with mild cognitive impairment (MCI) and Alzheimer’s, vascular, mixed, frontotemporal, Lewy Body and Parkinson’s dementia, with and without co-morbid depression. Caregivers (n = 875) without cognitive symptoms were included as normal controls. The median age of patients was 77 (Interquartile = ±9) years. Both instruments accurately differentiated cognitive impairment (MCI or dementia) from controls. The SMMSE most accurately differentiated Alzheimer’s (AUC 0.94) and Lewy Body dementia (AUC 0.94) and least accurately identified MCI (AUC 0.73), vascular (AUC 0.74), and Parkinson’s dementia (AUC 0.81). The Qmci had statistically similar or greater accuracy in distinguishing all dementia subtypes but particularly MCI (AUC 0.85). Co-morbid depression affected accuracy in those with MCI. The SMMSE and Qmci have good-excellent accuracy in established dementia. The SMMSE is less suitable in MCI, vascular and Parkinson’s dementia, where alternatives including the Qmci screen may be used. The influence of co-morbid depression on scores merits further investigation.

scholarly journals Screening for Alzheimer's disease in low-educated or illiterate older adults in Brazil: a systematic review

2019 ◽  
Vol 77 (4) ◽  
pp. 279-288 ◽  
Author(s):  
Luciane de Fátima Viola Ortega ◽  
Ivan Aprahamian ◽  
Marcus Kiiti Borges ◽  
João de Castilho Cação ◽  
Mônica Sanches Yassuda
Keyword(s):  
Alzheimer’S Disease ◽  
Alzheimer's Disease ◽  
Diagnostic Accuracy ◽  
Screening Tools ◽  
Screening Tests ◽  
Screening Instruments ◽  
Cognitive Screening ◽  
Inclusion Criteria ◽  
Low Education ◽  
Low Educated

ABSTRACT Cognitive screening instruments are influenced by education and/or culture. In Brazil, as illiteracy and low education rates are high, it is necessary to identify the screening tools with the highest diagnostic accuracy for Alzheimer's disease (AD). Objective: To identify the cognitive screening instruments applied in the Brazilian population with greater accuracy, to detect AD in individuals with a low educational level or who are illiterate. Methods: Systematic search in SciELO, PubMed and LILACS databases of studies that used cognitive screening tests to detect AD in older Brazilian adults with low or no education. Results: We found 328 articles and nine met the inclusion criteria. The identified instruments showed adequate or high diagnostic accuracy. Conclusion: For valid cognitive screening it is important to consider sociocultural and educational factors in the interpretation of results. The construction of specific instruments for the low educated or illiterate elderly should better reflect the difficulties of the Brazilian elderly in different regions of the country.

Validation of the Sinhala Version of the Addenbrooke’s Cognitive Examination-Revised for the Detection of Dementia in Sri Lanka: Comparison with the Mini-Mental Status Examination and the Montreal Cognitive Assessment

2019 ◽  
Vol 47 (4-6) ◽  
pp. 198-208 ◽  
Author(s):  
Vindika Suriyakumara ◽  
Srinivasan  Srikanth ◽  
Ruwani  Wijeyekoon ◽  
Harsha  Gunasekara ◽  
Chanaka  Muthukuda ◽  
...  
Keyword(s):  
Sri Lanka ◽  
Diagnostic Accuracy ◽  
Cognitive Assessment ◽  
Intraclass Correlation ◽  
Screening Tools ◽  
Montreal Cognitive Assessment ◽  
Cutoff Score ◽  
Cognitive Screening ◽  
Dementia Prevalence ◽  
Addenbrooke’S Cognitive Examination

Background: Sri Lanka is a rapidly aging country, where dementia prevalence will increase significantly in the future. Thus, inexpensive and sensitive cognitive screening tools are crucial. Objectives: To assess the reliability, validity, and diagnostic accuracy of the Sinhalese version of the Addenbrooke’s Cognitive Examination-Revised (ACE-R s). Method: The ACE-R was translated into Sinhala with cultural and linguistic adaptations and administered, together with the Sinhala version of the Montreal Cognitive Assessment (MoCA), to 99 patients with dementia and 93 gender-matched controls. Results: The ACE-R s cutoff score for dementia was 80 (sensitivity 91.9%, specificity 76.3%). The areas under the curve for the ACE-R s, Mini-Mental State Examination (MMSE) and MoCA were 0.90, 0.86, and 0.86, respectively. The ­ACE-R s had good interrater reliability (intraclass correlation = 0.94), test-retest reliability (intraclass correlation = 0.99), and internal consistency (Cronbach’s α = 0.8442). Conclusions: The ACE-R s is sensitive, specific and reliable to detect dementia in persons aged ≥50 years in a Sinhala-speaking population and its diagnostic accuracy is superior to previously validated tools (MMSE and MoCA).

scholarly journals Cognitive screening instruments for dementia: comparing metrics of test limitation

2021 ◽  
Vol 15 (4) ◽  
pp. 458-463
Author(s):  
Andrew J. Larner
Keyword(s):  
Cognitive Impairment ◽  
High Sensitivity ◽  
Misclassification Rate ◽  
Test Accuracy ◽  
Screening Instruments ◽  
Cognitive Screening ◽  
Net Benefit ◽  
Likelihood Ratios ◽  
Sensitivity Specificity ◽  
State Examination

ABSTRACT Cognitive screening instruments (CSIs) for dementia and mild cognitive impairment are usually characterized in terms of measures of discrimination such as sensitivity, specificity, and likelihood ratios, but these CSIs also have limitations. Objective: The aim of this study was to calculate various measures of test limitation for commonly used CSIs, namely, misclassification rate (MR), net harm/net benefit ratio (H/B), and the likelihood to be diagnosed or misdiagnosed (LDM). Methods: Data from several previously reported pragmatic test accuracy studies of CSIs (Mini-Mental State Examination, the Montreal Cognitive Assessment, Mini-Addenbrooke’s Cognitive Examination, Six-item Cognitive Impairment Test, informant Ascertain Dementia 8, Test Your Memory test, and Free-Cog) undertaken in a single clinic were reanalyzed to calculate and compare MR, H/B, and the LDM for each test. Results: Some CSIs with very high sensitivity but low specificity for dementia fared poorly on measures of limitation, with high MRs, low H/B, and low LDM; some had likelihoods favoring misdiagnosis over diagnosis. Tests with a better balance of sensitivity and specificity fared better on measures of limitation. Conclusions: When deciding which CSI to administer, measures of test limitation as well as measures of test discrimination should be considered. Identification of CSIs with high MR, low H/B, and low LDM, may have implications for their use in clinical practice.

scholarly journals Cognitive screening test in primary care

2018 ◽  
Vol 52 ◽  
pp. 88 ◽  
Author(s):  
Juliana Emy Yokomizo ◽  
Katrin Seeher ◽  
Glaucia Martins de Oliveira ◽  
Laís dos Santos Vinholi e Silva Silva ◽  
Laura Saran ◽  
...  
Keyword(s):  
Primary Care ◽  
Diagnostic Accuracy ◽  
Educational Level ◽  
High Sensitivity ◽  
Cognitive Screening ◽  
Operating Characteristics ◽  
Predictive Values ◽  
Brazilian Version ◽  
Logistic Regressions ◽  
State Examination

OBJECTIVE: To establish the diagnostic accuracy of the Brazilian version of the General Practitioner Assessment of Cognition (GPCOG-Br) compared to the Mini-Mental State Examination (MMSE) in individuals with low educational level. METHODS: Ninety-three patients (≥ 60 years old) from Brazilian primary care units provided sociodemographic, cognitive, and functional data. Receiver operating characteristics, areas under the curve (AUC) and logistic regressions were conducted. RESULTS: Sixty-eight patients with 0–4 years of education. Cases (n = 44) were older (p = 0.006) and performed worse than controls (n = 49) on all cognitive or functional measures (p < 0.001). The GPCOG-Br demonstrated similar diagnostic accuracy to the MMSE (AUC = 0.90 and 0.91, respectively) and similar positive and negative predictive values (PPV/NPV, respectively: 0.79/0.86 for GPCOG-Br and 0.79/0.81 for MMSE). Adjusted cut-points displayed high sensitivity (all 86%) and satisfactory specificity (65%–80%). Lower educational level predicted lower cognitive performance. CONCLUSIONS: The GPCOG-Br is clinically well-suited for use in primary care.

Comparative Diagnostic Accuracy of the ACE-III, MIS, MMSE, MoCA, and RUDAS for Screening of Alzheimer Disease

2017 ◽  
Vol 43 (5-6) ◽  
pp. 237-246 ◽  
Author(s):  
Jordi A. Matías-Guiu ◽  
María Valles-Salgado ◽  
Teresa Rognoni ◽  
Frank Hamre-Gil ◽  
Teresa Moreno-Ramos ◽  
...  
Keyword(s):  
Alzheimer Disease ◽  
Diagnostic Accuracy ◽  
Area Under The Curve ◽  
Cross Sectional Study ◽  
Screening Tests ◽  
Cross Sectional ◽  
Diagnostic Capacity ◽  
State Examination ◽  
Diagnostic Properties ◽  
Addenbrooke’S Cognitive Examination

Background: Our aim was to evaluate and compare the diagnostic properties of 5 screening tests for the diagnosis of mild Alzheimer disease (AD). Methods: We conducted a prospective and cross-sectional study of 92 patients with mild AD and of 68 healthy controls from our Department of Neurology. The diagnostic properties of the following tests were compared: Mini-Mental State Examination (MMSE), Addenbrooke's Cognitive Examination III (ACE-III), Memory Impairment Screen (MIS), Montreal Cognitive Assessment (MoCA), and Rowland Universal Dementia Assessment Scale (RUDAS). Results: All tests yielded high diagnostic accuracy, with the ACE-III achieving the best diagnostic properties. The area under the curve was 0.897 for the ACE-III, 0.889 for the RUDAS, 0.874 for the MMSE, 0.866 for the MIS, and 0.856 for the MoCA. The Mini-ACE score from the ACE-III showed the highest diagnostic capacity (area under the curve 0.939). Memory scores of the ACE-III and of the RUDAS showed a better diagnostic accuracy than those of the MMSE and of the MoCA. All tests, especially the ACE-III, conveyed a higher diagnostic accuracy in patients with full primary education than in the less educated group. Implementing normative data improved the diagnostic accuracy of the ACE-III but not that of the other tests. Conclusions: The ACE-III achieved the highest diagnostic accuracy. This better discrimination was more evident in the more educated group.

A meta-analysis of the accuracy of the Addenbrooke's Cognitive Examination (ACE) and the Addenbrooke's Cognitive Examination-Revised (ACE-R) in the detection of dementia

2013 ◽  
Vol 26 (4) ◽  
pp. 555-563 ◽  
Author(s):  
Andrew J. Larner ◽  
Alex J. Mitchell
Keyword(s):  
Primary Care ◽  
Diagnostic Accuracy ◽  
Mini Mental State Examination ◽  
Meta Analysis ◽  
Screening Tools ◽  
High Prevalence ◽  
Memory Clinics ◽  
State Examination ◽  
Addenbrooke’S Cognitive Examination ◽  
Prevalence Of Dementia

ABSTRACTBackground:The Addenbrooke's Cognitive Examination (ACE) and its Revised version (ACE-R) are relatively new screening tools for cognitive impairment that may improve upon the well-known Mini-Mental State Examination (MMSE) and other brief batteries. We systematically reviewed diagnostic accuracy studies of ACE and ACE-R.Methods:Published studies comparing ACE, ACE-R and MMSE were comprehensively sought and critically appraised. A meta-analysis of suitable studies was conducted.Results:Of 61 possible publications identified, meta-analysis of qualifying studies encompassed 5 for ACE (1,090 participants) and 5 for ACE-R (1156 participants); of these, 9 made direct comparisons with the MMSE. Sensitivity and specificity of the ACE were 96.9% (95% CI = 92.7% to 99.4%) and 77.4% (95% CI = 58.3% to 91.8%); and for the ACE-R were 95.7% (95% CI = 92.2% to 98.2%) and 87.5% (95% CI = 63.8% to 99.4%). In a modest prevalence setting, such as primary care or general hospital settings where the prevalence of dementia may be approximately 25%, overall accuracy of the ACE (0.823) was inferior to ACE-R (0.895) and MMSE (0.882). In high prevalence settings such as memory clinics where the prevalence of dementia may be 50% or higher, overall accuracy again favored ACE-R (0.916) over ACE (0.872) and MMSE (0.895).Conclusions:The ACE-R has somewhat superior diagnostic accuracy to the MMSE while the ACE appears to have inferior accuracy. The ACE-R is recommended in both modest and high prevalence settings. Accuracy of newer versions of the ACE remain to be determined.

Sign in / Sign up

Export Citation Format

Share Document