Inter-rater variability in scoring of Addenbrooke’s Cognitive Examination – third edition (ACE-III)

Background: Despite its wide use in dementia diagnosis on the basis of cut-off points, the inter-rater variability of the ACE-III has been poorly studied.Methods: 31 healthcare professionals from an older adults’ mental health team scored two ACE-III protocols based on mock patients in a computerised form. Scoring accuracy, as well as total and domain-specific scoring variability, were calculated; factors relevant to participants were obtained, including their level of experience and self-rated confidence administering the ACE-III.Results: There was considerable inter-rater variability (up to 18 points for one of the cases), and one case’s mean score was significantly higher (by four points) than the true score. The Fluency, Visuospatial and Attention domains had greater levels of variability than Language and Memory. Higher levels of scoring accuracy were not associated with either greater levels of experience not higher self-confidence in administering the ACE-III.Conclusions: The results suggest that the ACE-III is susceptible to scoring error and considerable inter-rater variability, which highlights the critical importance of initial, and continued, administration and scoring training.

REALISM OF FACE DEPICTION IN PORTRAITURE

This study presents the analysis of realism of portraiture in the context of physical anthropology. Standard descriptive traits such as the development of the upper eyelid fold, nasal profi le, etc., were scored on 120 portraits. To examine the accuracy of artistic rendition, these traits were assessed on 30 pairs of portraits of the same persons painted by different artists, and on 30 pairs of portraits versus photographs of the same persons. For each trait, the mean difference of scores was calculated. The mean differences are within the scoring error, indicating the artists’ high accuracy in rendering facial features. Next, four composite portraits were generated, two relating to 15th–16th century French aristocrats, and two to the 15th–17th century Dutch population, mainly that of Amsterdam. Composite portraits for every geographic region are virtually identical, suggesting that they represent a specifi c population rather than just a total of\individual data. Also, even though painters might be somewhat imprecise in depicting individual faces, these inaccuracies are leveled off in composite representations. In sum, portraiture is a very informative source of anthropometric information.

Computerized Neuropsychological Assessments

Computer-based measures to evaluate cognition have been used with growing frequency in recent years. These batteries are shown to be useful for identifying mild cognitive impairment (MCI) and dementia. There are few requirements to administer these tests. All that is typically needed is a computer, a response pad for patients to input their answers, and an examiner. In many cases, the examiner does not need to be a trained neuropsychologist. These computer-based assessments should yield a score report detailing the patient’s cognitive profile.An important advantage of computerized assessments over standard paper-and-pencil testing is that they can provide precise measurement at the millisecond level. This can be a more sensitive measure of cognitive impairment, especially in high-functioning older adults and in patients with milder levels of cognitive deficit. Computer tests also have a shorter assessment time. Many batteries take <1 hour to administer, whereas many standard neuropsychological batteries require >4 hours to complete. The presentation of items in some batteries can be adapted to patients’ performance levels to avoid floor effects (the test restricts how low a patients’ scores can be) and ceiling effects (the test restricts how high scores can be). Computer tests have increased standardization; they are administered the same way every time. Scoring is automatic, meaning the results are available immediately and human scoring error is reduced. Examiner effects are reduced, which is an important advantage because clinicians may differ in how they administer standard tests, which may impact patients’ responses. In addition, the batteries are easily transported, and multiple tasks can be made available on a single computer.

A Scoring Error in the Mini-Mental State Test

A scoring error is identified in the Mini-Mental and the Modified Mini-Mental State test. The major effect of this error is to produce a gap in the total score distribution of the test, resulting in a potential spurious underestimation of the score. Two possible remedies are suggested to resolve this problem.