The predictive performance of criminal risk assessment tools used at sentencing
Although risk assessment tools have been widely used to inform sentencing decisions, there is uncertainty about the extent and quality of evidence of their predictive performance when validated in new samples. Following PRISMA guidelines, we conducted a systematic review of validation studies of ten commonly used risk assessment tools for sentencing. We identified 33 studies with 596,618 participants, among which were 24 independent validation studies with 176,664 individuals. Overall, the predictive performances of the included risk assessment tools was mixed, and ranged from poor to moderate. Tool performance was typically overestimated in studies with smaller sample sizes or studies in which tool developers were co-authors. Most studies only reported Area Under the Curve(AUC), which ranged from 0.57 to 0.75 in independent studies with more than 500participants. The majority did not report key performance measures, such as calibration and rates of false positives and negatives. In addition, most validation studies had a high risk of bias, partly due to inappropriate analytical approach used. We conclude that the key research priority is for future studies to address the key methodological shortcomings identified in this review, and policy makers should enable this research. More sufficiently powered independent validation studies are necessary.