The Wisdom of the Coherent: Improving Correspondence with Coherence-Weighted Aggregation

Previous research shows that variation in coherence (i.e., degrees of respect for axioms of probability calculus), when used as a basis for performance-weighted aggregation, can improve the accuracy of probability judgments. However, many aspects of coherence-weighted aggregation remain a mystery, including both prescriptive issues (e.g. how best to use coherence measures) and theoretical issues (e.g. why coherence-weighted aggregation is effective). Using data from previous experiments employing either general-knowledge or statistical information-integration tasks, we addressed many of these issues. Of prescriptive relevance, we examined the effectiveness of coherence-weighted aggregation as a function of judgment elicitation method, group size, weighting function, and the aggressivity of the function’s tuning parameter. Of descriptive relevance, we propose that coherence-weighted aggregation can improve accuracy via two distinct, task-dependent routes: a deterministic route in which the bases for scoring accuracy depend on conformity to coherence principles (e.g., Bayesian information integration) and a diagnostic route in which coherence serves as a cue to correct knowledge. The findings provide support for the efficacy of both routes, but they also highlight why coherence weighting, especially the most aggressive forms, sometimes impose costs to accuracy. We conclude by sketching a decision-theoretic approach to how the wisdom of the coherent within the wisdom of the crowd can be sensibly leveraged.