On the Quantification of Crowd Wisdom
Crowd wisdom is a fascinating metaphor in the realm of collective intelligence. However, even for the simple case of estimation tasks of one continuous value, the quantification of the phenomenon lacks some conceptual clarity. Two interrelated questions of quantification are at stake. First, how can we best aggregate the collective decision from a sample of estimates, with the mean or the median? Arguments are not only statistical but also related to the question if democratic decision-making can have an epistemic quality. A practical result of this study is that we should usually aggregate democratic decisions by the median, but have a backup with the mean when the decision space has two natural bounds and societies polarize. The second question is, how we can quantify the degree of crowd wisdom in a sample and how it can be distinguished from the individual wisdom of its members? Two measures will be presented and discussed. One can also be used to quantify optimal crowd sizes. Even purely statistical, it turns out that smaller crowds are more advisable when intermediate systematic errors in estimating crowds are frequent. In such cases, larger crowds are more likely to be outperformed by a single estimator.