Communicating Uncertainty in National Security Intelligence: Expert and Non-expert Interpretations of and Preferences for Verbal and Numeric Formats

Organizations in several domains including national security intelligence communicate judgments under uncertainty using verbal probabilities (e.g., likely) instead of numeric probabilities (e.g., 75% chance), despite research indicating that the former have variable meanings across individuals. In the intelligence domain, uncertainty is also communicated using terms such as low, moderate, or high to describe the analyst’s confidence level. However, little research has examined how intelligence professionals interpret these terms and whether they prefer them to numeric uncertainty quantifiers. In two experiments (N = 481 and 624, respectively), uncertainty communication preferences of expert (n = 41 intelligence analysts inExperiment 1) and non-expert intelligence consumers were elicited. We examined which format participants judged to be more informative and simpler to process. We further tested whether participants treated probability and confidence as independent constructs and whether participants provided coherent numeric probability translations of verbal probabilities. Results showed that whereas most non-experts favored the numeric format, experts were about equally split, and most participants in both samples regarded the numeric format as more informative.Experts and non-experts consistently conflated probability and confidence. For instance, confidence intervals inferred from verbal confidence terms had a greater effect on the location of the estimate than the width of the estimate, contrary to normative expectation. Approximately ¼ of experts and over ½ of non-experts provided incoherent numeric probability translations of best estimates and lower and upper bounds when elicitations were spaced by intervening tasks.