In: Assessment and Communication of Uncertainty in Intelligence to Support Decision Making: Final Report of Research Task Group SAS-114. Brussels, Belgium: NATO Science and Technology Organization, 2019
AbstractOrganizations in several domains including national security intelligence communicate judgments under uncertainty using verbal probabilities (e.g., likely) instead of numeric probabilities (e.g., 75% chance), despite research indicating that the former have variable meanings across individuals. In the intelligence domain, uncertainty is also communicated using terms such as low, moderate, or high to describe the analyst's confidence level. However, little research has examined how intelligence professionals interpret these terms and whether they prefer them to numeric uncertainty quantifiers. In two experiments (N = 481 and 624, respectively), uncertainty communication preferences of expert (n = 41 intelligence analysts in Experiment 1) and nonexpert intelligence consumers were elicited. We examined which format participants judged to be more informative and simpler to process. We further tested whether participants treated verbal probability and confidence terms as independent constructs and whether participants provided coherent numeric probability translations of verbal probabilities. Results showed that although most nonexperts favored the numeric format, experts were about equally split, and most participants in both samples regarded the numeric format as more informative. Experts and nonexperts consistently conflated probability and confidence. For instance, confidence intervals inferred from verbal confidence terms had a greater effect on the location of the estimate than the width of the estimate, contrary to normative expectation. Approximately one‐fourth of experts and over one‐half of nonexperts provided incoherent numeric probability translations for the terms likely and unlikely when the elicitation of best estimates and lower and upper bounds were briefly spaced by intervening tasks.
The world is becoming a less conflict-ridden place, and it has been for decades. Wars and violent conflicts are on the decline, according to expert consensus. Abundant evidence suggests that the world is not deteriorating, from Steven Pinker's conclusion of a long-term general decline in global violence to Azar Gat's analysis pointing to a decline in war since the mid-1990s to the 2013 Human Security Report proclaiming a substantial decline in conflicts since the early 1990s. However, a systematic analysis of world crisis data from a global leader in crisis tracking showed a different result. Here, Tikuisis and Mandel examine the International Crisis Group's monthly CrisisWatch statistics for the crises it has tracked from its inception in Sep 2003 to Mar 2014. Crisis Group's data paint a startlingly different picture: a world that has deteriorated substantially over the past decade. Adapted from the source document.