Effective uncertainty visualization for aftershock forecast maps

Earthquake models can produce aftershock forecasts, which have recently been released to lay audiences following large earthquakes. While visualization literature suggests that displaying forecast uncertainty can improve how forecast maps are used, research on uncertainty visualization is missing from earthquake science. We designed a pre-registered online experiment to test the effectiveness of three visualization techniques for displaying aftershock forecast maps and their uncertainty. These maps showed the forecasted number of aftershocks at each location for a week following a hypothetical mainshock, along with the uncertainty around each location’s forecast. Three different uncertainty visualizations were produced: (1) forecast and uncertainty maps adjacent to one another; (2) the forecast map depicted in a color scheme, with the uncertainty shown by the transparency of the color; and (3) two maps that showed the lower and upper bounds of the forecast distribution at each location. Unlike previous experiments, we compared the three uncertainty visualizations using tasks that are systematically designed to address broadly applicable and user-generated communication goals. We compared task responses between participants using uncertainty visualizations and using the forecast map shown without its uncertainty (the current practice). Participants completed two map-reading tasks that targeted several dimensions of the readability of uncertainty visualizations. Participants then performed a comparative judgment task, which demonstrated whether a visualization was successful in reaching two key communication goals: indicating where many aftershocks and no aftershocks are likely (sure bets) and where the forecast is low but the uncertainty is high enough to imply potential risk (surprises). All visualizations performed equally well in the goal of communicating sure bet situations. But the visualization with lower and upper bounds was substantially better than the other designs at communicating surprises. These results have implications for the communication of forecast uncertainty both within and beyond earthquake science.

Examining Effort in 1D Uncertainty Communication Using Individual Differences in Working Memory and NASA-TLX

As uncertainty visualizations for general audiences become increasingly common, designers must understand the full impact of uncertainty communication techniques on viewers' decision processes. Prior work demonstrates mixed performance outcomes with respect to how individuals make decisions using various visual and textual depictions of uncertainty. Part of the inconsistency across findings may be due to an over-reliance on task accuracy, which cannot, on its own, provide a comprehensive understanding of how uncertainty visualization techniques support reasoning processes. In this work, we advance the debate surrounding the efficacy of modern 1D uncertainty visualizations by conducting converging quantitative and qualitative analyses of both the effort and strategies used by individuals when provided with quantile dotplots, density plots, interval plots, mean plots, and textual descriptions of uncertainty. We utilize two approaches for examining effort across uncertainty communication techniques: a measure of individual differences in working-memory capacity known as an operation span (OSPAN) task and self-reports of perceived workload via the NASA-TLX. The results reveal that both visualization methods and working-memory capacity impact participants' decisions. Specifically, quantile dotplots and density plots (i.e., distributional annotations) result in more accurate judgments than interval plots, textual descriptions of uncertainty, and mean plots (i.e., summary annotations). Additionally, participants' open-ended responses suggest that individuals viewing distributional annotations are more likely to employ a strategy that explicitly incorporates uncertainty into their judgments than those viewing summary annotations. When comparing quantile dotplots to density plots, this work finds that both methods are equally effective for low-working-memory individuals. However, for individuals with high-working-memory capacity, quantile dotplots evoke more accurate responses with less perceived effort. Given these results, we advocate for the inclusion of converging behavioral and subjective workload metrics in addition to accuracy performance to further disambiguate meaningful differences among visualization techniques.

Multiple Hazard Uncertainty Visualization Challenges and Paths Forward

Making decisions with uncertainty is challenging for the general public, policymakers, and even highly trained scientists. Nevertheless, when faced with the need to respond to a potential hazard, people must make high-risk decisions with uncertainty. In some cases, people have to consider multiple hazards with various types of uncertainties. Multiple hazards can be interconnected by location, time, and/or environmental systems, and the hazards may interact, producing complex relationships among their associated uncertainties. The interaction between multiple hazards and their uncertainties can have nonlinear effects, where the resultant risk and uncertainty are greater than the sum of the risk and uncertainty associated with individual hazards. Effectively communicating the uncertainties related to such complicated systems should be a high priority because the frequency and variability of multiple hazard events due to climate change continue to increase. However, the communication of multiple hazard uncertainties and their interactions remains largely unexplored. The lack of practical guidance on conveying multiple hazard uncertainties is likely due in part to the field’s vast expanse, making it challenging to identify entry points. Here, we offer a perspective on three critical challenges related to uncertainty communication across various multiple hazard contexts to galvanize the research community. We advocate for systematic considerations of multiple hazard uncertainty communication that focus on trade-offs between complexity and factors, including mental effort, trust, and usability.

Effects of COVID-19 Uncertainty Visualizations on Novice Risk Estimates

Policy-makers and the general public have made decisions using COVID-19 data visualizations that have affected the health of the global population. However, the impact that such wide use of data visualizations has had on people's beliefs about their personal risk for COVID-19 is unclear. We conducted two experiments (N = 2,549) during the height of the COVID-19 epidemic in the United States to examine if real-time COVID-19 visualizations influenced participants' beliefs about the risk of the pandemic to themselves and others. This work also examined the impact of two elements of COVID-19 data visualizations, data properties (cumulative- vs. incident-death metrics) and uncertainty visualization techniques (historical data only, and forecasts with no uncertainty, vs. nine uncertainty visualization techniques). The results revealed that viewing COVID-19 visualizations with rising trends resulted in participants believing themselves and others at greater risk than before viewing the COVID-19 visualizations. Further, uncertainty visualization techniques that showed six or more models evoked the largest increases in risk estimates compared to the visualizations tested. These results could inform the design of public pandemic risk communication.

Leveraging Different Visual Designs for Communication of Severe Weather Events and their Uncertainty

<p>Effective communication of potential weather hazards and its uncertainty to the general public is key to prevent and mitigate negative outcomes from weather hazards. The general public needs effective tools at hand that can allow them to make the best decision as possible during a severe weather event. Currently, there are many approaches for weather forecast visualization, such as contour and thematic maps [5]. However, guidelines and best practices in visualization can help to improve these designs and make them more effective [1, 2].</p><p>In this work, we present several interactive visual designs for mobile visualization of severe weather events for the communication of weather hazards, their risks, uncertainty, and recommended actions. Our approach is based on previous work on uncertainty visualization [5], cognitive science [6], and decision sciences for risk management [3, 4]. We propose six configurations that vary the ratio of text vs graphics used in the visual display, and the interaction workflow needed for a non-expert user to make an informed decision and effective actions. Our goal is to test how efficient these configurations are and to what degree they are suitable to communicate weather hazards, associated uncertainty, risk, and recommended actions to non-experts. Future steps include two cycle of evaluations, consisting of a first pilot to rapidly test the prototype with a small number of participants, collect actionable insights, and incorporate potential improvements. In a second user study, we will perform a crowd-sourced extensive evaluation of the visualization prototypes.</p><p><strong>References</strong></p><p>[1] A. Diehl, A. Abdul-Rahman, M. El-Assady, B. Bach, D. A. Keim, and M. Chen. Visguides: A forum for discussing visualization guidelines. In <em>Proceedings of the EuroVis Short Papers</em>, pages 61–65, 2018.</p><p>[2]  A. Diehl, E. E. Firat, T. Torsney-Weir, A. Abdul-Rahman, B. Bach, R. S. Laramee, R. Pajarola, and M. Chen. VisGuided: A community-driven approach for education in visualization.  In Proceedings Eurographics Education Papers, to appear, 2021.</p><p>[3] N. Fleischhut and S. M. Herzog. Wie laesst sich die unsicherheit von vorhersagen sinnvoll kommu- nizieren? In <em>Wetterwarnungen: Von der Extremereignisinformation zu Kommunikation und Handlung. Beiträge aus dem Forschungsprojekt WEXICOM</em>, pages 63–81. 2019.</p><p>[4] G. Gigerenzer, R. Hertwig, E. Van Den Broek, B. Fasolo, and K. V. Katsikopoulos. “A 30% chance of rain tomorrow”: How does the public understand probabilistic weather forecasts? <em>Risk Analysis: An International Journal</em>, 25(3):623–629, 2005.</p><p>[5] I. Kübler, K.-F. Richter, and S. I. Fabrikant. Against all odds: multicriteria decision making with hazard prediction maps depicting uncertainty. <em>Annals of the American Association of Geographers</em>, 110(3):661–683, 2020.</p><p>[6] L. M. Padilla, I. T. Ruginski, and S. H. Creem-Regehr. Effects of ensemble and summary displays on interpretations of geospatial uncertainty data. <em>Cognitive research: principles and implications</em>, 2(1):1–16, 2017.</p>