Many of you take this class with the prospect of applying geoscience knowledge t

Question

Many of you take this class with the prospect of applying geoscience knowledge to your future profession. IBM acquired the Weather Company in 2016, for example, because "weather means business". This week, let us try a virtual jigsaw (cooperative learning) to learn from a very useful lecture on how to best communicate forecast uncertainty to decision-makers.

Uncertainty Forecasts & General Public End-Users - Dr. Susan Joslyn, University of Washington

Duration: (30:54)
User: n/a - Added: 3/21/16

YouTube URL: http://www.youtube.com/watch?v=SfXlt40StpA

The video is entitled “Uncertainty Forecasts & General Public End-Users” by Dr. Susan Joslyn, University of Washington. She asked an over-arching research question of “Should public forecasts include uncertainty information?” Her presentation answered following questions:
Q1. Can people understand expressions of uncertainty?
Q2. Do people make good decisions with uncertainty information? (Starting at 12:00) and
Q3. Does uncertainty information increase or decrease trust? (continued from question 2)
Q4. What is the best way to communicating uncertainty information? (Starting at 20:00)
with conclusions starting at 28:10.

This is a long video, so each group can divide viewing tasks and help each other learn by following these steps:
1. Everyone please watch the first 12 mins of the video for background
2. Then, each person please pick either

Q2 & Q3, or

Q4 to focus on.

3. At last, please don’t forget to watch the conclusions
4. In your posts, please summarize what Dr. Joslyn said to answer the question(s) of your choice and extend with your own thoughts to your group members!
5. Then, reply to one of your group member's post with constructive feedback.

Grading Criteria:
Advanced (AD): Expressing individual thoughts and opinions with critical thinking in one thread, AND replying at least to another person with thoughtful and constructive comments or discussions.

Competent (CP): Expressing individual thoughts and opinions with critical thinking in one thread; not replying to others' threads or only replying with "I agree with you."

Not Competent (NC): Only restating others' threads (without demonstrating understanding or individual thinking) or only replying with "I agree with you."

Explanation / Answer

It can be especially challenging to communicate the uncertainty associated with a decision made in an emergency situation, such as a hazardous chemical spill. Under such circumstances, EPA must communicate not only with those involved in containment and cleanup, but also with members of the public who might be affected by the spill, and the communication may need to be done in coordination with other agencies, with governments, and with stakeholders such as private companies involved in the spill. Such communication, sometimes called crisis communication, is often carried out at a time when there are a number of large uncertainties about the event and its potential consequences on human health and the environment (Reynolds and Matthew, 2005). The time frame within which a decision is needed in an emergency situation can limit the time and opportunities available for communication, and the purpose of communication in such a situation can differ from traditional risk communications in that crisis communication often is principally informative (Reynolds and Matthew, 2005). Although it is generally not possible to predict the timing and extent of an emergency, the nature of many potential emergencies can, and often are, known and planned for. Communicating with stakeholders about the uncertainties that might follow an emergency during the planning for such an emergency and explicitly including the communication of uncertainties in emergency plans can help facilitate communications when an environmental crisis requiring an emergency response occurs. Given the need for a quick decision and the large amount of uncertainty that often occurs in emergency situations, it is important that communication strategies include plans to collect information that might reduce uncertainties or plans to revisit the decision once more data are gathered.

The decision context could also determine whom the agency and its technical staff should communicate with. Furthermore, as discussed below, the characteristics of those with whom EPA is communicating should also affect the strategy for communicating the decision, including the uncertainty in the decision. The communication strategy for a decision that will affect only a small region will differ from the communication strategy for a decision that will have consequences on a national scale. A decision might also have a greater effect on one subgroup than another (for example, a decision that affects the levels of a chemical in fish might affect anglers more than other people), and those subgroups that are more at risk should be identified during the problem-formulation phase, and discussions about potential uncertainties should be initiated during that phase.

The Type and Source of the Uncertainty

Some research indicates that, when communicating uncertainties in the results of risk assessments to decision makers, it is valuable to be specific about the nature or types of the uncertainties. Bier (2001b)discusses two types of outcome uncertainties: (1) state of knowledge or assessment uncertainty, and (2) variability or randomness (in other words, uncertainties arising from variability or natural variation elements in such factors as environments, populations and exposure paths), which cannot be controlled and are thus not reducible.9 Those two categories of uncertainties correspond to what the committee refers to as model and parameter uncertainty, and variability and heterogeneity, respectively.10 Bier (2001b) suggests that, when communicating uncertainties to decision makers, it is helpful to distinguish between the two types of the uncertainties so that the decision maker can understand how much of the uncertainty in the decision may be reducible. For example, if it is not possible to wait for research to reduce state-of-knowledge uncertainty, a decision maker may give more priority to a risk for which there is large state-of-knowledge uncertainty and a small population variability rather than to a risk for which there is large population variability and small state-of-knowledge uncertainty (Bier, 2001b). Others have argued, however, that in most instances this distinction is not a useful one to make since it can result in an overly complicated and confusing analysis (Morgan and Henrion, 1990). It is also important to communicate the sources of uncertainty—for example, whether it arises from the estimates of human health, estimates of costs, the availability of technology, or other factors—and to include the relative impact of the different sources on the decision. Such communication should also discuss the results of any sensitivity analyses, so that the uncertainty is bounded.

Technical experts often communicate uncertainties to decision makers in aggregate. However, aggregate estimates of uncertainty do not necessarily provide the decision makers with an understanding of the uncertainty and its implications for a decision. Information about the type and sources of uncertainty can help decision makers decide whether further research is warranted to decrease the uncertainty or whether to refine the decision to reduce the effects of the uncertainty. Descriptions of where there are uncertainties can also indicate which groups or stakeholders might bear the burden of a higher-than-anticipated health risk or cost because of the uncertainty. Knowing who is likely to be affected by the uncertainty in the costs and benefits would allow decision makers to design the initial proposed regulations to address or prepare for those potential outcomes in advance. If the individual sources of uncertainty that contribute to the overall uncertainty can be determined, then uncertainty analyses in, for instance, cost—benefit analyses or cost-effectiveness analyses could incorporate graphic representations displaying the relative importance of the different sources of uncertainty sequentially so as to provide an easily interpretable graphic display of the sources of uncertainty (Krupnick et al., 2006).

Audience Characteristics

Level of Technical Expertise

The audience for the communication of uncertainty in environmental decisions, such as those made by the EPA, will have a broad array of backgrounds and roles (Wardekker et al., 2008). EPA's scientific and technical staff communicates about uncertainty in health risk estimates, economic analyses, and other factors with agency decision makers. The agency discusses the uncertainties in its decisions with stakeholders, including individuals who might have little to no technical knowledge, as well as with industry specialists and others with high levels of technical expertise. The uncertainty that affects decisions should be discussed with all stakeholders, but the strategy used for those discussions might vary with the technical expertise of the audience. For example, agency decision makers will often have strong technical backgrounds and might need to see specific numbers to best understand the extent of uncertainty and how it affects their decisions. Industry and advocacy group scientists similarly might prefer specific numbers, as such numbers might provide them with a complete picture and the data needed for them to conduct their own independent analyses. Members of the public without strong technical backgrounds might benefit more from graphic representations of the uncertainties along with discussions about how those uncertainties will be considered in a decision, the potential consequences of those uncertainties, and whether and how EPA plans to decrease those uncertainties. Regardless of the audience, however, EPA should use its communication opportunities to provide audiences with information as well as to gather information from the audience that could help either decrease acknowledged uncertainties or identify additional uncertainties that might affect the decision.

Another potential option now that many documents are available electronically through the Internet is to use layered hypertext for more complex uncertainty analysis. That is, the main body of text and the summary sections of EPA's decision document could contain a summary of the uncertainty analyses conducted and could also include a link to appendixes or other documents that present full details of the analyses. That would provide summary information for all audiences as well as further details of the uncertainty analyses for technical audiences or others with an interest in seeing all the details.

Biases

Uncertainty information concerning probabilities has been found to be susceptible to biases by both experts and non-experts (Hoffrage et al., 2000; Kloprogge et al., 2007; Slovic, 2000; Slovic et al., 1979, 1981; Tversky and Kahneman, 1974). When people's judgments about a risk are biased, risk management and communication efforts may not be as effective as they would otherwise be. Biases can stem from the characteristics of an individual or group or can be embedded in the framing of a message; both types can influence the interpretation of a message. Communicators of information about uncertainty cannot completely eliminate these biases, but they should be aware of the potential for biases to influence the acceptance of and reaction to probabilistic information and, to the extent possible, account for these biases by adjusting communication efforts. These types of biases are discussed below.

Personal Biases One bias that can affect how people interpret probabilistic information is termed availability bias. People tend to judge events that are easily recalled as more risky or more likely to occur than events that are not readily available to memory (see Kloprogge et al., 2007; Slovic et al., 1979; Tversky and Kahneman, 1974). An event may have more availability if it occurred recently, if it was a high-profile event, or if it has some other significance for an individual or group. The overestimation of rare causes of death that have been sensationalized by the media is an example of availability bias. One implication of availability bias that communicators of risk and uncertainty information should be aware of is that the discussion of a risk may increase its perceived riskiness, regardless of what the actual risk may be (Kloprogge et al., 2007). For example, evidence indicates that women overestimate their risk of having breast cancer; women believe that their risk of breast cancer is higher than their risk of cardiovascular disease, despite the fact that cardiovascular disease affects and kills more women than breast cancer (Blanchard et al., 2002).

A second bias that can influence the communication of health risks and their uncertainties is confirmation bias. Confirmation bias refers to the filtering of new information to fit previously formed views; in particular, it is the tendency to accept as reliable new information that supports existing views, but to see as unreliable or erroneous and filter out new information that is contrary to current views (Russo et al., 1996). People may ignore or dismiss uncertainty information if it contradicts their current beliefs (Kloprogge et al., 2007). Evidence indicates that probability judgments are subject to confirmation bias (Smithson et al., 2011). Communicators of risk information, therefore, should be aware that peoples' preexisting views about a risk, particularly when those views are very strong, may be difficult to change even with what some would consider to be “convincing” evidence with little uncertainty.

A third bias is confidence bias. People have a tendency to be overconfident about the judgments they make based on the use of heuristics. When people judge how well they know an uncertain quantity, they may set the range of their uncertainty too narrowly (Morgan, 2009). Research by Moore and Cain (2007) supports the notion that people may overestimate or underestimate their judgments based on their level of confidence. Referred to as the overconfidence bias, this tendency seems to have its basis in a psychological insensitivity to questioning of the assumptions upon which judgments are based (Slovic et al., 1979, 1981).

Group Biases The literature on public participation emphasizes the importance of interaction among stakeholders as a way of minimizing the cognitive biases that shape how people react to risk information (see Renn, 1999, 2004). Kerr and Tindale (2004), for example, caution that the more homogeneous a group is with respect to knowledge and preferences, the more strongly the knowledge and preferences will affect a group decision. Uncertainty can be either amplified or downplayed, depending on a group's biases toward the evidence.

Assessment and explicit acknowledgement of the biases of the people that the agency is communicating with might be critical to successful communication. People may be more willing to listen to new information and other points of view after their own concerns have been acknowledged and validated (Bier, 2001a).

EPA's scientists and technical staff are themselves not immune to these biases. An awareness of the possible biases within EPA and when they occur would be a first step toward identifying biases and helping decrease the possibility that such biases influence the interpretation and presentation of scientific evidence.

Considerations for Communicating with Journalists

The statement of task asks the committee if there are specific communication techniques that could improve understanding of uncertainty among journalists. This is an important question, as most members of the public get their information about risks from the media. Journalists and the media help to identify conflicts about risk, and they can be channels of information during the resolution of those conflicts (NRC, 1989). Journalists do generally care about news accuracy and objectivity (NRC, 1989; Sandman, 1986) and about balance in representation of opinions, but journalists vary widely in their backgrounds, technical expertise, and ability to accurately report and explain environmental decisions. Even those who cover environmental policy making will not necessarily be familiar with the details of risk assessment and its inherent uncertainties, making it challenging to convey the rationale for decisions based, in part, on those assessments.

Uncertainty is not unique to reporting on environmental health risks, of course. Studies of how the U.S. news media handle uncertainty in science in general have found that journalists tend to make science appear more certain and solid than it is (see Fahnestock, 1986; Singer and Endreny, 1993; Weiss et al., 1988). In a quantitative content analysis, for example, Singer and Endreny (1993) found that the media tended to minimize uncertainties of the risks associated with natural and manmade hazards. The issue of which factors might contribute to this tendency to minimize uncertainties has not yet been studied, but the tendency could be related to journalists' understanding of uncertain information versus their incentive to develop attention-grabbing stories that omit or downplay uncertainties. It should be expected that journalists, just like most other people, will tend to interpret risk messages based on their existing beliefs. The reporting of risk and uncertainty information in the media will be influenced accordingly.

Because the journalists and the media are a major avenue for framing risk information and its inherent uncertainty, efforts are needed to ensure that they are well informed of what is known about risks and risk-management options, including the sources and magnitude of uncertainty and its implications; a particularly useful approach would be to provide journalists with short, concise summaries about those implications. Although such summaries can be a challenge to develop, it can be done. For example, as discussed in Chapter 2, the summary of the regulatory impact analysis for the CAIR (EPA, 2005) contains a summary discussion of the uncertainty analysis. Those who are most familiar with the risk and uncertainties should provide the perspective that the journalists seek and should recognize the limitations and constraints of the media. Although little research has been carried out on the best means of providing journalists with such a perspective, providing agency personnel with training on how to communicate effectively with media representatives about uncertainties may prove helpful to journalists, as might providing journalists with access to the agency officials who were involved in the decision making. Providing the media with summaries of the uncertainties in the risk assessment and risk management in a variety of formats may also help ensure that the uncertainties are conveyed accurately.

Social Trust

An important concept related to stakeholder values and perceptions is social trust. Trust has long been considered of central importance to risk management and communication (Earle, 2010; Earle et al., 2007; Kasperson et al., 1992; Löfstedt, 2009; Renn and Levine, 1991). Slovic (1993) noted an inverse relationship between the level of trust in decision makers and the public's concern about or perception of a risk—that is, the lower the trust, the higher the perception of risk. The importance of organizational reputation is not unique to EPA; in Reputation and Power: Organizational Image and Pharmaceutical Regulation at the FDA, Carpenter (2010) emphasized the importance that the U.S. Food and Drug Administration's reputation plays in its regulatory authority.

Frewer and Salter (2012) point out that beliefs about the underlying causes of trust or distrust and about the best approaches for increasing trust have changed over the past few decades. In contrast to the old idea that increasing knowledge will increase trust, Frewer et al. (1996) found that certain inherent aspects of the source of information—such as having a good track record, being truthful, having a history of being concerned with public welfare, and being seen as knowledgeable—lead to increased trust. Similarly, Peters et al. (1997) found that the source of the information being seen as having “knowledge and expertise, honesty and openness, and concern and care” was an important contributor to trust (p. 10). In a study looking at attitudes toward genetically modified foods, however, Frewer et al. (2003) found that neither the information itself nor the strategy for communicating the risks had much effect on people's attitudes toward genetically modified foods; in this case, people's attitudes toward genetically modified foods tended to determine their level of trust in the source of information, rather than the trust in the source determining their attitudes toward the foods. It is important to remember, however, that there are reasons to communicate uncertainties beyond the potential to increase social trust (Stirling, 2010).

Earle (2010) reviewed the distinction between trust, which is about relationships between people, and confidence, which concerns a relationship between people and objects, and the role of both in social trust. As Earle (2010) pointed out, although some people believe that decisions should be made on the basis of data or numbers (Baron, 1998; Bazerman et al., 2002; Sunstein, 2005), any “confidence-based approach presupposes a relation of trust” (p. 570). For decisions concerning hazards of high moral importance, that trust does not necessarily exist (Earle, 2010).

As discussed by Fischhoff (1995) and by Leiss (1996), at earlier stages in the evolution of risk communication sciences it was thought that public education via increased communication would lead to an increased understanding of the concept of risk and, subsequently, to increased trust. Furthermore, some research had indicated that a decrease in public confidence in regulatory agencies and scientific institutions—and their motives—led to decreased trust (Frewer and Salter, 2002; Pew Research Center for the People and the Press, 2010). Given those observations, it was thought that increasing transparency would be one way to increase trust. As discussed by Frewer and Salter (2012), however, there is limited evidence that transparency actually does increase trust, although there is evidence that a lack of transparency can lead to increased distrust (Frewer et al., 1996). As highlighted in the discussion of the committee's framework for decision making in Chapter 5, all aspects of the decision-making process, including the more technical risk assessment process, require value judgments. Thus engaging the public and policy makers, in addition to scientists, in the process of health risk assessments not only improves the assessment, but can also increase both trust in the process and communications about health risks by allowing the perspectives of all stakeholders to inform the assessment. Frewer and Salter (2002)described the communications by the United Kingdom's regulatory agencies related to the bovine spongiform encephalopathy (BSE) outbreak in the mid-1990s in the United Kingdom as an example of the consequences of inadequate public participation in the decision process. Communications about the outbreak and the outbreak response did not address many of the concerns of the public and led to public outrage about the response.

Concerning the communication of uncertainties in risks, Frewer and Salter (2012) pointed out that distrust in risks assessments will increase when uncertainties are not included in the discussion of the assessments. Although some researchers noted, for the BSE outbreak in the United Kingdom, an apparent view by government officials “that the public [is] unable to conceptualize uncertainty” (Frewer et al., 2002, p. 363), research on risks related to food safety indicates a preference by the public to be informed of uncertainties in risk information (Frewer et al., 2002) and finds that not discussing uncertainties “increases public distrust in institutional activities designed to manage risk” (Frewer and Salter, 2012, p. 153).

Although, there is insufficient information to develop guidelines or best practices for communicating the uncertainty and variability in health risk estimates (Frewer and Salter, 2012), there is evidence that the public can differentiate between different types and sources of uncertainty (see below for further discussion). As discussed by Kloprogge et al. (2007), it is possible to communicate to the public various aspects of uncertainty information, such as how uncertainty was dealt with in the analysis as well as the implications of uncertainties and what can or cannot be done about uncertainties.

The need for a communication plan is increased when there are—or are expected to be—more uncertainties associated with a decision-making process, because there are likely to be more challenges in communicating with stakeholders. Research demonstrates a heightened interest by the public in evaluating the credibility of information sources when they perceive uncertainty (Brashers, 2001; Halfacre et al., 2000; van den Bos, 2001), and studies also indicate that the public is more likely to challenge the reliability and adequacy of risk estimates and be less accepting of reassurances in the presence of uncertainty (Kroll-Smith and Couch, 1991; Rich et al., 1995). Concerns about procedural fairness and trust appear to be even more salient when there is scientific uncertainty (NRC, 2008), and risk communication can serve to facilitate stakeholder trust (Conchie and Burns, 2008; Heath et al., 1998; Peters et al., 1997).

Go to:

KEY FINDINGS

RECOMMENDATION 8.1

U.S. Environmental Protection Agency senior managers should be transparent in communicating the basis of the agency's decisions, including the extent to which uncertainty may have influenced decisions.

RECOMMENDATION 8.2

U.S. Environmental Protection Agency decision documents and communications to the public should include a discussion of which uncertainties are and are not reducible in the near term. The implications of each to policy making should be provided in other communication documents when it might be useful for readers.

The best presentation style will depend on the audience and their needs. When communicating with decision makers, for example, because of the problem of variability in interpretation of verbal presentations, such presentations should be accompanied by a numeric representation. When communicating with individuals with limited numeracy or with a variety of stakeholders, providing numeric presentations of uncertainty may be insufficient. Often a combination of numeric, verbal, and graphic displays of uncertainty information may be the best option. In general, however, the most appropriate communication strategy for uncertainty depends on

RECOMMENDATION 9.1

The U.S. Environmental Protection Agency, alone or in collaboration with other relevant agencies, should fund or conduct research on communication of uncertainties for different types of decisions and to different audiences, develop a compilation of best practices, and systematically evaluate its communications.

RECOMMENDATION 9.2

As part of an initiative evaluating uncertainties in public sentiment and communication, U.S. Environmental Protection Agency senior managers should assess agency expertise in the social and behavioral sciences (for example, communication, decision analysis, and economics), and ensure it is adequate to implement the recommendations in this report.

Related Questions

Navigate

• Browse (All)
• Browse M
• Subjects

Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.