The survey responses were collected through e-mail by Marketing Systems Group (MSG) and SurveySavvy, a professional survey company. The baseline survey was conducted between April 9 and April 17, 2013. A follow-up survey was conducted 6 months later, between September 30 and October 28, 2013. Subjects were recruited from SurveySavvy’s current pool of survey respondents. SurveySavvy recruits subjects into the survey pool through the company’s website and a proprietary system of online referrals. In addition, SurveySavvy reaches out to groups that are under-represented in their pool via e-mail and telephone, both landline and cellular, in order to ensure that the survey pool is close to nationally representative. Recruitment information screen shots, the text of the e-mail sent to the prospective participants, and a recruitment flowchart can be found in S1 Text.

The study received approval from the University of Illinois, Urbana-Champaign’s Institutional Review Board for non-biomedical human subject research. The only criterion for participating in the baseline survey was for respondents to be 18 years of age or older (self-reported) and located in the United States. An additional criterion for the 6-month follow-up survey was for respondents to have completed the baseline survey. No vulnerable populations were targeted, and none of the subjects were likely to be associated with the researcher. All participants read an informed consent form prior to starting the survey. This form informed the participants about the purpose of the study, the duration of the survey (approximately 10 minutes), the incentive for survey completion (the opportunity to win one of six $50 prizes in addition to a financial incentive provided by SurveySavvy), the existence of a 6-months follow-up survey, the fact that participation was completely voluntary and could be terminated at any point, and the anonymity and processing of individual data. The form also provided investigator contact information. Subjects were not exposed to any risks or deception and were able to quit the survey at any point. At the end of the form, the participants could either terminate participation or click “Yes” to continue to the survey. Due to the online nature of the survey and minimal risk to the subjects, IRB granted a waiver of written informed consent. The full informed consent forms for the initial survey and 6-month follow-up are available in S2 Text. The survey did not ask for personally identifiable information, such as name or email. In order to ensure complete anonymity, results are presented in the aggregate across all survey respondents. Finally, the data are stored on the hard drives of the researchers’ password-protected computers.

The “closed” electronic survey was sent to prospective participants via e-mail that provided a unique link to the survey. The survey was programmed online using SurveyMonkey.com. The uniqueness of the link allowed duplicate responses to be identified and dropped. Thus, it was not necessary to use cookies to track respondents. Similar questions were grouped on the same screen. There were a total of 14 screens in the baseline survey: a screen with the informed consent information, 12 question screens, and a screen thanking respondents for their participation. There were a total of 13 screens in the 6-month follow-up survey: a screen with the informed consent information, 11 question screens, and a screen thanking respondents for their participation. Participants could not go back to review or change their answers to previous questions in either survey. The order of questions was kept the same across all participants, and adaptive questioning was not applicable. The order of answers within each multiple choice question was randomized, but the order of the questions themselves did not vary. The full survey text is available in S3 Text. Prior to running the full-scale survey, the usability and technical functionality of the electronic questionnaire was tested with 5 volunteers. A pilot survey with 201 respondents was conducted to test whether questions were well-formulated. Because there was no follow-up with the pilot respondents, they are excluded from the results reported in this paper. However, their inclusion would not significantly change the baseline results.

Out of the 2,484 invited participants, 1,593 respondents initiated and 1,300 completed the baseline survey (for a participation rate of 64% and a completion rate of 82%). Because the survey was sent to prospective participants via e-mail, the “view rate” is equal to the participation rate in our survey; we did not have the capability to track whether respondents opened the email or not. Out of the 1,300 participants who completed the baseline survey, 886 initiated and 747 completed the follow-up survey (for a participation rate of 68% and a completion rate of 84%). While there were some statistically significant differences between the characteristics of the initial sample and of those who completed the follow-up survey (see S4 Text), there was no differential attrition by treatment status.

Only completed questionnaires were analyzed. We did not measure the time respondents spent answering the questions; all completed questionnaires were used in our analysis. No statistical correction methods were applied to either the baseline or the follow-up samples. Summary statistics of respondent characteristics can be found in S4 Text. Our sample was broadly similar to the US as a whole, with the exception of education: a significantly higher share of our respondents had a bachelor’s degree or higher [18].

In the baseline survey, all respondents first answered questions that elicited general attitudes toward climate change science and assessed knowledge about climate science (e.g., being able to name greenhouse gases). Next, the participants were randomly assigned to one of three equal groups. The control group received no information to provide benchmark measurements of (a) beliefs about the scientific consensus and (b) the relationships between beliefs about the scientific consensus, beliefs about climate change, and political ideology.

The treatment group viewed a screen with the following passage that informed them about scientists’ beliefs about climate change (“hard information treatment,” hereafter HI). To present the information as objectively as possible, we used exact quotes from the survey of scientists from which the information was obtained.

The third group viewed a screen with similarly worded but vague information about climate change (“soft information treatment,” hereafter SI).

This treatment allows us to test whether simply drawing respondents’ attention to the fact that a consensus exists, without providing any concrete evidence, affects beliefs. Here, we are agnostic as to whether the impact on beliefs, if any, is due to (a) being reminded about something subjects already knew (the salience channel), (b) the novelty of this vague information to at least some subjects, or (c) both.

Following the information screen, the HI group answered questions about the credibility of the information we provided in order to gauge their level of skepticism. The SI and the control groups answered questions about their perception of what scientists believe. It is possible that by asking HI respondents about the credibility of the information, we caused some subjects to think that the information was incorrect, undermining its credibility. If that were the case, our estimated treatment effect would have been smaller than if we had asked the questions about the credibility of the information last.

All groups then answered the same key questions of interest, which elicited respondents’ own beliefs about climate change and their willingness to sacrifice a portion of a monetary prize in order to contribute toward a cause that counters climate change by promoting energy efficiency. The wording of the statements was chosen to correspond closely to the scientist survey in [1]. Specifically, questions regarding beliefs about climate change were phrased in the following way.

We then asked the following question to gauge whether beliefs translate into willingness to pay.

The decision about donations was thus incentivized by asking respondents how much of their own potential winnings they would be willing to sacrifice to address the issue of climate change. We selected a relatively unknown organization, whose methods (promoting energy efficiency) are relatively uncontroversial, conditional on being involved in climate change. By specifying that the donation would be “anonymously sent”, we tried to reduce the possibility that the study would be viewed as being associated with or conducted by that organization. However, because this question was last among those used to elicit climate change beliefs, any differential skepticism on behalf of the treated and control groups at this point could not have affected answers to the previous questions.

The survey concluded with detailed demographic questions including ideology and educational attainment. Finally, in order to see whether the treatment produced lasting effects, we conducted a follow-up survey 6 months later. The key questions were identical to the baseline except that the follow-up survey did not contain any treatment or treatment-specific questions.

The responses of the control group about scientists’ views on climate change reveal that the public underestimates the degree of scientific consensus, as measured by the survey of scientists we use to provide information to the treated group [1]. In particular, respondents in the control group believe that only 72% of scientists agree that global warming is a process that is already underway (different from 95%, the percentage of scientists agreeing with the same statement, at p < 0.001), that only 69% of scientists believe that human activities are accelerating global warming (different from 88% at p < 0.001), and that 32% of scientists would say that there is no need for immediate policy decisions (different from 9% at p < 0.001).

We also find a great degree of skepticism toward the information shown to the HI treatment group. Almost two-thirds (65%) of the treated group did not think the information from the scientist survey was accurately representing the views of all scientists who were knowledgeable about climate change. Only about 20% of the skeptical respondents thought that participating scientists misstated their true views. The skepticism largely stems from the concern that the scientists polled (“US scientists who published in top journals”) were not representative of all scientists knowledgeable about climate change: 85% of skeptical respondents chose that as one of the reasons for thinking that the information was inaccurate. Being unemployed, liberal or conservative (versus moderate) were the only significant predictors of not trusting the information from the survey of US climate scientists.

Those in the treatment group who were skeptical of the information also believed that scientists were less certain about climate change than the control group (by about 4 percentage points, on average). On the other hand, the control group did not think that there would be significant differences in the answers by the following two groups of scientists: (a) all scientists knowledgeable about climate change, and (b) only US scientists who published in top journals. The fact that the treated respondents reported that (b) would not be representative of (a) while the control group does not expect there to be significant differences between them suggests that some people in our sample may suffer from “self-justification” bias. “Self-justification” refers to the phenomenon of justifying one’s behavior or beliefs when facing evidence that is inconsistent with them [20]. For example, in our case, the self-justification bias may explain why some respondents continue to believe that climate change is not caused by humans when facing the fact that most scientists believe that it is. Self-justification bias may also be responsible for the disconnect described above: one way to justify continuing to believe that climate change is not occurring or is not caused by humans in light of the scientific consensus information is to claim that the survey does not accurately reflect all scientists’ views. Alternatively, it may be that learning about US climate scientists’ views on climate change caused some of the treated respondents to rationally conclude that these scientists’ views differ from the views of all scientists knowledgeable about climate change. A more detailed analysis of skepticism can be found in S5 Text.

In this section, we econometrically estimate the short-run effects of information on respondents’ beliefs about climate change and their willingness to contribute toward preventing the onset of climate change. Table 1 shows the treatment effect of information on our various measures of beliefs, conditional on extensive respondent controls, including age, age squared, and indicators for gender, race, employment status, education, income, and political ideology. All specifications also include an indicator for soft information treatment. All subsequent specifications include these controls, unless otherwise specified. Excluding the soft information indicator or doing the analysis using the treated and control groups only does not substantively change the results. All specifications cluster standard errors at the state level. The unconditional mean of each dependent variable is shown in the row “Dep. var. mean”.

Columns (1) and (2) reveal a significant treatment effect: receiving the information about scientists’ beliefs raises respondents’ beliefs that climate change is already underway and that it has been caused by human activity by 6 and 5 percentage points, respectively. The beliefs of those in the soft information group, on the other hand, do not differ significantly from those in the control group, which suggests that beliefs are impacted by the content and evidence, rather than just by the availability of vague information.

Columns (3)-(6) report the effects of information on expected temperature changes due to climate change by year 2050. Overall, the treatment group believes that an increase in temperatures of 2.5 or 5 degrees Fahrenheit is about 6 percentage points more likely than the control group (Columns 3 and 4). On the other hand, there is no difference between the treated and control group in terms of the probability that temperatures decrease over this time (Columns 5 and 6). Finally, the treatment group’s answers about the expected change in temperature by 2050 in degrees Fahrenheit does not differ significantly from the control group (Column 7).

Table 2 investigates whether the effect of information on beliefs translates into policy preferences or actions. Column (1) estimates the effect on information on the belief that “there is enough scientific uncertainty about the rate and extent of global warming and climate change that there is no need for immediate policy decisions”. We find no significant treatment effect. This is consistent with a model where people look to climate scientists for objective scientific information but not public policy recommendations, which also require economic (i.e. cost-benefit) and ethical considerations. The point estimates of the effect of information on the willingness to donate some of the potential winnings (Column 2) and on the donation amount (Column 3) are positive, but not significant. We find a significant difference between the hard information treatment and the soft information treatment, however. In fact, vague information seems to decrease donations. This finding is consistent with previous work by [21] who find that information reduces misperceptions that vaccines cause autism but nonetheless decreases intent to vaccinate among parents who had the least favorable vaccine attitudes.

Next, we decompose the treatment effect according to respondents’ political ideology, education, and climate change knowledge. Table 3 shows the effects of information broken down by reported political ideology. In Column 1, the belief that climate change is already underway seems to be most strongly affected by information among the liberals and the conservatives. However, the standard errors are large and we are unable to reject that the three coefficients are significantly different from one another. Columns 2, 4, and 5 suggest that information most strongly affects moderates’ beliefs about humans causing climate change and about the probability of temperature increases, although once again, we are unable to reject that the three ideological groups respond to information in an identical manner. Beliefs about policy actions are not significantly affected by information for any of the ideological groups (Column 3). Finally, treated conservatives donate marginally more to climate change causes, although we fail to reject that the point estimates for the three groups are equal (Column 6).

Table 4 decomposes the effect of information by educational attainment. In our survey, educational attainment is broken up into 8 categories, ranging from less than high school to doctoral degrees (PhD) and professional degrees (MD, JD, DDS, etc.). For the purpose of this analysis, however, we combine the categories into two bins: some college or 2-year degree and below (low education) and 4-year college degree and above (high education). We find that receiving hard information significantly impacts the highly educated in terms of increasing their beliefs about climate change being already underway, beliefs that there would be an increase in temperature by more than 5 degrees Fahrenheit by 2050, and the amount donated to climate change causes (Columns 1, 5, and 6). On the other, the low education treatment group is more likely to believe that climate change has been caused by human activities (Column 2). However, once again, in all cases, we cannot reject that the high and the low education groups respond to the treatment in the same way.

Finally, Table 5 breaks down the effect of information by prior knowledge about climate science. We measure this knowledge by the number of correct answers to questions about climate science, such as being able to name greenhouse gases (see detailed survey in S4 Text). The high knowledge group answered more than two questions correctly, while the low knowledge group answered 2 or fewer questions. Overall, the precision of our estimates does not allow us to statistically distinguish between the two groups. We find that both high and low knowledge respondents are positively affect by information in terms of their beliefs about climate change being underway (Column 1) and the probability that global temperature will rise by more than 5 degree Fahrenheit by 2050 (Column 5). The beliefs that climate change is caused by humans and that temperature would rise by more than 2.5 degree Fahrenheit by 2050 are significantly increased for the treated low knowledge respondents (Columns 2 and 4), although the coefficients for the high knowledge group are not significantly different.

A 6-month follow-up survey allows us to gauge whether the short-run effects persist over time. In the follow-up, we ask the same questions about beliefs about climate change as we do in the baseline survey, but do not re-administer the treatment. Tables 6 and 7 summarize the results. Although the point estimates on the hard information treatment indicator are positive, we no longer find a significant effect of information, except for the probability that temperatures increase or decrease by 2.5 degrees Fahrenheit or more (Table 6, Columns 3 and 6). At the same time, the standard errors are large, so that we cannot reject the hypothesis that the treatment effects are persistent. Because only roughly half of the original respondents completed the follow-up survey, our small sample size may mask the existence of a long-run effect (or lack thereof).

Power analysis reveals that, holding constant the estimated effect on the beliefs about whether climate change is occurring (Column 1 of Table 6), we would need about 2,600 observations in total (1,300 treated and 1,300 control) to have an 80% chance of rejecting a null hypothesis of no treatment effect with 95% confidence. For the beliefs about whether humans are causing climate change (Column 2 of Table 6), we would need about 5,200 individuals to do so. If these two long-run estimates were significant, they would show that about 50–55% of the original effect of the “hard information” treatment persists for at least six months. Given the magnitude of these estimates, the simplicity of the treatment (i.e., each treated respondent read only one paragraph), and the relative lack of research about whether or not information treatment effects persist, obtaining a larger sample size to see whether the long-run effects are indeed significant should be a fruitful area for future research.