The authors have declared that no competing interests exist.
Conceived and designed the experiments: SK ML PJL. Performed the experiments: ML. Analyzed the data: SK ML PJL. Contributed reagents/materials/analysis tools: SK PJL. Wrote the paper: SK PJL.
Many theorists argue that the probabilities of unique events, even real possibilities such as President Obama's reelection, are meaningless. As a consequence, psychologists have seldom investigated them. We propose a new theory (implemented in a computer program) in which such estimates depend on an intuitive nonnumerical system capable only of simple procedures, and a deliberative system that maps intuitions into numbers. The theory predicts that estimates of the probabilities of conjunctions should often tend to split the difference between the probabilities of the two conjuncts. We report two experiments showing that individuals commit such violations of the probability calculus, and corroborating other predictions of the theory, e.g., individuals err in the same way even when they make nonnumerical verbal estimates, such as that an event is
Everyone from Aristotle to aboriginals engages in probabilistic thinking, even if they know nothing of the probability calculus. In April 2012, we judged the probability that this paper would appear in
Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student she was deeply concerned with issues of discrimination and social justice and also participated in antinuclear demonstrations.
The participants ranked the probability that Linda is a feminist bank teller as higher than the probability that Linda is a bank teller. The description is more representative of the former than the latter. Frequentists retorted that such a flagrant violation of the probability calculus was a result of a psychological experiment that obscured the rationality of the participants, and that the norms of the calculus are relevant only to judgments about naturally occurring frequencies
We show that naive individuals violate the probability calculus in simple estimates of real possibilities, not just in scenarios contrived to elicit the use of the representativeness of a description as a guide to its probability. Previous studies have seldom examined estimates of such probabilities, e.g.:
What is the chance that Obama is reelected President in November?
As our theory predicts, they too lead to systematic errors. A major mystery about such estimates is the mental operations that underlie them, and an even bigger mystery is where the numbers come from and what determines their magnitudes. To solve these mysteries, we developed a theory based on mental models
Suppose that you are asked the question about the possible reelection of Obama; what estimate would you give? At the time of writing, you are likely to estimate a probability of around 54% (as evinced in online betting sites, such as intrade.com). A numerical estimate comes to mind quite readily for most individuals, but the process – according to the present theory – is quite complex, and depends on two separate components, an intuitive prenumerical component and a deliberative component that carries out arithmetic and that maps intuitions into numerical estimates. This sort of distinction is familiar in “dual process” accounts of reasoning and decision making
The intuitive system constructs an iconic, nonnumerical magnitude representing the strength of belief in a proposition. The first step in estimating, say, the probability of Obama's reelection is to call to mind relevant evidence, such as:
Most incumbent US Presidents are reelected.
According to the theory, mental models represent possibilities
Each row in this diagram represents an incumbent, and so the first row represents an incumbent who is reelected, as do the second and third rows, and the last row represents an incumbent who is not reelected. Mental models, of course, represent individuals, and we use words in the diagram above solely for convenience. The absolute numbers of individuals in the model are not fixed, and during inference they can be modified, or even tagged with numerical values, provided that the result does not contravene the meaning of the quantified assertion as embodied in a separate “intensional” representation
The intuitive prenumerical system yields an analog magnitude monotonically related to the proportion of possibilities in the mental model in which Obama is reelected. We refer to this system as “prenumerical” because it uses a representation of numbers of the sort that is found in infants
The left vertical represents impossibility, the right vertical represents certainty, and the proportional length of the line represents the probability of the event. This representation can be translated into a verbal estimate, such as:
The reelection of Obama as US President is
The theory assumes that individuals can draw from more than one source of evidence and accordingly alter their degree of belief. Some evidence may already be in the form of a probability. But, other evidence may not be, e.g.:
Few Presidents during economic recessions are reelected.
The mental models of this evidence can also yield an analog representation:
In isolation this representation yields an evaluation of the reelection as
In general, given P(
The answer to this question reflects a fundamental aspect of the prenumerical system. It has no access to working memory, and so it can hold at most one icon representing a belief
A simple computation of a compromise is to shift the pointer towards the righthand end of the line and at the same time to shift the end of this line towards the pointer until they meet, i.e., the result of splitting the distance roughly halves it. And the point where they meet is the new degree of belief that takes into account both pieces of evidence:
An analogous problem occurs when individuals have to estimate the probability of a conjunction, P(
What is the likelihood that US unemployment declines by several percentage points this year and that Obama is reelected President?
Once again, the intuitive system has only a limited number of options in coping with conjunctions. One option is to treat them in the same way as separate pieces of evidence, that is, as calling for a compromise. Hence, an icon representing the probability of the preceding conjunction, in effect, splits the difference between the intuitive probabilities of its respective conjuncts. Some of our unpublished experiments suggest that this procedure is used for other sorts of compound assertions too, even inclusive disjunctions. Naive individuals confuse uncertainty with improbability: a disjunction creates uncertainty, and so in error they take it to be more improbable than one or both of its disjuncts.
Another option is available to the prenumerical system, and embodied in mReasoner. It can be illustrated in the case of a
In contrast to the intuitive system, the arithmetical system makes use of a working memory for the results of intermediate computations
into numerical values, such as: 25%. Its conversions are subject to error, which is an inevitable consequence of mapping icons into a fine numerical scale. A corollary is that the mapping can err without yielding a conjunction fallacy, e.g., P(
Once individuals map iconic representations to numerical ones, they can in principle hold the numerical estimates in working memory and make more sophisticated estimates of the probability of conjunctions. If you estimate the chance of Obama's reelection as 60%, and the chance of an economic recovery in the USA as 40%, then you might take 60% of 40%, or vice versa, as your estimate of their joint occurrence. Such multiplicative estimates are in accordance with the probability calculus provided that the two events are independent – a condition that the Obama example violates, and so it calls for the computation of P(
The arithmetical system can try to keep track of the complete
In sum, the theory makes four main predictions about the estimates of the probabilities of real but unique possibilities:
Participants from the same population have access to roughly the same sorts of evidence about real possibilities, and so their estimates should concur reliably.
Estimates of the conjunctions of events should yield frequent violations of the JPD, where a violation is defined as a negative probability in at least one of the four probabilities that comprise the JPD, i.e., P(
Violations of the JPD should be reduced if individuals have already made numerical estimates of the probabilities of the respective conjuncts, because these estimates allow them to use more sophisticated numerical estimates, such as taking a percentage of a percentage, i.e., a “multiplicative” estimate. Such estimates, of course, are irrational in the case that the two conjuncts are not independent.
Violations of the JPD should also occur, but to a reduced degree, with a verbal scale of probabilities in comparison with a full percentage scale.
We carried out several experiments to test these predictions, and report the two most important and representative of them, but their principal results have been replicated in other studies.
Experiment 1 tested three predictions: first, participants should concur in their estimates of the probabilities of unique events; second, their estimates should frequently violate the JPD; and, third, these errors should be reduced in favor of more sophisticated multiplicative procedure when participants have already made numerical estimates of the likelihoods of the conjuncts before they estimate the likelihood of their conjunction.
39 participants completed Experiment 1 for monetary compensation (a $10 lottery) on Amazon Mechanical Turk, an online platform hosted on Amazon.com
Order  Question  Probability estimate 
1  What is the probability that a nuclear weapon will be used in a terrorist attack in the next decade?  P( 
2  What is the probability that there will be a substantial decrease in terrorist activity in the next 10 years?  P( 
3  What is the probability that a nuclear weapon will be used in a terrorist attack in the next decade 
P( 
Participants responded to questions 1–3 with numerical estimates ranging from 0 through 100.
Conjunctive events (preceded by “What is the probability that…”)  P( 
P( 
P( 


…the United States will sign the Kyoto Protocol and commit to reducing CO2 emissions and global temperatures reach a theoretical point of no return in the next 100 years?  47  42  44 
…US companies focus their advertising on the Web next year and the 
69  41  42 
…intellectual property law in the US will be updated to a reflect advances in technology by the year 2040 and Russia will become the world center for software development by 2040?  54  24  27 
…a nuclear weapon will be used in a terrorist attack in the next decade and there will be a substantial decrease in terrorist activity in the next 10 years?  39  27  26 
…the United States adopts an open border policy of universal acceptance and English is legally declared the official language of the United States?  15  46  26 
…Greece will make a full economic recovery in the next 10 years and Greece will be forced to leave the EU?  33  33  25 
…scientists will discover a cure for Parkinson's disease in 10 years and the number of patients who suffer from Parkinson's disease will triple by 2050?  39  32  25 
…Honda will go bankrupt in 2012 and Ford will go bankrupt before the end of 2013?  19  23  15 


…a new illegal but synthetic drug becomes popular in the USA over the next two years and the movement to decriminalize drugs doubles its numbers by 2015?  58  48  49 
…3dimensional graphics will be required to contain explicit markers to indicate their unreal nature by 2020 and competitive video game playing will achieve mainstream acceptance by 2020?  41  52  45 
…the Supreme Court rules on the constitutionality of gay marriage in the next 5 years and a gay person will be elected as president in the next 50 years?  65  40  38 
…a significant upturn in the economy occurs next year and Obama will be reelected President in 2012?  36  55  38 
…in less than 15 years, millions of people will live past 100 and advances in genetics will end the shortage of replacement organs in the next 15 years?  36  38  37 
…space tourism will achieve widespread popularity in the next 50 years and advances in material science will lead to the development of antigravity materials in the next 50 years?  34  40  36 
…at least one head of state will be assassinated by 2012 and NATO will grant military support to Arab Spring movements in several countries?  39  36  32 
…intelligent alien life is found outside the solar system in the next 10 years and world governments dedicate more resources to contacting extraterrestrials?  20  18  17 
The table presents the contents in which
Experiment 2 used the same method as the preceding experiment to test the theory's prediction that an inconsistent JPD should occur in both verbal and numerical estimates, but tend to be greater with numerical estimates because of the use of a finer scale. The verbal judgments were on a 7point ordinal scale:
18 participants completed Experiment 2 for monetary compensation on the same online platform as in the previous study. All of the participants stated that they were native English speakers.
To mask the relation between the conjunctions and their conjuncts, participants made four probability estimates for each problem, and there were four forms of problem:
P(A&B), P(A), P(B)
P(¬A&B), P(¬A), P(B)
P(A&¬B), P(A), P(¬B)
P(¬A&¬B), P(¬A), P(¬B)
In each case, there was a fourth judgment corresponding to the probability of a conjunction of the respective negations of the two propositions in the initial conjunction. As in the previous study, half the problems were those in which
As
Latency of estimates (in s)  
Order of estimates  Percentage of violations of JPD  P( 
P( 
P( 
P(A), P(B), P(A&B)  51  8.46  7.53  7.49 
P(A&B), P(A), P(B)  62  6.47  5.84  12.77 
Panel A shows the estimates of 2000 simulated runs of the computational model and its best fitting linear regression plane, and Panel B shows participants' estimates. Participants' estimates were separated by whether the estimate reflected zero, one, or two violations of the JPD. A violation was defined as a negative probability in the JPD extrapolated from the estimates. In the 2D scatterplots, estimates of P(
The study revealed an effect of content. If event
Participants violated the JPD on 34% of their verbal judgments and on 68% of their numerical judgments (Wilcoxon test, z = 3.55, p<.0005) even though their verbal estimates were faster than their numerical estimates (65.9 s to estimate all four probabilities vs. 85.1 s, Wilcoxon test, z = 3.33, p<.001). These results corroborated the prediction that violations should occur in verbal estimates, but at a reduced rate because of the relative coarseness of the scale. The four different sorts of problem yielded no reliable differences in participants' tendency to yield inconsistent JPDs (Friedman test, χ^{2} = 4.19, p = .12), and the two different relations between
Participants' estimates were separated by whether the estimate reflected zero, one, or two violations of the JPD. A violation was defined as a negative probability in the JPD extrapolated from the estimates. In the 2D scatterplots, estimates of P(
The mechanisms underlying naive estimates of the probabilities of unique events are largely inaccessible to consciousness, but they are open to psychological investigation. We proposed a modelbased theory, which was designed to solve the mystery of the mental operations underlying these estimates, and the deeper mystery of where the numbers come from and what determines their magnitudes. Like other theories of judgment and reasoning
Two experiments corroborated the main predictions of this theory and its implementation. They showed that individuals tend to concur in the rank order of their estimates of the probabilities of unique events (prediction 1). For example, they inferred that the US is much less likely to adopt an open border policy (mean estimate: 15%) than to make English the official language of the country (mean estimate 46%; see
Could splitting the difference be an artifact, or a result of the participants merely guessing probabilities? Two results suggest otherwise. First, the reliable concordances of the estimated probabilities showed that the participants were relying to some degree on beliefs and procedures in common. Second, the large and reliable increase in time to estimate the probability of conjunctions when these estimates occurred before the estimates of their respective conjuncts (in Experiment 1) showed that the participants were thinking in order to make their estimates, and thought nearly twice as long to estimate P(
At present, no rival theories propose mechanisms for the estimates of the probabilities of unique but real possibilities. Critics might argue, however, that the role of mental models and belief icons in yielding the present predictions is superfluous. Any theory, whether it represented probabilities with vague verbal quantifiers, or precise numerical values, could simulate the principles of the present theory and succeed as well in accounting for the results. We have two reactions to this claim. On the one hand, of course a theory might be formulated ex post facto to account for our results, but the strength of the model theory is that its principles emerge naturally from its unification of deductive inference and the representation of quantified assertions, such as:
Model fits  
R^{2}  RMSE  





Wyer (1976)  .65  .17 
Fantino et al. (1997)  .64  .18 



Wyer (1976)  .53  .19 
Fantino et al. (1997)  .54  .19 



Wyer (1976)  .22  .24 
Fantino et al. (1997)  .25  .24 
One final issue warrants discussion. Our experiments examined pairs of events, which a previous norming study showed were not independent of one another. Hence, it is natural to wonder what would happen in estimates of the conjunction of independent events
What is the probability that a cure for Parkinson's disease will be found in ten years and that Greece will leave the EU in the next ten years?
The correct estimate in accordance with the probability calculus is to multiply the probabilities of the two conjuncts. If participants could be persuaded that such questions are sensible, then they should be likely to consider each conjunct independently, and as a result to be biased towards a multiplicative response. At present, mReasoner does not model the putative effects of
Frequentists argue that the probability calculus is inapplicable to the probabilities of unique events
We are grateful for helpful criticisms from Sam Glucksberg, Adele Goldberg, Hua Gao, Geoffrey Goodwin, Matt Johnson, Olivia Kang, Dan Osherson, and Laura Suttle. We are also grateful to Vittorio Girotto, Michel Gonzalez, and especially Nuria Carriedo, for collaborating with the third author in some preliminary studies of the probabilities of unique events.