Psychological evidence relating to the TFD is inconclusive. Thorndike [10] measured the effect of one year of schooling in various combinations of subjects on performance on an intelligence test. His findings revealed small improvements associated with the study of French, chemistry and trigonometry, while arithmetic, geometry and algebra were associated with improvements barely above zero. These and other findings (e.g., [11], [12]) have led many researchers to conclude that reasoning skills cannot be divorced from the context in which they are learnt, and therefore to reject the TFD.

Cheng, Holyoak, Nisbett and Oliver [13] found that even training in formal logic did not improve performance on a conditional logic task. Their participants were given 40 hours of training in the logic of the conditional, including modus ponens, modus tollens, denial of the antecedent, affirmation of the consequent and the distinction between the conditional and biconditional. Despite such comprehensive training, they found no significant improvement in performance on four Wason selection tasks [14]. However, this result is not easy to interpret given the varying contexts in which Cheng et al. situated their problems. More recently researchers have questioned whether selection tasks, and in particular contextualized selection tasks, measure conditional reasoning at all (e.g., [15], [16]).

Despite these negative findings, there has been some support for the idea that studying mathematics might develop conditional reasoning ability. Lehman and Nisbett [17] tracked the development of statistical reasoning, verbal reasoning and conditional reasoning in US undergraduates over their four years of study. Although they did not study mathematics students, they did find a significant correlation between improvement in conditional reasoning and the number of mathematics courses taken by the natural science students in their sample. However, their conditional reasoning test consisted of only one abstract, one causal-framed and one permission-framed Selection Task, and one biconditional Selection Task, and so suffered from the same limitations as that used by Cheng et al. [13].

Inglis and Simpson [18] found that mathematics undergraduates ‘outperformed’ intelligence-matched comparison undergraduates on a 32-item abstract conditional inference task [19]: in other words that their behavior more closely matched a material interpretation of conditionals (discussed below). However, across the course of their first year of studies there were no changes in reasoning behavior. Inglis and Simpson offered two possible explanations for the initial between-groups difference on entry to university: either those who are more likely to adopt the material conditional are disproportionately filtered into studying university-level mathematics, or that studying post-compulsory but pre-university mathematics influences conditional reasoning behaviour. This latter account is plausible because in England (where, like ours, Inglis & Simpson's study was conducted) students are able to drop mathematical study at age 16. A minority choose to study it at ‘Advanced Level’ (commonly referred to as A-Level), a two year course, the results of which are used by universities to select incoming undergraduates. Students typically take three or four subjects at A-Level, of which mathematics might be one, or (rarely) two. It might be that studying A-Level mathematics develops one's ability to reason logically, although the A-Level syllabus contains no tuition on conditional statements. A third possibility is that the difference found by Inglis and Simpson was due to between-group differences unrelated to intelligence (which they controlled for), such as thinking dispositions. In this paper we aim to distinguish between these three hypotheses.

Abstract conditional reasoning consists of drawing conclusions from a conditional statement ‘if p then q’ and a premise. Here we restrict our interest to what Evans, Handley, Neilens & Over [20] referred to as basic conditionals: those concerning abstract relationships which are, at least in principle, empirically verifiable (e.g. “if there is a T on the card, then there is a 7 on the card”). Four inferences are typically drawn by participants: modus ponens (MP), denial of the antecedent (DA), affirmation of the consequent (AC) and modus tollens (MT). These inferences are summarised in Table 1 (so an example of a DA inference would be to conclude not-7 from the premises ‘if T then 7’ and not-T). The MP, DA, AC and MT inferences are respectively drawn by around 100%, 55%, 75% and 60% of reasoners [20].

The validity of these four inferences depends upon how the reasoner interprets ‘if p then q’. Here we briefly summarise four possible interpretations of basic conditionals: the material conditional, the biconditional, the defective conditional, and the conjunctive conditional. Truth tables for these interpretations are shown in Table 2.

The material conditional ‘if p then q’ is true except when p and not-q are true; under this interpretation the MP and MT inferences are valid, and the DA and AC inferences invalid. Although the material interpretation is that favored by logicians, it is clear that this is not the meaning which arises in day-to-day conversation, as it has the paradoxical consequence that the truth of not-p implies anything (“if Maastrict is in Belgium, then Rome is in Italy” is a true statement under the material conditional). While proponents of the TFD typically fail to explicitly state which model of the conditional they believe mathematical study promotes, we interpret claims about the development of “man's purely logical faculties” ([21], p. 19) as most likely being concerned with the (logically normative) material conditional.

Under the biconditional interpretation all four inferences are valid: ‘if p then q’ is interpreted to mean ‘p if and only if q’. Although this could be a conjunction of two material conditionals, Evans et al. [20] suggested that at least some reasoners who adopt the biconditional are actually using a ‘simple equivalence’ strategy. Rather than conjoining two materials, they merely expect that p and q must go together (hence MP and AC), and that not-p and not-q must go together (hence DA and MT).

Some reasoners believe that ‘if p then q’ is only relevant when p is true [22]. Under this so-called ‘defective’ interpretation only MP is (immediately) valid. DA, AC and MT are not since none involves a p premise, so the conditional adds no additional information. Nevertheless, it is possible to draw the MT inference under a defective interpretation using a complex combination of MP and a contradiction argument (assume for contradiction p, conclude q by MP, but this is a contradiction with the minor premise not-q, so the assumption p cannot be correct, hence not-p). Mental logic theorists suggest that the relative complexity of this string of deductions is why MT is not as frequently made as MP (e.g., [23]). Finally, reasoners may interpret ‘if p then q’ to mean simply ‘p and q’ [24]. Under this conjunctive interpretation both MP and AC are valid, but neither DA nor MT are (since neither has a p or q premise).

Theories of reasoning differ on the causes of the different interpretations. For example, the mental models theory [25] suggests that reasoners typically represent a conditional ‘if p then q’ with one explicit mental model, together with an implicit model that denotes the possible existence of not-p cases:

Some high-ability reasoners may flesh out the implicit model (a cognitively demanding task), giving them access to the material conditional and the MT inference. But reasoners who forget about the implicit model, or who lack the working memory capacity to flesh it out, are left with their initial explicit model, leading to either the defective or conjunctive interpretation.

In contrast, Evans et al.'s [20] suppositional account suggests that there are two groups of reasoners: (i) a less sophisticated group who see the probability of a conditional P(if p then q) as being equal to the probability of the conjunction P(p & q), this results in the ‘simple equivalence’ strategy discussed earlier; and (ii) a more sophisticated group who see it as being equal to the conditional probability P(q|p), which results in the defective interpretation. Evans et al. suggested that their account can be distinguished from the mental models theory by considering the MT inference. Under Johnson-Laird & Byrne's [25] account, the MT inference should be made by relatively high-ability reasoners (since it involves the fleshing out of implicit models). In contrast the suppositional account suggests that higher-ability participants should draw the MT inference less, as it does not immediately follow from the defective interpretation. In support of this latter account, several studies have found that measures of intelligence are negatively correlated with the frequency of the MT inference [20], [26].

Our goal here was to determine whether, as predicted by TFD, studying mathematics impacts upon students' conditional reasoning. In particular, we investigated whether the extent to which students adopted the material, biconditional, defective and conjunctive interpretations of the conditional changed following a year of mathematical study.

While the TFD claims that studying mathematics develops one's reasoning skills, it does not suggest any cognitive mechanisms for the change. Reasoning performance is related to measures of cognitive capacity (i.e., general intelligence; [20], [26], [27]) and thinking dispositions (i.e., the tendency to use one's cognitive capacity to solve problems; [28]). It is therefore possible that if studying mathematics did change conditional reasoning behavior it might do so via changes in either cognitive capacity or thinking disposition. Here we investigated whether either of these possibilities could provide plausible mechanisms by which the TFD might operate.

Of course, it is neither practical nor ethical to randomly assign participants to courses when high-stakes qualifications are at stake. However, our inclusion of a comparison group who were studying English literature allowed us to attenuate the non-random assignment to conditions to some extent. The comparison group allowed us to distinguish changes that occur simply due to age or education from those specifically related to some aspect of (or related to a factor correlated with) studying mathematics.

In sum, we asked two main questions. First, does studying post-compulsory mathematics influence how one reasons with conditionals? Second, if there is development of conditional reasoning skills, is this the result of a domain-general change in cognitive capacity or thinking disposition?

One hundred and twenty four participants (aged 15 years 4 months–17 years 8 months, M = 16 years 6 months, at Time 1) were recruited from five schools in Leicestershire, Hampshire and Derbyshire, UK. Seventy-seven (41 male) were studying mathematics amongst any other subjects and 47 (17 male) were studying English literature and not mathematics. The literature students served as a comparison group. To avoid factors such as stereotype threat [29] influencing responses, participants were not told about the specific hypothesis, or about the mathematics versus literature comparison. All participants provided written informed consent, and the study was approved by Loughborough University's Ethical Advisory Committee.

The study followed a longitudinal quasi-experimental design. Participants were recruited after they had chosen their post-compulsory subjects and were tested at the beginning (during the first term and as close to the start of term as possible) and end (after teaching had finished) of their first year of post-compulsory study. They completed the same set of tasks at both time points.

Participants in the mathematics group were all studying the first year of Advanced-Level mathematics. Although there are three different versions of this course available to students in England, all have similar content. Among other topics, the syllabus contained sections on algebra, geometry, calculus, trigonometry, probability, mathematical modeling, kinematics and forces (e.g., [30]). Most importantly, students were not taught any proof-based mathematics, nor were they taught the definition of the conditional statement. To formally establish this, as well as inspecting the syllabus, we conducted an analysis of every first year A-level mathematics examination between 2009 and 2011. Of 929 questions set, only one contained an explicit “if…then” sentence, and there were no mentions at all of the terms “modus ponens”, “modus tollens” or “conditional”.

Participants took part in groups (5–34) during the school day under examination conditions. All tasks were given in a single paper booklet. The RAPM task was always completed first with a 15 minute time limit, and the order of the subsequent tasks was counterbalanced between-participants following a Latin Square design. Participants were instructed to work at their own pace until they had completed all tasks and the sessions lasted approximately 45 minutes.

Endorsement rates of each inference type were analysed with a 2×4×2 ANOVA with two within-subjects factors: Time (start and end of the year) and Inference Type (MP, DA, AC, MT), and one between-subjects factor: Group (mathematics and literature). This revealed a significant three-way interaction, F(3,228) = 7.476, p<.001, η_ρ² = .090, shown in Figure 3, which remained significant after controlling for Time 1 RAPM, Time 1 CRT and prior academic attainment, F(3,216) = 5.103, p = .002, η_ρ² = .066. Compared to Time 1, the mathematics students at Time 2 endorsed more MP inferences, t(42) = 2.420, p = .020, d = .413, and rejected more DA, t(42) = 3.978, p<.001, d = −.607, AC, t(42) = 3.060, p = .004, d = −.468, and MT inferences, t(42) = 2.877, p = .006, d = −.446. In contrast, the literature group showed no significant differences between Time 1 and Time 2 scores for any inference, although there was a marginally significant increase in the number of DA inferences endorsed, t(34) = 1.795, p = .082, d = .309.

These responses appear most consistent with an increased tendency for the mathematics students to adopt a defective conditional interpretation (more MP inferences and fewer DA, AC and MT inferences were made at Time 2 compared to Time 1). To test for this we calculated four indices for each participant (at each time point) giving the proportion of responses consistent with each of the four interpretations of the conditional. For example, a person responding entirely in line with the material conditional would respond ‘yes’ to all MP and MT inferences and ‘no’ to all DA and AC inferences. A Material Conditional Index was therefore computed as: number of MP inferences endorsed + number of MT inferences endorsed + (8 – number of DA inferences endorsed) + (8 – number of AC inferences endorsed), giving a material conditional index score out of 32 for each participant at each time point. The consistency scores for each interpretation are shown in Figure 4, and were subjected to a series of 2×2 ANOVAs with one within-subjects factor: Time (start and end of the year) and one between-subjects factor: Group (mathematics, literature). We consider each interpretation in turn.

On the material conditional analysis Group and Time interacted, F(1,76) = 11.860, p = .001, η_ρ² = .135 (p = .007 with covariates). The mathematics group became more material, t(42) = 3.171, p = .003, d = .493, whereas the literature group did not change, p = .092. On the biconditional analysis, Group and Time also interacted, F(1,76) = 7.966, p = .006, η_ρ² = .095, although this was only marginally significant when covariates were included, F(1,72) = 3.697, p = .058, η_ρ² = .049. The mathematics group became less biconditional, t(42) = 3.323, p = .002, d = −.508, whereas the literature group did not change, p = .500.

On the defective analysis, Group and Time interacted, F(1,76) = 17.651, p<.001, η_ρ² = .188 (p = .002 with covariates). The mathematics group became more defective, t(42) = 5.756, p<.001, d = .880, whereas the literature group did not change, p = .767. Finally, on the conjunctive analysis, Group and Time also interacted, F(1,76) = 8.525, p = .005, η_ρ² = .101 (p = .014 with covariates). The mathematics group became more conjunctive, t(42) = 3.534, p = .001, d = .548, whereas the literature group did not change, p = .693.

Comparing the effect sizes of these analyses confirms that the change in the mathematics group is best understood as an increased tendency to adopt the defective interpretation of the conditional. In other words, that over time the mathematics group became more likely to endorse the MP inference, but less likely to endorse the DA, AC and MT inferences. Next we considered whether changes in either cognitive capacity or thinking disposition could represent domain-general mechanisms for this change in conditional reasoning behavior.

To investigate whether changes in the domain-general reasoning measures could account for the changes in the mathematics group's conditional reasoning behaviour, we regressed participants' defective conditional change scores (Time 2 defective conditional index minus Time 1 defective conditional index) against their Time 1 RAPM and CRT scores, their prior academic attainment, their RAPM and CRT change scores (the difference between their Time 2 and Time 1 scores), the group they were in, and the two group by change-score interaction terms. If the increased defective conditional indices of the mathematics students could be accounted for by changes in domain general factors, we would expect that some of the change scores or the group by change-score interactions would be significant predictors. However, if the primary factor was the experience of studying mathematics, we would expect the group factor to be the only significant predictor.

The regression model is presented in Table 4. The only significant predictor of change in defective conditional scores was Group, β = .337. None of the change scores, nor the change by group interactions approached significance. This analysis seems to suggest that the change in conditional reasoning behavior in the mathematics group is most likely to be related to experiences gained in their mathematical study, not to domain-general changes in cognitive capacity or thinking disposition.