Sample size calculations are an integral part of scientifically useful and ethical research [1]. A study which is too small may not answer the research question, wasting resources and potentially putting participants at risk for no purpose [2]. Studies which are too large can also waste resources and expose participants to the potential harms of research needlessly, as well as delaying results and their translation into practice. The computation of sample size a priori is usually dependent upon predetermined values for power and level of significance, an estimate of the expected variability in the sample and an effect size of practical or clinical importance. By convention, the choice of power and level of significance is usually at least and no more than respectively. When a practically important effect size is unknown, there are several recommendations in the literature to guide the researcher. In his seminal paper, Cohen [3] gives operationally defined small, medium and large effect sizes for various, common significance tests. The use of effect size recommendations should not replace differences of clinical or practical importance [4] and may not be appropriate for all disciplines. In basic science research, for example, large effect sizes by Cohen's criteria are common and, therefore, require small sample sizes. On the other hand, clinical and epidemiological research often deals with small effect sizes and often requires large, population-based studies. While there are some approaches to estimating a minimum important effect [5], there are instances where this information is simply not known. Thus, effect size recommendations assist with the balance between overly small and overly large sample sizes.

When the researcher is interested in contingency tables, a common measure of effect size is which, in this instance, is equivalent to Pearson's correlation coefficient [6]. Cohen [3] recommends effect sizes of and for small, medium and large effect sizes respectively and are identical to his recommendations for the correlation coefficient. Although Cohen [3] denotes this statistic as , much of the literature uses [6]–[10] and the remainder of this manuscript follows this convention. To support his recommended effect sizes for correlation coefficients, Cohen [11] chose equivalent values for the difference in two means through the connection with point biserial correlation. Additionally, is applicable to logistic regression since it can be converted to an odds ratio () when the row (or column) marginal probabilities of the table are fixed. For example, when the marginal probabilities are uniform (i.e., for row and column probabilities), Cohen's recommended effect sizes are equivalent to odds ratios of and . It will be demonstrated that the connection between the odds ratio and is largely dependent on the marginal probabilities and these values should not be used in general.

A problem arises when using the effect size for tables as the full range of correlation coefficients are only possible under very restrictive circumstances and are not justified in general [12]. On the other hand, odds ratios are valid effect size measures that are not constrained by the marginal probabilities. Ferguson [10] recommends small, medium, and large odds ratio effect sizes of and , but urges caution in their use as they are not “anchored” to Pearson's correlation coefficient. Although many have pointed out problems with as an association measure and advocate the use of odds ratios as an alternative, effect size recommendations for odds ratios do not exist in general.

It is common in randomised controlled trials and case-control studies to fix one of the marginal probabilities in the table as it directly relates to the ratio of participant allocation. For instance, a marginal probability of corresponds to a 1:1 case-control ratio while a 2:1 ratio is a marginal probability of (or equivalently for 1:2).

The aims of this paper are to demonstrate: (1) the equivalence of effect size measures for contingency tables, in particular the relationship between and the odds ratio; (2) that recommended odds ratio effect sizes can be derived from Cohen's work using the maximum value of as a guideline for fixed marginal probabilities; (3) the shortcomings of and the strength of the odds ratio as an effect size measure; and (4) that conservative odds ratio effect size recommendations can be derived without relying on fixed margins. We provide an example that investigates the association between helmet wearing by bicyclists and overtaking distance by automobiles.

When the marginal probabilities are fixed constants, is an increasing linear function of . Further, is bounded by

These bounds are due to all cell probabilities being non-negative and the relationship of with the other cell probabilities given above. As a result, is bounded as well and attains its maximum when . Using the upper bound of the above inequality, it can be shown thatwhere to ensure . It is clear from the formula for that the full range of correlation coefficients, i.e., , is attainable only when the marginal probabilities are equal, i.e., or . This has an intuitive appeal as perfect correlation for two binary variables is only possible when two cell probabilities are zero. For example, when all observations are in either the or cells, . However, it would appear highly unlikely both marginal probabilities will be equal in practice. For example, in a 1:1 case-control study with mortality as the primary outcome, half of all patients would need to die for perfect correlation to be possible. On the other hand, if of all patients die, the maximum correlation possible is which is near a medium recommended effect size. So, in this situation, all estimates of , computed from observed proportions, are bounded by

Importantly, odds ratios are not bounded with possible values of as varies on the interval . In fact, as approaches , the increases without bound. Figure 1 demonstrates this relationship. Importantly, this indicates has serious limitations as a measure of association and that these limitations are not applicable to the odds ratio.

In many practical instances, the marginal probabilities are not equal, making the full range of values for impossible with the potential of making Cohen's recommended effect sizes unusable for tables. Although not equivalent to perfect correlation, can be interpreted as the maximum possible correlation given the marginal probabilities. In fact, / has been proposed as an association measure with the interpretation as the proportion of observed correlation relative to the maximum attainable with fixed marginal probabilities [7], although the researcher is cautioned when the marginal probabilities diverge [6]. Note that is not equivalent to Cohen's similarity/agreement measure . However, suffers from the same boundary problems as and the two are equivalent when scaled to their maximum values, i.e., //, making the two measures similar [6].

This paper was motivated by a reanalysis of passing distances for motor vehicles overtaking a bicyclist [16]. One of the primary results of this study was a significant association between helmet wearing and less overtaking distance, supporting a theory of risk perception for motor vehicle drivers directed towards bicyclists. Prior to collecting data, Walker [16] reported computing a sample size of overtaking manoeuvres based on a fixed effects factorial ANOVA for a small effect size , level of significance and power. The factors for this study were helmet wearing (2 levels) and bicycle position relative to the kerb (5 levels). It has been noted, however, that passing distances are often recommended and sometimes legislated to one metre or more [17]. So, passing manoeuvres of at least a metre are considered safe and less than a metre unsafe, with the implication that large differences in passing distance are unimportant beyond one metre in terms of bicycle safety. When compared with helmet wearing, safe/unsafe passing distances can be analysed using a table. Since Walker's study was powered at an unusually high level with subsequent increased probability of a type I error, bootstrap standard errors were estimated for more reasonable values for power of , and . Operationally defined small, medium and large effect sizes were also used since a meaningful difference in overtaking distance is unknown.

The relevant observed data from Walker [16] is given in Table 3. The observed marginal proportions here are for helmet wearing and for unsafe passing manoeuvres. Using the marginal probabilities, the maximum attainable effect size is and the estimated correlation is . A consequence is the effect size for the association between helmet wearing and safe passing distance is, at best, much less than a small effect size by Cohen's index. The corresponding small, medium and large odds ratio effect sizes using increments of are and for and . Note that these values are not much greater than the minimal recommended odds ratios mentioned in the previous section, further suggesting the association between safe/unsafe passing distance and helmet wearing is, at best, a small effect size. In fact, the unadjusted odds ratio is and non-significant by the chi-square test (). Conversely, sample sizes for a future study can be computed from the observed probabilities using G*Power for logistic regression with a single binomially distributed predictor for and power [18] resulting in and observations for small, medium and large odds ratios. To put these sample size computations into perspective, a future study would need to extend the sampling period by a factor greater than seven to detect a significant association between helmet wearing and safe/unsafe overtaking distance given a small effect size and identical marginal probabilities.

We present a demonstration that many contingency table correlation measures are equivalent for the case and their use is limited due to constraints created by fixed marginal probabilities. The odds ratio, which is a function of these measures for fixed marginal probabilities, is not problematic, is regularly used in statistical analyses and has a direct application to logistic regression. Recommended odds ratios have been proposed from Cohen's small, medium and large effect sizes for relative to the maximum attainable correlation . Further, minimal odds ratios can be computed with only knowledge of participant allocation.

The use of effect size recommendations should be avoided in situations in which clinical or practical differences are known. However, they can help the researcher balance between overly large or overly small sample size calculations when such information is unknown. In these situations, conservative estimates for odds ratio effect sizes can be derived from only the allocation ratio leading to a general result and, when a 1:1 allocation is chosen for optimal power, odds ratios of and correspond to small, medium and large effect sizes.