PLoS BiolplosplosbiolPLOS Biology1544-91731545-7885Public Library of ScienceSan Francisco, CA USA10.1371/journal.pbio.3002129PBIOLOGY-D-23-00262Meta-Research ArticlePhysical sciencesMathematicsStatisticsStatistical dataScience policyScience and technology workforceCareers in researchScientistsPeople and placesPopulation groupingsProfessionsScientistsResearch and analysis methodsResearch designExperimental designFactorial designResearch and analysis methodsSimulation and modelingResearch and analysis methodsResearch designComputer and information sciencesData managementData visualizationInfographicsGraphsResearch and analysis methodsMathematical and statistical techniquesStatistical methodsPhysical sciencesMathematicsStatisticsStatistical methodsPhysical sciencesMathematicsProbability theoryProbability distributionNormal distributionStatistical simulations show that scientists need not increase overall sample size by default when including both sexes in in vivo studiesScientists should not increase the overall n by default when including both sexes in in vivo studieshttps://orcid.org/0000-0001-9161-8903PhillipsBenjaminConceptualizationFormal analysisVisualizationWriting – original draftWriting – review & editing1HaschlerTimo N.Data curationWriting – review & editing2https://orcid.org/0000-0002-8404-2907KarpNatasha A.ConceptualizationFormal analysisProject administrationWriting – original draftWriting – review & editing1*Data Sciences & Quantitative Biology, Discovery Sciences, R&D, AstraZeneca, Cambridge, United KingdomBioscience Renal, Research and Early Development Cardiovascular, Renal and Metabolism, Biopharmaceutical R&D, AstraZeneca, Cambridge, United KingdomMunafòMarcusAcademic EditorUniversity of Bristol, UNITED KINGDOM
I have read the journal’s policy and the authors of this manuscript have the following competing interests: BP, TH, and NAK have shareholdings in AstraZeneca.
* E-mail: natasha.karp@astrazeneca.com86202362023216e3002129301202318420232023Phillips et alThis is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
In recent years, there has been a strong drive to improve the inclusion of animals of both sexes in the design of in vivo research studies, driven by a need to increase sex representation in fundamental biology and drug development. This has resulted in inclusion mandates by funding bodies and journals, alongside numerous published manuscripts highlighting the issue and providing guidance to scientists. However, progress is slow and barriers to the routine use of both sexes remain. A frequent, major concern is the perceived need for a higher overall sample size to achieve an equivalent level of statistical power, which would result in an increased ethical and resource burden. This perception arises from either the belief that sex inclusion will increase variability in the data (either through a baseline difference or a treatment effect that depends on sex), thus reducing the sensitivity of statistical tests, or from misapprehensions about the correct way to analyse the data, including disaggregation or pooling by sex. Here, we conduct an in-depth examination of the consequences of including both sexes on statistical power. We performed simulations by constructing artificial datasets that encompass a range of outcomes that may occur in studies studying a treatment effect in the context of both sexes. This includes both baseline sex differences and situations in which the size of the treatment effect depends on sex in both the same and opposite directions. The data were then analysed using either a factorial analysis approach, which is appropriate for the design, or a t test approach following pooling or disaggregation of the data, which are common but erroneous strategies. The results demonstrate that there is no loss of power to detect treatment effects when splitting the sample size across sexes in most scenarios, providing that the data are analysed using an appropriate factorial analysis method (e.g., two-way ANOVA). In the rare situations where power is lost, the benefit of understanding the role of sex outweighs the power considerations. Additionally, use of the inappropriate analysis pipelines results in a loss of statistical power. Therefore, we recommend analysing data collected from both sexes using factorial analysis and splitting the sample size across male and female mice as a standard strategy.
Many scientists believe that studying both sexes by default requires an increase in the number of animals needed; is this correct? This study provides an in-depth examination of the topic through simulations to assess the reality and truly answer the question.
The authors received no specific funding for this work.PLOS Publication Stagevor-update-to-uncorrected-proofPublication Update2023-06-21Data AvailabilityThe R scripts used to generate the data and figures and analyse the in vivo data have been made available as a Zenodo repository (https://doi.org/10.5281/zenodo.7806724).Introduction
There has been a bias towards using a single sex in in vivo research. Though there is variation between subdisciplines, this strategy has tended to result in a heavy bias towards male animals. For example, in 2009, only 26% of studies used both sexes and among the remainder there was a male bias in 80% of studies [1]. The negative consequences of these shortcomings on scientific enterprise are beginning to be better understood as evidence emerges that our current fundamental biological knowledge base may be biased. For example, a recent report concluded that the fundamental molecular basis of pain is highly sex dimorphic, yet much of our knowledge in this area is derived from studies solely using male organisms [2]. This situation risks generating a knowledge imbalance that persists through the research pipeline, ultimately manifesting in the clinic.
To improve the translation of results from animals to humans, there has been a push to include both male and female animals in in vivo studies. For example, numerous funding bodies, including the NIH in the United States and the MRC in the United Kingdom, now have inclusion mandates. These policies do not require scientists to study differences between males and females per se, but instead aim to improve the generalisability of studies by calculating an average effect estimated from both sexes. If, however, there is a large, meaningful sex difference in the treatment effect, studies should be designed in such a way that the visualisation and analysis detects it [3,4]. The NIH policy introduced the term Sex as a Biological Variable (SABV), and, here, we use the term to represent a sex inclusive research philosophy that emphasises the importance of automatic inclusion, with a focus on treatment effect estimates. Any of a wide range of factors including animal strain, age, health status, or others could also be the focus of a campaign to improve research generalisability. However, sex is a particularly pressing and timely direction for improved representation since females account for such a large proportion of the population of interest but are currently largely overlooked.
Over time, the proportion of studies including both sexes has improved [5,6], with one study estimating an increase between 2009 and 2019 from 26% to 48% of studies [6]. Scientists tend to be supportive of efforts to improve sex representation in in vivo research [7]. Unfortunately, in studies where both sexes are tested, a large proportion commit errors at the statistical analysis stage [8]. Thus, despite an overall increase in inclusion, the proportion of studies appropriately interpreting the influence of sex is still low [5]. Overall, the pace of change is slow, owing to a persistent and broad range of perceived statistical and practical barriers. Consequently, scientists believe that including both sexes will introduce a significant ethical, practical, and financial burden [9]. The barriers include now debunked beliefs that female animals produce more variable data [10–12], institution-level ingrained cultural belief about the value of studying 2 sexes [13,14], and a skill-gap in handling data collected under factorial designs [8]. There is also a general belief that it is necessary to greatly increase the experimental sample size (N) when investigating treatment effects in 2 sexes [14,15]. For example, a recent report cites that 27% of published papers justified a single sex approach due to concerns around experimental variability [6]. Though this misconception has been addressed previously [11,16], and guidance on appropriate analysis exists [17,18], it remains widespread, and there is a need for a deeper exploration of the impact of including both sexes on statistical power. Revisiting this is critical to enable the community to address this significant barrier to sex-inclusive research due to the misguided belief that there is a trade-off between pursuing the 3Rs (replacement, reduction, refinement) by means of reducing animal usage on the one hand and designing more generalisable studies on the other [14].
When considering sex-inclusive research, the following misconceptions and data analysis errors have been reported:
Misconception 1: Designs that include both sexes will require a doubling of sample size to achieve the same power [9,14,19].
Misconception 2: Belief that the possibility of sex effects (either a baseline differences or a treatment effect that depends on sex) will increase variability and consequently require an increased N to maintain the power [11,12].
Error 1: Inappropriate pooling of male and female data for a treatment group (i.e., combining the data from both sexes and ignoring sex as a factor in the analysis) [8].
Error 2: Disaggregation of the data by sex and independent statistical comparison between the control and treated group. Then, comparing the p-values from the independent tests [8,20].
Error 3: Incorrect groups in statistical comparison: comparison of treated males and treated females [6].
Of these, the misconceptions (1 and 2) contain empirical claims, and for these we have constructed a range of simulated datasets to extensively test statistical power for a range of plausible biological scenarios where both sexes are tested and subsequently analysed by an appropriate factorial pipeline. The consequences of pooling male and female data and disaggregating the data by sex (error numbers 1 and 2) are additionally demonstrated as part of these simulations. A case study example analysis (S1) explores error 3.
Through simulations, we have conducted an in-depth examination of the impact of sex inclusion on statistical power for a variety of commonly implemented analysis strategies. Our methodology has been designed to demonstrate the problems that result from using the wrong analysis strategies and address common misconceptions around power when using factorial designs and analysis. Our results demonstrate that including both male and female animals does not reduce statistical power across a wide range of outcomes when investigating a treatment effect. We demonstrate that power loss is (a) rare and (b) indicative of a sex dimorphism that it would be important to be aware of. Our comparison of statistical analysis strategies demonstrates that inappropriate methods, including both pooling the data from males and females and disaggregation of the data by sex result in a loss of statistical power. Therefore, the importance of adopting a factorial analysis method is central to appropriately analyse data from studies testing a treatment effect in both sexes. To support the adoption of appropriate analysis, we provide a case study (S1 Case Study) demonstrating an example pipeline for analysing data collected under such designs, intended as a practical guide for scientists. A compendium of common statistical terms used within this manuscript is also included as a guide to readers (see Box 1 Glossary).
Results and discussionIntroduction to simulations to explore the impact of sex-related biological differences on statistical power
It is often believed that the inclusion of both sexes increases the sample size needed to detect an equivalently sized treatment effect. We have systematically investigated this claim by the construction of simulated datasets to represent sex-inclusive designs where sex can introduce variance, either as a baseline difference between the sexes or as a treatment effect that is dependent on sex, to assess the impact on power. The objective of these simulations is to understand general trends and relationships and consequently the exact values are not relevant but represent typical animal experiments. They are constructed to support inclusive designs for research programmes in which sex is not the primary question of interest, but where the researcher seeks to understand the generalisability of a treatment effect, thus enabling inference about the effect on both sexes.
In the following simulations, we construct datasets with defined differences in the size of main effects (sex and treatment) and interactions. By repeating this construction process multiple times and testing the resultant data, we can calculate the statistical power of each scenario by returning the proportion of times a statistically significant result occurs (Fig 1). The datasets were analysed with either a factorial pipeline (Fig 2) or a pooled analysis pipeline. The post hoc pairwise statistical tests (e.g., a statistical comparison between untreated and treated animals of each sex), typically implemented when a significant interaction is present, is statistically equivalent to a disaggregation strategy where the data is split by sex and subjected to 2 independent Student’s t tests.
10.1371/journal.pbio.3002129.g001
Power analysis by simulation an explainer.
10.1371/journal.pbio.3002129.g002
An explainer on factorial design and analysis.
Scenario 1: Impact on statistical power of a baseline sex difference
In the first scenario, we tested the effect of increasing baseline sex difference (none = 0, small = 0.5, large = 1 added to the males) on the ability to detect a main effect of treatment (effect size: 0 to 1 added to both sexes) (Fig 3A). This is representative of the most common situation observed in studies run including both sexes [21]. As the effect size of the baseline sex differences increased, there was no change in statistical power when a factorial statistical method was adopted (Fig 3B). However, there was a loss in power when the data were pooled and a Student’s t test applied (error 1) (Fig 3B). This is because factorial analysis is accounting for variability in the data that arose from a baseline difference in the sexes, whereas the effect is not accounted for in the pooled analysis. This highlights the error that is committed when data from both sexes are pooled, resulting in significant loss of power to detect treatment effects.
10.1371/journal.pbio.3002129.g003
Results of simulations to explore impact of a baseline sex differences on statistical power.
(A) Illustrative plots of constructed datasets, ranging from zero to maximum baseline sex differences and treatment effect. (B) Statistical power from both a factorial and pooled analysis to detect a main effect of treatment when the main effect of sex varies (none = 0, small = 0.5, large = 1). (C) Power for each model term within factorial analysis output. (D) Power for post hoc comparison of control vs. treated within each sex independently. Simulation N = 1,000 for each scenario assessed. For the graphs of power (B–D), the horizontal dashed line indicates the target power. The data underlying this figure can be found in https://doi.org/10.5281/zenodo.7806724.
As the treatment effect was added to both sexes equally no biological interaction occurred. Consequently, there was no power to detect an interaction effect across the range of treatment and sex main effects (Fig 3C). The power to detect a baseline sex difference increased as the effect size increased (Fig 3C). The power for post hoc tests increased equivalently for both sexes as the overall treatment effect increased (Fig 3D). Notably, since the power for the post hoc treatment test is equivalent to the power of analysing the 2 sexes independently, the loss of power when disaggregating the data is evident from comparison of Fig 3D and 3B. This loss of power is a concern because the interaction was not significant, and the treatment effect should be evaluated as a main effect. Furthermore, where there is no baseline sex effect in this scenario, these simulations show that the power to detect a treatment effect is identical between the factorial and pooled strategy (Fig 3B). Often scientists justify the need to pool data to maximise sensitivity arguing that the simpler model would be more powerful when sex does not explain variation. However, these simulations found no statistical cost to using the more complex analysis strategy when there is no variance attributable to sex.
In summary, there is no loss of power to detect a treatment effect when there is a baseline sex effect in the data provided data is not disaggregated.
Scenario 2: Impact on statistical power when there is a treatment effect of different sizes for the 2 sexes
To understand the impact of an interaction between treatment and sex on the power, where the interaction was in the same direction as a main effect of treatment, datasets were constructed to allow an exploration of the impact of an increasing interaction effect (none = 0, small = 0.5, large = 1) in the presence of a main effect of treatment (varied from 0 to 1) (Fig 4A, all graphs). There was no baseline sex difference in the data. In the factorial pipeline, as the treatment by sex interaction increased (Fig 4A, bottom left and bottom right), the power to detect a main effect of treatment increased in line with the size of the interaction effect (Fig 4B). There was also an increase in power to detect the treatment by sex interaction as the interaction effect size increased (Fig 4C). In the post hoc tests, the power was highest in the sex that exhibited the largest treatment effect, which increased as a function of effect size (Fig 4D).
10.1371/journal.pbio.3002129.g004
The impact of a sex-dependent treatment effect on statistical power.
Results of simulations to calculate statistical power where there is an interaction between treatment and sex in the data where the interaction effect is in the same direction as the main treatment effect. (A) Illustrative plots of constructed datasets, ranging from no effect to maximum treatment by sex interaction (none = 0, small = 0.5, large = 1) for varying sizes of a main treatment effect (0–1). (B) Statistical power to detect treatment effect per size of interaction effect and statistical method. (C) Power for each model term within factorial analysis output. (D) Power for post hoc comparison control vs. treated within each sex. Simulation N = 1,000 for each scenario assessed. For the graphs of power (B–D), the horizontal dashed line indicates the target power. The data underlying this figure can be found in https://doi.org/10.5281/zenodo.7806724.
In the pooled analysis pipeline, as the main treatment effect increased the power increased (Fig 4A, top right and bottom right). However, as the interaction size increased, the pooled statistical pipeline had lower power to detect the main effect of treatment compared to the factorial pipeline (Fig 4B), as there was an increase in sex-related variance not accounted for. This analysis aligns with the belief that sex-related variance reduces statistical power to detect a treatment effect but is arising from inappropriate analysis.
The factorial pipeline first simultaneously tests the main and interaction effects. If the interaction effect is statistically significant and of biological interest, then a subsequent post hoc testing step allows an assessment of treatment effect independently for each sex. If researchers follow a disaggregation pipeline, where data is automatically disaggregated by sex and Student’s t tests conducted on each separately, there is no ability to assess for a treatment by sex interaction. Often researchers will compare the calls of significance to assess for a differential treatment effect (error 2). However, it is a statistical error to compare 2 independent statistical tests (e.g., a significant and nonsignificant result) and claim a difference on this basis [22–24]. Additionally, disaggregation of the data will reduce the power for the treatment effect (comparison of Fig 4B and 4D). Thus, a factorial analysis is required to properly implement the SABV philosophy and to avoid misinterpretation of the data.
In summary, a factorial pipeline is essential to maintain power and support a claim of a differential sex effect.
Scenario 3: Impact on statistical power when there is a treatment effect in 1 sex only
An extreme example of a treatment by sex interaction is when there is a treatment effect in 1 sex only (Fig 5A). As the interaction size increased, statistical power for the treatment effect was higher in the factorial pipeline compared to the pooled pipeline (Fig 5B). The lower power for the pooled pipeline arises as the between-sex variance was not accounted for. In this scenario, the power to detect the main effects of sex and treatment and the treatment by sex interaction were identical as the interaction effect size increased (Fig 5C). There was only power to detect a post hoc effect in the sex that exhibited the treatment effect, which increased as the interaction effect size increased (Fig 5D).
10.1371/journal.pbio.3002129.g005
The impact of a treatment effect specific to 1 sex on statistical power.
Results of simulations to calculate statistical power where there is an interaction driven by a treatment effect in 1 sex only. (A) Illustrative plots of constructed datasets, ranging from a zero to maximum treatment by sex interaction effect (0–2). (B) Statistical power to detect the main effect of treatment effect as a function of the interaction effect size for each statistical method. (C) Power for each model term within factorial analysis output. (D) Power for post hoc comparison control vs. treated within each sex. (E) Power for contrasting design strategies—10 males vs. 5 females and 5 males. Simulation N = 1,000 for each scenario assessed. For the graphs of power (B–E), the horizontal dashed line indicates the target power. The data underlying this figure can be found in https://doi.org/10.5281/zenodo.7806724.
For this biological situation, where the treatment effect is unique to one sex, an additional simulation was run (Fig 5E) to compare the power when a single sex was used versus an inclusive design where the N was split across the 2 sexes. The simulations confirm that there was a loss of power to using a sex-inclusive design compared to a design where the sex exhibiting the treatment effect was selected and studied in isolation (Fig 5E). In this rare scenario [21], we would argue that the conclusions obtained by studying only a single sex would be skewed. If the sex showing the treatment effect was included in the study, the researcher may have mistakenly assumed that the conclusion generalised to the entire population. Conversely, if the sex displaying low sensitivity to treatment were included, the researchers would have erroneously concluded that the treatment was ineffective.
In summary, when the treatment effect is unique to 1 sex, the simulations show there is a power benefit when a factorial pipeline is adopted compared to pooled pipeline. In comparison to a disaggregated strategy where the “right” sex had been selected, there is a reduction in power. However, our knowledge of the treatment effect is increased and unbiased when using an inclusive design with a factorial analysis.
Scenario 4: Impact on statistical power when there is an interaction driven by opposite sex effects
Though rare biologically [21], it is possible a treatment may produce an opposite effect between sexes (Fig 6A). For both the factorial and the pooled pipelines, the simulations show the statistical power for the main effect of treatment was low as expected (Fig 6B) because the mean effect of the treatment is zero as the treatment effect in each sex increases equivalently in opposite directions. The factorial analysis produced the expected false call rate (proportion of the time that a significant result is detected when there is no true difference) for the 0.05 significance threshold of approximately 5% for all effect sizes (Fig 6B). For the pooled pipeline, however, the simulations found that the false positive rate decreased as the size of the interaction effect increased (Fig 6B). The conservative behaviour of the pooling strategy arises from the influence of unpartitioned variance that is attributable to sex.
10.1371/journal.pbio.3002129.g006
The impact of an opposite treatment effect on statistical power.
Results of simulations to calculate statistical power where there is an interaction driven by opposite treatment effect in each sex. (A) Illustrative plots of constructed datasets, ranging from no effect to maximum opposite effect in each sex (0–1 male, 0 to −1 female). (B) Statistical power to detect the main effect of treatment as a function of the interaction effect size for each statistical method. (C) Power for each model term within factorial analysis output. (D) Power for post hoc comparison control vs. treated within each sex. Simulation N = 1,000 for each scenario assessed. For the graphs of power (B–D), the horizontal dashed line indicates the target power. The data underlying this figure can be found in https://doi.org/10.5281/zenodo.7806724.
In the factorial analysis, power for the additional terms (sex and interaction) were equivalent and increased as the effect size increased (Fig 6C). In these situations, where you have a significant interaction but not main effect, the simulations demonstrate that the post hoc tests have high power in each sex (Fig 6D).
In summary, in this rare situation where there is no discernible main effect, but an interaction is significant, the post hoc tests are a crucial step to evaluate the effect. While the power is low for the main effect in this scenario, the power for the interaction is high which will trigger the correct interpretation. The pooled pipeline would fail to discern this critical biological insight. A disaggregated pipeline would have equivalent power but would not have a statistical test to support the conclusion that the effect depended on sex.
The power of the factorial design, where the N is split across the 2 sexes is lower than a single sex design using the same N. However, the inclusion of only a single sex in this scenario would have resulted in a highly biased conclusion that should not be generalised to the broader population of interest. A strong opposite effect would merit follow-up studies to understand the basis of the differential treatment effect.
Conclusions
The SABV philosophy is to include both sexes to allow a generalisable estimate with the potential to detect if there is a major difference in treatment response between sexes. This inclusive design, when combined with a factorial analysis, allows us to statistically test for whether the treatment effect depends on sex. This is driven by a desire to increase the translational confidence of the results and does not require experiments to be powered to detect treatment by sex effects. Our simulations show that for most biologically expected situations (where treatment effects are similar across the sexes or there is a baseline sex difference), there is no need to increase the N needed in the studies, rather the intended N can be shared across the 2 sexes for a treatment. This strategy implies that power calculations for the treatment effect can be simplified to a 2-group comparison to estimate the total N needed for a treatment, which is then shared between the 2 sexes. Alternatively, power calculations for factorial designs can be calculated using other methods (e.g., the Superpower package [25]). When there is a small difference in the size of the treatment by sex interaction, estimating the average effect is ideal for translational understanding as this is a generalisable conclusion. The simulations did demonstrate a loss of power when there is a large treatment by sex interaction (e.g., opposite or the effect only occurs in 1 sex). In this uncommon scenario and where the N has been split across the 2 sexes, we may fail to detect a significant treatment effect (due to the lower power) but would gain important knowledge suggestive of a large sex dimorphism. Taken together, these conclusions support the recommendation to split the intended N across the 2 sexes.
Our simulations additionally reveal the negative consequences of erroneously pooling or disaggregating the data by sex for analysis. For pooling, our results show that there is a loss of power to detect a treatment effect across most scenarios, including where there is a baseline sex difference and when there is a treatment by sex interaction. Pooling the data by sex also necessarily precludes identification of sex-specific effects, thus important biological knowledge would be lost in these scenarios. For disaggregation, when both sexes display a treatment effect, there is less power to detect it in each sex independently than via the main treatment effect term of a factorial analysis. Moreover, disaggregating precludes the detection of interaction effects, thus losing the ability to statistically assert a differential treatment by sex effect.
There are limitations of the simulations that we have carried out, and consequently, the conclusions we have reached in this manuscript. First, the simulations have been conducted on typical research scenarios aimed at determining a change in means between 2 groups and model continuous data with a normal distribution, equal variance, independent observations, and a balanced design. Where the goal of the study is not to test a mean difference in this context, our conclusions may not apply. The conclusions may not extend to a situation where the n is extremely low, and this may also preclude a halving strategy (e.g., halving of 3 per treatment groups is unfeasible). Our investigations are also conducted in the context of a null hypothesis significance testing scenario where a decision is based on evaluating the p-value against a threshold (most typically p < 0.05). This is an area of significant ongoing discussion and debate [26]. However, given the current prevalence of p-values and their necessity for standard power calculations, we would argue that this limitation has minimal impact. Critically, the scope of this manuscript is limited to studies where the intention is to estimate a generalisable treatment effect, rather than exploring the dependence of a treatment effect on sex.
To facilitate progress in sex inclusion in in vivo research, it is crucial to provide scientists with both evidence and practical resources that challenge the barriers that currently stand in the way of change. This manuscript provides an in-depth analysis to explore the topic of power when both sexes are included to address the barrier that is the belief that inclusion requires an increase in the sample size. In this analysis, we have performed extensive statistical simulations to evaluate power under a range of common biological scenarios when splitting the N across 2 sexes. Critically, we did not identify any common scenarios that result in a loss of power to detect treatment effects. Rarely, large interactions in the data may produce an appreciable decrease in treatment effect power. In these scenarios, we would argue the knowledge gained that the treatment has a differential impact between sexes outweighs the statistical loss of power. Furthermore, if a disease affects both sexes but the effect in the research model is observed in only one, this may bring into question the validity of the model or the generalisability of the treatment. The simulations also demonstrate the pitfalls of some frequent analysis mistakes, including the inappropriate pooling and disaggregation of data collected from 2 sexes, which result in a loss of power to detect a treatment effect compared to a factorial analysis applied to the same data.
Additionally, we provide an example pipeline for analysing data collected from both sexes as a practical guide for scientists (S1 Case Study). The approaches above heavily depend upon the appropriate application of the correct factorial analysis methods, and it is therefore critical that laboratory scientists receive focused support in developing their statistical capabilities.
MethodsEthics statement
All animal experiments were conducted in accordance with the United Kingdom Animal (Scientific Procedures) Act 1986 and associated guidelines, approved by institutional ethical review committees (Alderley Park Animal Welfare and Ethical Review Board; Babraham Institute Animal Welfare and Ethical Review Board) and conducted under the authority of the Home Office Project Licences (PF344F0A0). All animal facilities have been approved by the United Kingdom Home Office Licensing Authority and meet all current regulations and standards of the United Kingdom.
Statistical simulations: Dataset construction
To explore the impact of sex either as a main effect (baseline sex differences) or when it interacts with the treatment on power, simulation studies were conducted. In the simulations, representative datasets with 5 animals per treatment group per sex were constructed by randomly sampling from a normal distribution after defining the mean and standard deviation of each treatment group. This process was repeated multiple times for each scenario of interest (N = 1,000) that enabled the subsequent evaluation of statistical power for each analysis pipeline of interest. We moved stepwise through 4 scenarios encompassing possible outcomes from studies testing a treatment effect in both sexes. Thus, the simulations differed by altering the specified means in each group (e.g., baseline sex difference, treatment by sex interaction) (see Table 1).
10.1371/journal.pbio.3002129.t001
Details of how the simulation datasets were constructed to represent various biological situations.
Simulation
Objective
Main effect of sex
Main effect of treatment
Interaction between the treatment and sex
1
Impact of a baseline sex difference on statistical power
VariedSignal added to means of males: none (0), small (0.5), or large (1)
VariedSignal added to the treatment group mean: 0–1 increments of 0.1
None
2
Impact of an interaction between sex and treatment on statistical power
None
VariedSignal added to the treatment group mean0–1 increments of 0.1
VariedSignal added to the mean of the treated malesNone (0), small (0.5), or large (1)
3
There is a treatment effect in 1 sex only
None
None
VariedSignal added to males only: 0–2 in increments of 0.2
4
There is an interaction resulting from a treatment effect that is opposite across the sexes
None
None
VariedSignal added to both sexes symmetrically in opposite directionMale: 0–1 increments of 0.1Female: 0 to −1 increments of 0.1
Data was sampled from a normal distribution where the baseline characteristics were a mean of 1 and a variance of 0.5.
Statistical simulations: Statistical analysis
The constructed datasets were statistically analysed either using a factorial pipeline or a pooled pipeline.
In the factorial pipeline, a regression analysis in R equivalent to a two-way ANOVA provided an assessment of the main effect of treatment, sex, and the interaction of treatment by sex. This was followed by a set of uncorrected post hoc pairwise tests between untreated and treated data in each sex using the R package emmeans. Sex is an unusual biological factor, as it depends on mendelian randomisation rather than experimenter randomisation in the allocation. The scenario we are considering, where both sex and treatment are included in the experimental design, may also be referred to as a stratified design, particularly in the clinical literature [27]. Here, we use the term factorial throughout as this is the typical terminology used by the biological community.
In the pooled pipeline, a Student’s t test, after combining the data across the sexes for each treatment level, was conducted.
Statistical simulations: Assessment of statistical power
The resulting datasets were analysed using both the factorial and pooled pipelines, and statistical power was defined as the proportion of the time a statistically significant effect was called for the model term of interest, at a significance threshold p < 0.05.
Box 1. Glossary.
Common statistical terms used within this manuscript, adapted from [28] and placed in the context of in vivo research are detailed below:
Effect size: Quantitative measure of differences between groups or strength of relationships between variables.
Factor: Factors are independent categorical variables that the experimenter controls during an experiment in order to determine their effect on the outcome variable. Example factors include sex or treatment.
Factorial design: An experimental design that is used to study 2 or more factors, each with multiple discrete possible values or levels.
Independent variable: A variable that either the experimenter controls (e.g., treatment given or dose) or is a property of the sample (sex) or a technical feature (e.g., batch or cage) that can potentially affect the outcome variable.
Interaction effect: When the effect of one independent variable (factor) depends on the level of another. For example, the observed treatment effect depends on the sex of the animals.
Levels: Are the values that the factor can take. For example, for the factor sex the levels are male and female.
Main effect: A main effect is the overall effect of one independent variable on the outcome variable averaging across the levels of the other independent variable.
Outcome variable: A variable captured during a study to assess the effects of a treatment. Also known as dependent variable or response variable.
Power: For a predefined, biologically meaningful effect size, the probability that the statistical test will detect the effect if it exists (i.e., the null hypothesis is rejected correctly). Can also be called sensitivity.
Treatment: A process or action that is the focus of the experiment. For example, a drug treatment or a genetic modification.
Sex as a biological variable (SABV): The research philosophy that emphasises the importance of including both sexes in in vivo studies in such a way that a generalisable treatment effect is detectable. Critically, sex should be treated as a variable of primary biological interest. There is no requirement to prospectively power a study to detect a baseline difference between the sexes or treatment by sex interaction, but studies will detect large differences where they exist.
Supporting information
Example application of factorial analysis for data collected from 2 sexes.
This supplementary case study is intended as an example pipeline for analysing data from in vivo experiments collected from both sexes. It is not intended as an exhaustive tutorial, and there are other appropriate methods for analysing the type of data that we are presenting.
(DOCX)
We are grateful to Lorraine Miller for technical assistance with the experimental work and Chris Heath and Esther Pearl for helpful comments on earlier drafts of the manuscript.
ReferencesBeeryAK, ZuckerI. Sex bias in neuroscience and biomedical research. Neurosci Biobehav Rev. 2011Jan;35(3):565–572. doi: 10.1016/j.neubiorev.2010.07.00220620164MogilJS. Qualitative sex differences in pain processing: emerging evidence of a biased literature.Nat Rev Neurosci. 2020Jul;21(7):353–365. doi: 10.1038/s41583-020-0310-632440016ArnegardME, WhittenLA, HunterC, ClaytonJA. Sex as a Biological Variable: A 5-Year Progress Report and Call to Action.J Womens Health (Larchmt).2020Jun;29(6):858–864. doi: 10.1089/jwh.2019.824731971851Sex in experimental design–MRC [Internet]. [cited 2023 Jan 23]. Available from: https://www.ukri.org/councils/mrc/guidance-for-applicants/policies-and-guidance-for-researchers/sex-in-experimental-design/.RechlinRK, SplinterTFL, HodgesTE, AlbertAY, GaleaLAM. An analysis of neuroscience and psychiatry papers published from 2009 and 2019 outlines opportunities for increasing discovery of sex differences.Nat Commun.2022Apr19;13(1):2137. doi: 10.1038/s41467-022-29903-335440664WoitowichNC, BeeryA, WoodruffT. A 10-year follow-up study of sex inclusion in the biological sciences. Sugimoto C, Rodgers P, Shansky R, Schiebinger L, editors.eLife. 2020Jun9;9:e56344.WoitowichNC, WoodruffTK. Implementation of the NIH Sex-Inclusion Policy: Attitudes and Opinions of Study Section Members.J Womens Health.2019Jan;28(1):9–16. doi: 10.1089/jwh.2018.739630539677Garcia-SifuentesY, ManeyDL. Reporting and misreporting of sex differences in the biological sciences.AllisonDB, ZaidiM, VorlandCJ, KahathuduwaC, editors. eLife. 2021Nov2;10:e70817. doi: 10.7554/eLife.7081734726154MRC Working Group. Working Group on Sex in Experimental Design of Animal Research [Internet]. 2021 [cited 2023 Jan 23]. Available from: https://www.ukri.org/wp-content/uploads/2022/03/MRC-090322-SexInExperimentalDesign-SummaryReport.pdf.BeckerJB, PrendergastBJ, LiangJW. Female rats are not more variable than male rats: a meta-analysis of neuroscience studies. Biol Sex Differ. 2016;7:34. doi: 10.1186/s13293-016-0087-527468347BeeryAK. Inclusion of females does not increase variability in rodent research studies.Curr Opin Behav Sci.2018Oct;23:143–149. doi: 10.1016/j.cobeha.2018.06.01630560152PrendergastBJ, OnishiKG, ZuckerI. Female mice liberated for inclusion in neuroscience and biomedical research. Neurosci Biobehav Rev. 2014Mar;40:1–5. doi: 10.1016/j.neubiorev.2014.01.00124456941KarpNA, ReaveyN. Sex bias in preclinical research and an exploration of how to change the status quo. Br J Pharmacol. 2019Nov;176(21):4107–4118. doi: 10.1111/bph.1453930418665WaltzM, FisherJA, LyerlyAD, WalkerRL. Evaluating the National Institutes of Health’s Sex as a Biological Variable Policy: Conflicting Accounts from the Front Lines of Animal Research.J Womens Health (Larchmt).2021Mar1;30(3):348–354. doi: 10.1089/jwh.2020.867433211575FieldsRD. NIH policy: Mandate goes too far. Nature. 2014Jun;510(7505):340–340. doi: 10.1038/510340a24943947BuchT, MoosK, FerreiraFM, FröhlichH, GebhardC, TreschA. Benefits of a factorial design focusing on inclusion of female and male animals in one experiment. J Mol Med. 2019Jun1;97(6):871–877. doi: 10.1007/s00109-019-01774-030980104MillerLR, MarksC, BeckerJB, HurnPD, ChenWJ, WoodruffT, et al. Considering sex as a biological variable in preclinical research. FASEB J. 2017;31(1):29–34. doi: 10.1096/fj.201600781R27682203BeltzAM, BeeryAK, BeckerJB. Analysis of sex differences in pre-clinical and clinical data sets.Neuropsychopharmacol.2019Dec;44(13):2155–2158. doi: 10.1038/s41386-019-0524-331527863Three Years In: “Sex as a Biological Variable” Policy in Practice—and an Invitation to Collaborate [Internet]. GenderSci Lab. [cited 2023 Jan 23]. Available from: https://www.genderscilab.org/blog/three-years-in-sex-as-a-biological-variable-policy-in-practice-and-an-invitation-to-collaborate.WillTR, ProañoSB, ThomasAM, KunzLM, ThompsonKC, GinnariLA, et al. Problems and Progress regarding Sex Bias and Omission in Neuroscience Research.eNeuro. 2017Dec;4(6):ENEURO.0278-17.2017. doi: 10.1523/ENEURO.0278-17.201729134192KarpNA, MasonJ, BeaudetAL, BenjaminiY, BowerL, BraunRE, et al. Prevalence of sexual dimorphism in mammalian phenotypic traits.Nat Commun.2017Jun 26;8(1):15475. doi: 10.1038/ncomms1547528650954MakinTR, Orban de XivryJJ. Ten common statistical mistakes to watch out for when writing or reviewing a manuscript.RodgersP, ParsonsN, HolmesN, editors. eLife. 2019Oct9;8:e48175. doi: 10.7554/eLife.4817531596231NieuwenhuisS, ForstmannBU, WagenmakersEJ. Erroneous analyses of interactions in neuroscience: a problem of significance. Nat Neurosci. 2011Sep;14(9):1105–1107. doi: 10.1038/nn.288621878926GelmanA, SternH. The Difference Between “Significant” and “Not Significant” is not Itself Statistically Significant.Am Stat.2006Nov1;60(4):328–331.LakensD, CaldwellAR. Simulation-Based Power Analysis for Factorial Analysis of Variance Designs.Adv Met Pract Psychol Sci.2021Jan 1;4(1):2515245920951503.AmrheinV, GreenlandS, McShaneB. Scientists rise up against statistical significance. Nature. 2019Mar;567(7748):305. doi: 10.1038/d41586-019-00857-930894741WallachJD, SullivanPG, TrepanowskiJF, SteyerbergEW, IoannidisJPA. Sex based subgroup differences in randomized controlled trials: empirical evidence from Cochrane meta-analyses. BMJ. 2016Nov24;355:i5826. doi: 10.1136/bmj.i582627884869Percie du SertN, HurstV, AhluwaliaA, AlamS, AveyMT, BakerM, et al. The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research. Exp Physiol. 2020;105(9):1459–1466. doi: 10.1113/EP0888703266654610.1371/journal.pbio.3002129.r001Decision Letter 0RobertsRoland GSenior Editor2023Roland G RobertsThis is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.Submission Version0
9 Feb 2023
Dear Natasha,
Thank you for submitting your manuscript entitled "Scientists should not increase the overall n by default when including both sexes in in vivo studies: evidence from statistical simulations" for consideration as a Meta-Research Article by PLOS Biology.
Your manuscript has now been evaluated by the PLOS Biology editorial staff, as well as by an academic editor with relevant expertise, and I'm writing to let you know that we would like to send your submission out for external peer review.
However, before we can send your manuscript to reviewers, we need you to complete your submission by providing the metadata that is required for full assessment. To this end, please login to Editorial Manager where you will find the paper in the 'Submissions Needing Revisions' folder on your homepage. Please click 'Revise Submission' from the Action Links and complete all additional questions in the submission questionnaire.
Once your full submission is complete, your paper will undergo a series of checks in preparation for peer review. After your manuscript has passed the checks it will be sent out for review. To provide the metadata for your submission, please Login to Editorial Manager (https://www.editorialmanager.com/pbiology) within two working days, i.e. by Feb 13 2023 11:59PM.
If your manuscript has been previously peer-reviewed at another journal, PLOS Biology is willing to work with those reviews in order to avoid re-starting the process. Submission of the previous reviews is entirely optional and our ability to use them effectively will depend on the willingness of the previous journal to confirm the content of the reports and share the reviewer identities. Please note that we reserve the right to invite additional reviewers if we consider that additional/independent reviewers are needed, although we aim to avoid this as far as possible. In our experience, working with previous reviews does save time.
If you would like us to consider previous reviewer reports, please edit your cover letter to let us know and include the name of the journal where the work was previously considered and the manuscript ID it was given. In addition, please upload a response to the reviews as a 'Prior Peer Review' file type, which should include the reports in full and a point-by-point reply detailing how you have or plan to address the reviewers' concerns.
During the process of completing your manuscript submission, you will be invited to opt-in to posting your pre-review manuscript as a bioRxiv preprint. Visit http://journals.plos.org/plosbiology/s/preprints for full details. If you consent to posting your current manuscript as a preprint, please upload a single Preprint PDF.
Feel free to email us at plosbiology@plos.org if you have any queries relating to your submission.
Kind regards,
Roli
Roland Roberts, PhD
Senior Editor
PLOS Biology
rroberts@plos.org
10.1371/journal.pbio.3002129.r002Decision Letter 1RobertsRoland GSenior Editor2023Roland G RobertsThis is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.Submission Version1
22 Mar 2023
Dear Natasha,
Thank you for your patience while your manuscript "Scientists should not increase the overall n by default when including both sexes in in vivo studies: evidence from statistical simulations" was peer-reviewed at PLOS Biology. It has now been evaluated by the PLOS Biology editors, an Academic Editor with relevant expertise, and by three independent reviewers.
Based on the reviews, we are likely to accept this manuscript for publication, provided you satisfactorily address the remaining points raised by the reviewers and the following data and other policy-related requests.
IMPORTANT - please address the following:
a) Please attend to the requests from the reviewers.
b) Regarding reviewer #1's comment "Perhaps another Article Type is more appropriate?," the answer is "yes" - I hadn't noticed that this was submitted as a regular Research Article; I have now switched it to a Meta-Research Article, which we feel should address this reviewer's valid concern - the advance here is in meta-research, not statistics.
c) Please change the Title to "Statistical simulations show that scientists need not increase overall sample size by default when including both sexes in in vivo studies"
d) Please address my Data Policy requests below; specifically, we need you to supply the numerical values underlying Figs 1BCD, 2BCD, 3BCDE, 4BCD, S1ABC, either as a supplementary data file or as a permanent DOI’d deposition. I note that you already have a Zenodo deposition, but access is currently restricted, so I can’t check compliance; please rectify this.
e) Please cite the location of the data clearly in all relevant main and supplementary Figure legends, e.g. “The data underlying this Figure can be found in S1 Data” or “The data underlying this Figure can be found in https://doi.org/10.5281/zenodo.7629021
As you address these items, please take this last chance to review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the cover letter that accompanies your revised manuscript.
We expect to receive your revised manuscript within two weeks.
To submit your revision, please go to https://www.editorialmanager.com/pbiology/ and log in as an Author. Click the link labelled 'Submissions Needing Revision' to find your submission record. Your revised submission must include the following:
- a cover letter that should detail your responses to any editorial requests, if applicable, and whether changes have been made to the reference list
- a Response to Reviewers file that provides a detailed response to the reviewers' comments (if applicable)
- a track-changes file indicating any changes that you have made to the manuscript.
NOTE: If Supporting Information files are included with your article, note that these are not copyedited and will be published as they are submitted. Please ensure that these files are legible and of high quality (at least 300 dpi) in an easily accessible file format. For this reason, please be aware that any references listed in an SI file will not be indexed. For more information, see our Supporting Information guidelines:
Please note that you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out. Please see here for more details:
Should you, your institution's press office or the journal office choose to press release your paper, please ensure you have opted out of Early Article Posting on the submission form. We ask that you notify us as soon as possible if you or your institution is planning to press release the article.
*Protocols deposition*
To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols
Please do not hesitate to contact me should you have any questions.
You may be aware of the PLOS Data Policy, which requires that all data be made available without restriction: http://journals.plos.org/plosbiology/s/data-availability. For more information, please also see this editorial: http://dx.doi.org/10.1371/journal.pbio.1001797
Note that we do not require all raw data. Rather, we ask that all individual quantitative observations that underlie the data summarized in the figures and results of your paper be made available in one of the following forms:
1) Supplementary files (e.g., excel). Please ensure that all data files are uploaded as 'Supporting Information' and are invariably referred to (in the manuscript, figure legends, and the Description field when uploading your files) using the following format verbatim: S1 Data, S2 Data, etc. Multiple panels of a single or even several figures can be included as multiple sheets in one excel file that is saved using exactly the following convention: S1_Data.xlsx (using an underscore).
2) Deposition in a publicly available repository. Please also provide the accession code or a reviewer link so that we may view your data before publication.
Regardless of the method selected, please ensure that you provide the individual numerical values that underlie the summary data displayed in the following figure panels as they are essential for readers to assess your analysis and to reproduce it: Figs 1BCD, 2BCD, 3BCDE, 4BCD, S1ABC. NOTE: the numerical data provided should include all replicates AND the way in which the plotted mean and errors were derived (it should not present only the mean/average values).
IMPORTANT: Please also ensure that figure legends in your manuscript include information on where the underlying data can be found, and ensure your supplemental data file/s has a legend.
Please ensure that your Data Statement in the submission system accurately describes where your data can be found.
- Please note that per journal policy, we do not allow the mention of "data not shown", "personal communication", "manuscript in preparation" or other references to data that is not publicly available or contained within this manuscript. Please either remove mention of these data or provide figures presenting the results and the data underlying the figure(s).
Phillips et al. use simulations to show that including animals of both sexes and using an appropriate statistical analysis does not reduce the power to detect treatment effects, compared with using animals of one sex. The authors identify two misconceptions held by many scientists and three errors that are frequently observed in published papers. They then use simulations to demonstrate why the misconceptions are incorrect and provide better ways to analyse data.
This paper has been submitted as "Research Article" and I have been asked to comment on the "novelty and significance of the findings, its technical merit, the and experimental design". All the results are expected based on statistical theory and are well known to statisticians, and so are not novel or significant. However, demonstrating the results with simulations may be more convincing to many scientists. Having such a paper to reference when discussing these issues with colleagues and collaborators would be valuable. Perhaps another Article Type is more appropriate?
The statistical analyses are appropriate and the supplementary information provides a useful case study with real data. I cannot access the scripts and data on Zenodo as access is restricted.
I only have a few minor comments:
1) The font size is too small in all the figures and difficult to read. Adding
+ theme(text = element_text(size = 14))
to the plots should fix this.
2) Line 210: It would be useful to add Gelman and Stern's paper to the references here (https://www.tandfonline.com/doi/abs/10.1198/000313006X152649). I believe they were the first to draw attention to this problem.
3) Line 419-421: I think this sentence is missing some words. Is the intended meaning: "There is no requirement to perform a power calculation to detect …"?
4) Panel A in Figure 2 is referenced multiple times (lines 191, 193, 198, 200) but it's unclear where we should be looking as there are four graphs. Adding a more specific pointer would make it clearer (e.g. Fig 2A, top right).
Reviewer #2:
This is another outstanding and immensely practical contribution from this group. The paper does an excellent job in identifying the main issues, myths and all-too-common procedural errors involved in SABV studies. They provide transparent and readily reproducible methodology and code. I especially loved the explanatory boxes which made the methods clear without interrupting the flow of the text.
Ethical oversight assurances are provided, and most animal-based procedures reported in accordance with ARRIVE 2.0. However, there was no statement as to the eventual disposition of the mice, and how they were euthanased prior to collecting kidney tissue samples. Please provide description of these methods.
A few other minor comments:
1. The worked example is a really good idea. It could be made more practical for the average reader by streamlining the text quite a bit. The text is a bit clunky and focuses a lot on the results and less on the actual 'pipeline'. The text could be reduced by about 1/3rd & more bullet points added so it is easier to follow. Could the authors also consider placing this more upfront & incorporate something in the title & intro that states that the authors provide step by step methods on how to implement these features in actual studies (e.g. "tutorial" or "primer" into the title?) Or maybe consider a second companion paper? I am a bit worried that researchers who really need to see this will be put off by the heavy emphasis on the simulations & lessen the value of the practical implementation tips.
2, Although the authors incorporated bias minimisation methods such as allocation concealment and randomisation in their own studies, there is almost no mention up front. These are essential components of experimental design.
3. There is the added effect of cage. Was cage included as a block or were cage effects assumed to be zero?
4. The sample size justification statement is far too vague to be verifiable. A few sentences could be added to explain. Not saying that it is, but as it stands, it seems to reinforce the all-too-common practice of choosing 5 per "group" because that is the usual per-cage stocking density.
5. Mention should be made of sample size estimation in general for factorials. Powering factorials is different from the usual approach taken for two-group or single-factor completely randomised designs. As you know, main effects are estimated by comparing the means of 'sets of conditions' or a combination run, not by directly comparing the means of individual "groups". This needs to be mentioned upfront as well. Much of the resistance to adopting factorial designs is when researchers try to power them from G*Power which suggests far larger samples sizes are required. Sample sizes will also be too small for pairwise comparison of "groups". it is far more fruitful and less confusing to clarify that with factorials what is replicated are experimental runs (factor combinations) & that the researcher needs to think about total sample size, not group size. This is mentioned (sort of) in the text but it needs to be stated much more clearly. mention might be made of the Lakens and Caldwell paper (2021) on sample size for factorials. DOI:10.1177/2515245920951503
6. If the factorial is conducted for screening purposes, emphasis should be on how "relative promise" of the sex x trt interactions are evaluated e.g in terms of precision of the estimate. not statistical significance. mention of practical effect size could be added up front.
Reviewer #3:
Summary: In this manuscript, Philips and Colleagues conduct statistical simulations to address common misconceptions surrounding the use of both sexes in in vivo studies. It is commonly thought that in order to design an experiment which is statistically powered to analyze data by sex, the sample size (n) must double. To address this, they conduct power analyses which mirror a variety of experimental conditions with varied effects of treatment and sex. They provide clear explanations surrounding the power analyses and the results. Their simulations highlight the fact that in most scenarios, when sexes are split across treatment groups, the sample size is enough to sufficiently power the study.
Added Value of the Study: Several other articles have discussed how to design and analyze experiments involving both sexes. Most articles are written as reviews and provide overviews and/or tutorials of statistical methods. This manuscript expands upon that work by including the aforementioned power analyses and provides a case study which can be used for training purposes. It also reiterates the importance of factorial vs. pooled analyses, demonstrating that their incorrect use could lead to misleading results that ultimately detract from our understanding of how sex informs health and disease.
Overall, I find this article to be well written and easily accessible to a broad audience. However, I acknowledge that I do not have expertise in statistical modeling and hope that other reviewers can comment on strengths and/or weaknesses that pertain to the methods used here.
10.1371/journal.pbio.3002129.r003Author response to Decision Letter 1Submission Version2
10.1371/journal.pbio.3002129.r004Decision Letter 2RobertsRoland GSenior Editor2023Roland G RobertsThis is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.Submission Version2
18 Apr 2023
Dear Natasha,
Thank you for the submission of your revised Meta-Research Article "Statistical simulations show that scientists need not increase overall sample size by default when including both sexes in in vivo studies" for publication in PLOS Biology. On behalf of my colleagues and the Academic Editor, Marcus Munafò, I'm pleased to say that we can in principle accept your manuscript for publication, provided you address any remaining formatting and reporting issues. These will be detailed in an email you should receive within 2-3 business days from our colleagues in the journal operations team; no action is required from you until then. Please note that we will not be able to formally accept your manuscript and schedule it for publication until you have completed any requested changes.
Please take a minute to log into Editorial Manager at http://www.editorialmanager.com/pbiology/, click the "Update My Information" link at the top of the page, and update your user information to ensure an efficient production process.
PRESS: We frequently collaborate with press offices. If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximise its impact. If the press office is planning to promote your findings, we would be grateful if they could coordinate with biologypress@plos.org. If you have previously opted in to the early version process, we ask that you notify us immediately of any press plans so that we may opt out on your behalf.
We also ask that you take this opportunity to read our Embargo Policy regarding the discussion, promotion and media coverage of work that is yet to be published by PLOS. As your manuscript is not yet published, it is bound by the conditions of our Embargo Policy. Please be aware that this policy is in place both to ensure that any press coverage of your article is fully substantiated and to provide a direct link between such coverage and the published work. For full details of our Embargo Policy, please visit http://www.plos.org/about/media-inquiries/embargo-policy/.
Thank you again for choosing PLOS Biology for publication and supporting Open Access publishing. We look forward to publishing your study.