I want to congratulate John P. A. Ioannidis on his thought-provoking Essay [<xref ref-type="bibr" rid="pmed-0020386-b1">1</xref>]. I have two comments. In Corollary 1, he suggests that small sample sizes mean smaller power, and implies that larger studies with thousands of subjects are more likely to be true. I think it is important to stress that if the effect size is large (e.g., very small variance that is seen in physiological studies), then adequate power is obtained with small numbers. Furthermore, some would argue that exposing subjects to research risks unnecessarily (e.g., when fewer subjects would yield sufficient power) is unethical. Since the analysis is based on power, we should remember that larger is not always better. In Corollary 4, Ioannidis argues that greater flexibility in designs, definitions, etc. means the results are less likely to be true. I agree that replication of all aspects of the study is more likely to yield consistent results, but this does not necessarily mean true results. Since we don't know a priori which methodological details are most appropriate (e.g., dose, timing, etc.), heterogeneous results from different designs is an important source of information and can lead to a new, more in-depth understanding of the subject—and sometimes even paradigm shifts. I agree with the accompanying Editorial [<xref ref-type="bibr" rid="pmed-0020386-b2">2</xref>] to the article that we need to distinguish between the validity of the data and the validity of the authors' conclusions. </sec> </body> <back> <ref-list> <title>References

plos

pmed

PLoS Med

plosmed

PLoS Medicine

1549-12771549-1676

Public Library of Science

San Francisco, USA

10.1371/journal.pmed.0020386

Correspondence

Genetics and Genomics Science Policy Public Health and Epidemiology Mathematics/Statistics Public Health and Epidemiology

Communication in Health Care Editorial policies (including conflicts of interest) Medical journals

Power, Reliability, and Heterogeneous Results

Correspondence

Shrier

Ian

¹ ^*

McGill University, Montreal, Quebec, Canada

E-mail: ian.shrier@mcgill.ca

The author has declared that no competing interests exist.

11 2005

29 11 2005

211

e386

2005

Ian Shrier

This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Why Most Published Research Findings Are False

Truth, Probability, and Frameworks

Author's Reply

The Clinical Interpretation of Research

Minimizing Mistakes and Embracing Uncertainty

N/A.

I want to congratulate John P. A. Ioannidis on his thought-provoking Essay [<xref ref-type="bibr" rid="pmed-0020386-b1">1</xref>]. I have two comments. In Corollary 1, he suggests that small sample sizes mean smaller power, and implies that larger studies with thousands of subjects are more likely to be true. I think it is important to stress that if the effect size is large (e.g., very small variance that is seen in physiological studies), then adequate power is obtained with small numbers. Furthermore, some would argue that exposing subjects to research risks unnecessarily (e.g., when fewer subjects would yield sufficient power) is unethical. Since the analysis is based on power, we should remember that larger is not always better. In Corollary 4, Ioannidis argues that greater flexibility in designs, definitions, etc. means the results are less likely to be true. I agree that replication of all aspects of the study is more likely to yield consistent results, but this does not necessarily mean true results. Since we don't know a priori which methodological details are most appropriate (e.g., dose, timing, etc.), heterogeneous results from different designs is an important source of information and can lead to a new, more in-depth understanding of the subject—and sometimes even paradigm shifts. I agree with the accompanying Editorial [<xref ref-type="bibr" rid="pmed-0020386-b2">2</xref>] to the article that we need to distinguish between the validity of the data and the validity of the authors' conclusions. </sec> </body> <back> <ref-list> <title>References 1

Ioannidis

JPA

Why most published research findings are false.

PLoS Med 2005 2 e124 doi: 10.1371/journal.pmed.0020124

PLoS Medicine Editors

Minimizing mistakes and embracing uncertainty.

PLoS Med 2005 2 e272 doi: 10.1371/journal.pmed.0020272