Reproducibility and replicability in natural science research: Is there a crisis?
DOI:
https://doi.org/10.25260/EA.21.31.1.0.1060Keywords:
questionable research practices, non-replicability causesAbstract
We assume that because of the self-correction nature of the scientific knowledge, every published study will be revised by somebody that will replicate it and, eventually, publish a correction. Therefore, the process will discard errors and keep successes to build the established scientific knowledge. However, because of questionable research practices, the process does not work as well as expected, and has driven a reproducibility and replicability crisis (although some will deny such crisis). This document considers in general the state-of-the-art of the problem and presents a local example (at least as a proof of concept) showing that questionable research practices are more prevalent than we think. I conclude calling for spaces to reflect on the problem and I suggest ideas to reduce the impact of its causes, especially important for the training of young researchers.
References
Amrhein, V., S. Greenland, B. McShane, et al. 2019. Retire statistical significance. Nature 567:305-307.
Baker, M., D. Penny. 2016. Is there a reproducibility crisis in science? Nature 533:452-454. https://doi.org/10.1038/533452a.
Cummings, G. 2013. Dance of the p-values. URL: https://www.youtube.com/watch?v=5OL1RqHrZQ8.
Cummings, G. 2017. Significance roulette. URL: https://www.youtube.com/watch?v=OcJImS16jR4.
Farji-Brener, A., and A. Ruggiero. 2010. ¿Impulsividad o paciencia? ¿Qué estimula y qué selecciona el sistema científico argentino? Ecología Austral 20:307-314.
Fraser, H., T. Parker, S. Nakagawa, A. Barnett, and F. Fidler. 2018. Questionable research practices in ecology and evolution. PLOS One 13(7):e0200303. https://doi.org/10.1371/journal.pone.0200303.
GAISE. 2016. College Report ASA Revision Committee, Guidelines for Assessment and Instruction in Statistics. Education College Report 2016. URL: http://www.amstat.org/education/gaise.
Ioannidis, J. 2005. Why most published research findings are false. PLoS Med 2(8):e124. https://doi.org/10.1371/journal.pmed.0020124.
Ioannidis, J. 2016. Why Most Clinical Research Is Not Useful. PLOS Med 13(6):e1002049. https://doi.org/10.1371/journal.pmed.1002049.
Leek, J. T., and R. D. Peng. 2015. P values are just the tip of the iceberg. Nature 520:612. https://doi.org/10.1038/520612a.
Mazieres, M., and E. Kohler. 2014. Get me off Your Fucking Mailing List (accepted, unpublished). http://www.scs.stanford.edu/~dm/home/papers/remove.pdf.
Munafo, M. R., B. A. Nose, D. V. M. Bishop, K. S. Button, C. D. Chambers, N. P. du Sert, U. Simmondson, E. Wagenmakers, J. J. Ware, and J. P. A. Ioannidis. 2017. A manifesto for reproducible science. Nat Hum Behav 1, 0021. https://doi.org/10.1038/s41562-016-0021.
NAS (National Academies of Sciences, Engineering, and Medicine). 2016. Statistical Challenges in Assessing and Fostering the Reproducibility of Scientific Results: Summary of a Workshop. Washington, DC. The National Academies Press. https://doi.org/10.17226/21915.
NAS (National Academies of Sciences, Engineering, and Medicine). 2019. Reproducibility and Replicability in Sciences. Washington DC. The National Academic Press. https://doi.org/10.17226/25303.
O’Boyle, E. H., G. C. Banks, E. González-Mulé. 2014. The Chrysalis effect: how ugly initial results metamorphosize into beautiful articles. Journal of Management 20(10):1-24. https://doi.org/10.5465/ambpp.2013.43.
Steward-Lowndes, J. S. S., B. D. Best, C. Scarborough, J. C. Afflerbach, M. R. Frazier, C. C. O’Hara, N. Jiang, and B. S. Halpern. 2017. Our path to better science in less time using open data science tools.
Nat Ecol Evol 1(6):160. https://doi.org/10.1038/s41559-017-0160.
Wasserstein, R. L., and N. A. Lazar. 2016. The ASA Statement on p-Values: Context, Process, and Purpose. The American Statistician 70(2):129-133. https://doi.org/10.1080/00031305.2016.1154108.
Wasserstein, R. L., A. L. Schirm, N. A. Lazar. 2019. Moving to a world beyond P<0.05. The American Statistician 73(1):1-19. Special Issue Statistical Inference in the 21st Century: A World beyond P<0.05. https://doi.org/10.1080/00031305.2019.1583913.
Wittingham, M. J., P. A. Stephens, R. B. Bradbury, and R. P. Freckleton. 2006. Why do we still use stepwise modelling in ecology and behaviour? Journal of Animal Ecology 75:1182-1189. https://doi.org/10.1111/j.1365-2656.2006.01141.x.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2021 David E. Gorla
This work is licensed under a Creative Commons Attribution 3.0 Unported License.
Authors retain their rights as follows: 1) by granting the journal the right to its first publication, and 2) by registering the published article with a Creative Commons Attribution License (CC-BY 4.0), which allows authors and third parties to view and use it as long as they clearly mention its origin (citation or reference, including authorship and first publication in this journal). Authors can make other non-exclusive distribution agreements as long as they clearly indicate their origin and are encouraged to widely share and disseminate the published version of their work.
