Thursday 12 January 2017

A manifesto for reproducible science (2017)

Above is a figure from the latest paper by John Ioannidis and chums, just published by Nature Human Behaviour - A manifesto for reproducible science - the full paper is free to download HERE.

The ABSTRACT reads:
Improving the reliability and efficiency of scientific research will increase the credibility of the published scientific literature and accelerate discovery. Here we argue for the adoption of measures to optimize key elements of the scientific process: methods, reporting and dissemination, reproducibility, evaluation and incentives. There is some evidence from both simulations and empirical studies supporting the likely effectiveness of these measures, but their broad adoption by researchers, institutions, funders and journals will require iterative evaluation and improvement. We discuss the goals of these measures, and how they can be implemented, in the hope that this will facilitate action toward improving the transparency, reproducibility and efficiency of scientific research.
As a tax payer, already supporting millions of pounds worth of research by UK state, I don't think it is unreasonable to ask why all science isn't already done in the way that is described in this paper.

PS. I particularly like the acronym HARKing - meaning hypothesizing after the results are known - which I had not come across before. The original reference is Kerr, N. L. HARKing: hypothesizing after the results are known. Pers. Soc. Psychol. Rev. 2, 196–217 (1998).

PPS. Kerr's original paper from 1998 has the following ditty as an epigraph:

A reader quick, keen, and leery
Did wonder, ponder and query
When results clean and tight
Fit predictions just right
If the data preceded the theory