In August 2008 the editor-in-chief of Wired magazine, Chris Anderson, proclaimed, “The End of Theory: The Data Deluge Makes the Scientific Method Obsolete”. This claim was backed up by 12 case studies that he uses to illustrate his point; Google quality data analysis has made the search for causation obsolete and that “correlation is enough”. There will be many working scientists who will not lose much sleep over this claim. In the end the Supercrunchers have to have data to work on and in science good data is still not trivial to obtain, nevertheless this article and its premise should provoke active scientists and statisticians to think deeply about the implications.
In contrast to Wired we believe that the core tools of science and scientific inference are as valid today as they ever were. But the pitfalls for lack of rigour are worse. There are now even more ways to do it wrong than right. We believe that even more than ever scientists need to be skilled in the art of Integrated Quantification (IQ); the ability to take into account all aspects of quantification in a holistic manner to achieve high quality scientific research.
This skillset is not taught yet in current University courses; neither science degrees nor statistics degrees deal with post data deluge quantification in a holistic manner. The ability to critically appraise the output of a Google-Algorithm that has found a putative correlation will be more important than ever. In business it may be enough to say that the correlation helps give you and edge when interacting with billions of humans but in science this has yet to be proven. If we don’t build this skillset and build methods to utilise it in both academic and commercial research we will have a new generation of scientists who believe the Wired view; the scientific method is obsolete.