American medicinal plants (1887)

From HERE

Ninety Percent of everything is crud

Sturgeon's Law of Quality states that: 'Ninety percent of  everything is crud'. 

The last few posts here have shown that there is compelling evidence that Sturgeons Law of Quality applies to the peer reviewed papers that appear in our finest scientific journals.  

The original statement of this Law is below:

"... they say 'ninety percent of science fiction is crud.' Well, they're right. Ninety percent of science fiction is crud. But then ninety percent of everything is crud, and it's the ten percent that isn't crud that is important. and the ten percent of science fiction that isn't crud is as good as or better than anything being written anywhere."

Theodore Sturgeon 1953

From HERE

Note that the Wikipedia article on Sturgeon's Law refers to this Law as 'Sturgeon's Revelation' with an additional adage being listed as Sturgeon's Law - Nothing is always absolutely so.

See also the OED entry HERE

Science at-Large on Planet F345, Andromeda Galaxy, Year 3045268

John Ioannidis is on a roll. Ioannidis is a professor at Stanford School of Medicine who does a number of things - one of which is to expose what is wrong with current approaches to publishing science. In particular he enjoys finding methodological weaknesses and flaky statistics. He is the author of the excellent Why Most Published Research Findings Are False (HERE). 

Recently Ioannidis published a paper called Why Science Is Not Necessarily Self-Correcting HERE. The asbtract of this  paper begins "The ability to self-correct is considered a hallmark of science. However, self-correction does not always happen to scientific evidence by default". He goes on to describe a speculative future of science on Planet F345...

"Planet F345 in the Andromeda galaxy is inhabited by a highly intelligent humanoid species very similar to Homo sapiens sapiens. Here is the situation of science in the year 3045268 in that planet. Although there is considerable growth and diversity of scientific fields, the lion’s share of the research enterprise is conducted in a relatively limited number of very popular fields, each one of that attracting the efforts of tens of thousands of investigators and including hundreds of thousands of papers. Based on what we know from other civilizations in other galaxies, the majority of these fields are null fields—that is, fields where empirically it has been shown that there are very few or even no genuine nonnull effects to be discovered, thus whatever claims for discovery are made are mostly just the result of random error, bias, or both. The produced discoveries are just estimating the net bias operating in each of these null fields. Examples of such null fields are nutribogus epidemiology, pompompomics, social psychojunkology, and all the multifarious disciplines of brown cockroach research—brown cockroaches are considered to provide adequate models that can be readily extended to humanoids. Unfortunately, F345 scientists do not know that these are null fields and don’t even suspect that they are wasting their effort and their lives in these scientific bubbles.

Young investigators are taught early on that the only thing that matters is making new discoveries and finding statistically significant results at all cost. In a typical research team at any prestigious university in F345, dozens of pre-docs and post-docs sit day and night in front of their powerful computers in a common hall perpetually data dredging through huge databases. Whoever gets an extraordinary enough omega value (a number derived from some sort of statistical selection process) runs to the office of the senior investigator and proposes to write and submit a manuscript. The senior investigator gets all these glaring results and then allows only the manuscripts with the most extravagant results to move forward. The most prestigious journals do the same. Funding agencies do the same. Universities are practically run by financial officers that know nothing about science (and couldn’t care less about it), but are strong at maximizing financial gains. University presidents, provosts, and deans are mostly puppets good enough only for commencement speeches and other boring ceremonies and for making enthusiastic statements about new discoveries of that sort made at their institutions. Most of the financial officers of research institutions are recruited after successful careers as real estate agents, managers in supermarket chains, or employees in other corporate structures where they have proven that they can cut cost and make more money for their companies. Researchers advance if they make more extreme, extravagant claims and thus publish extravagant results, which get more funding even though almost all of them are wrong.

No one is interested in replicating anything in F345. Replication is considered a despicable exercise suitable only for idiots capable only of me-too mimicking, and it is definitely not serious science. The members of the royal and national academies of science are those who are most successful and prolific in the process of producing wrong results. Several types of research are conducted by industry, and in some fields such as clinical medicine this is almost always the case. The main motive is again to get extravagant results, so as to license new medical treatments, tests, and other technology and make more money, even though these treatments don’t really work. Studies are designed in a way so as to make sure that they will produce results with good enough omega values or at least allow some manipulation to produce nice-looking omega values.

Simple citizens are bombarded from the mass media on a daily basis with announcements about new discoveries, although no serious discovery has been made in F345 for many years now. Critical thinking and questioning is generally discredited in most countries in F345. At some point, the free markets destroyed the countries with democratic constitutions and freedom of thought, because it was felt that free and critical thinking was a nuisance. As a result, for example, the highest salaries for scientists and the most sophisticated research infrastructure are to be found in totalitarian countries with lack of freedom of speech or huge social inequalities—one of the most common being gender inequalities against men (e.g., men cannot drive a car and when they appear in public their whole body, including their head, must be covered with a heavy pink cloth). Science is flourishing where free thinking and critical questioning are rigorously restricted, since free thinking and critical questioning (including of course efforts for replicating claimed discoveries) are considered anathema for good science in F345."

Of course if science on Earth was performed today like it is on F345 it would be both depressing and very difficult to accurately discern the difference between real-science and psuedo-science.

 Image of Andromeda from HERE

Current trends in the reliability of science

A nice paper HERE by  Björn Brembs, Katherine Button and Marcus Munafò called "Deep impact: unintended consequences of journal rank".  Go download it.

Their Abstract reads:

Most researchers acknowledge an intrinsic hierarchy in the scholarly journals (“journal rank”) that they submit their work to, and adjust not only their submission but also their reading strategies accordingly. On the other hand, much has been written about the negative effects of institutionalizing journal rank as an impact measure. So far, contributions to the debate concerning the limitations of journal rank as a scientific impact assessment tool have either lacked data, or relied on only a few studies. In this review, we present the most recent and pertinent data on the consequences of our current scholarly communication system with respect to various measures of scientific quality (such as utility/citations, methodological soundness, expert ratings or retractions). These data corroborate previous hypotheses: using journal rank as an assessment tool is bad scientific practice. Moreover, the data lead us to argue that any journal rank (not only the currently-favored Impact Factor) would have this negative impact. Therefore, we suggest that abandoning journals altogether, in favor of a library-based scholarly communication system, will ultimately be necessary. This new system will use modern information technology to vastly improve the filter, sort and discovery functions of the current journal system.

And their Conclusions

While at this point it seems impossible to quantify the relative contributions of the different factors influencing the reliability of scientific publications, the current empirical literature on the effects of journal rank provides evidence supporting the following four conclusions: (1) journal rank is a weak to moderate predictor of utility and perceived importance; (2) journal rank is a moderate to strong predictor of both intentional and unintentional scientific unreliability; (3) journal rank is expensive, delays science and frustrates researchers; and, (4) journal rank as established by IF violates even the most basic scientific standards, but predicts subjective judgments of journal quality.

The following Figure from their paper shows (A) Exponential fit for PubMed retraction notices (data from pmretract.heroku.com) and (D) Linear regression with confidence intervals between Impact Factor and Retraction Index (data provided by Fang and Casadevall, 2011).

Cosmic View (1957)

Long before Charles and Ray Eames made 'Powers of Ten' (1968 & 1977) a Dutch writer called Kees Boeke wrote Cosmic View, it was published in 1957. It also inspired the computer game Spore.

The Introduction was by Arthur H. Compton:

INTRODUCTION

What are we? Where do we live? Who are our neighbors? Children and grown-ups, we all ask these questions.

The answers that Kees Boeke gives are only the beginning of the story, but that beginning is straightforward and clear. The author shows us a series of pictures of a little girl as seen from different distances. Around her are the things that form her world. We see her also as it were from within, showing the parts she is made of. These various views present one school child in an immense range of perspectives. We begin to understand how big things are and how we are related to them.

It is not easy to do what the author has done so well, to tell accurately and in simple language what the world is like. Here is a reliable framework to which further knowledge can be added. In describing this framework, the author has gone as far as the present state of our knowledge permits. Fifty years ago our cosmic view would have been much more limited. Nothing could have been drawn with confidence in pictures 20 to 26 or in pictures -8 to -14. There is reason to question whether we shall ever be able to draw what would be the next pictures, 27 or -14.

So it is that only now, in our day, we can see ourselves so clearly. In this immense and varied universe we find ourselves indeed one with other boys and girls, other men and women. In showing us how we ourselves look in perspective, Kees Boeke as a skillful teacher helps us also to know how and what our neighbors are.

The author deserves our thanks for giving us his answers to our questions in this fascinating and understandable form.

ARTHUR H. COMPTON 

A full set of the images and text are HERE.