An epidemic of false claims

Over recent decades there have been increasing signs of concern about scientific integrity, particularly in medical and climate research, but in many other scientific areas too.

This is a large and complex issue where those interested have to do their own digging, because much of the material lies below the mainstream radar. Evidently journalists find it quicker and easier to copy and paste the official scientific press release rather than check it out.

A name of particular significance in medical science is that of Professor John P. A. Ioannidis. Last year (June 2011) Ioannidis published An Epidemic of False Claims in Scientific American.

False positives and exaggerated results in peer-reviewed scientific studies have reached epidemic proportions in recent years. The problem is rampant in economics, the social sciences and even the natural sciences, but it is particularly egregious in biomedicine. Many studies that claim some drug or treatment is beneficial have turned out not to be true. We need only look to conflicting findings about beta-carotene, vitamin E, hormone treatments, Vioxx and Avandia. Even when effects are genuine, their true magnitude is often smaller than originally claimed.

Much research is conducted for reasons other than the pursuit of truth.

Note the reference to Avandia which as the BBC reported recently (July 2nd 2012) was one of the drugs involved in GlaxoSmithKline’s record $3 billion fine for off-label marketing.

GlaxoSmithKline is to pay $3bn (£1.9bn) in the largest healthcare fraud settlement in US history. The drug giant is to plead guilty to promoting two drugs for unapproved uses and failing to report safety data about a diabetes drug to the Food and Drug Administration (FDA). The company also conceded charges that it held back data and made unsupported safety claims over its diabetes drug Avandia.

An extract from one of the comments on the Epidemic of False Claims article is worth noting too.

The second real problem here is the almost complete breakdown in the peer review system. Papers with unfounded claims and unsubstantiated positive results are being passed routinely for publication. The bar has been lowered continuously over the last 25 years and the criteria for publication has [sic] been diluted.

In 2005 Ioannidis published a paper titled Why Most Published Research Findings Are False which has been the most downloaded technical paper from the journal PLoS Medicine.

  •  False findings may be the majority or even the vast majority of published research claims
  •  The greater the financial and other interests and prejudices in a scientific field, the less likely the research findings are to be true
  •  The hotter a scientific field (with more scientific teams involved), the less likely the research findings are to be true

In 2010 The Atlantic published an article on him titled Lies, Damned Lies, and Medical Science.

Much of what medical researchers conclude in their studies is misleading, exaggerated, or flat-out wrong. So why are doctors—to a striking extent—still drawing upon misinformation in their everyday practice? Dr. John Ioannidis has spent his career challenging his peers by exposing their bad science.

But it isn’t just medical science. Inadequate protocols, statistical errors, lack of reproducibility and outright fraud are gradually seeping into our perceptions of how science is done in a world with far too many second-rate scientists churned out by university PhD mills. Added to that are numerous temptations and opportunities for those with little talent, grants to earn and enough ambition to lower the bar even further.

Scattered and poorly reported the examples may be, but there is plenty of evidence out there and it seems to be growing. If there are financial and professional gains to be had from lowering standards, then those standards will be stomped into the ground by someone, somewhere.

Scientists are no longer perceived exclusively as guardians of objective truth, but also as smart promoters of their own interests in a media-driven marketplace.

Haerlin & Parr, Nature, 1999 [Greenpeace members. Well they should know.]

The scale of the problem isn’t easy to judge, because it is still not widely reported in spite of all the evidence out there. Even the corruption of climate science, by far the worst and most obvious example of modern times, is not well reported and ignored entirely by the BBC.

Yet here we have a sub-prime example of the wholesale collapse of scientific integrity. We have seen the usurpation of evidence by policy – abetted by greed, vanity and the most mind-boggling stupidity.

Finally on a slightly lighter note we have J Scott Armstrong on publication bias, or the problems encountered in trying to rock the scientific boat. Armstrong’s paper is thirty years old this year – a tongue in cheek look at the issue, but does anyone think the situation is better today than it was a generation ago?

Barriers to Scientific Contributions: The Author’s Formula:

  1. Do not pick an important problem
  2. Do not challenge existing beliefs
  3. Do not obtain surprising results
  4. Do not use simple methods
  5. Do not provide full disclosure
  6. Do not write clearly

6 comments for “An epidemic of false claims

  1. July 4, 2012 at 11:52 am

    I’ve replied at your place. Hope you don’t mind me co-opting this for the next post?

    • July 4, 2012 at 12:52 pm

      No problem.

  2. Voice of Reason
    July 4, 2012 at 8:50 pm

    That’s one of the reasons why I’m happy in theoretical mathematics. It has to be correct, or it’s not published (except extremely rare cases).

    As for the drug studies, one reason (which some faculty are pushing to change), is that there is no requirement to publish studies. Thus, those paid for by drug companies which get negative results don’t get published.

    I know people who are studying the ephemeral nature of some medical results.

    I will stand by my contention that the average scientist is as honest as humanly possible.

    • July 4, 2012 at 9:53 pm

      “I will stand by my contention that the average scientist is as honest as humanly possible.”

      My experiences suggest this too, but I don’t think scientists are any more honest than anyone else. Why should they be? I think the problem lies in a general unwillingness to accept that there is a problem which should be brought out into the open.

      • Voice of Reason
        July 6, 2012 at 4:09 am

        Which is why the concept of peer review is so brilliant, and so unnatural. Even one’s favourite idea can die.

  3. Greg Tingey
    July 5, 2012 at 7:36 am

    Excuse me, but are not the scientists, and the scientific establishment actually investigating these supposed false claims?
    Which somewhat renders your asupposed argument false – again ??????

    There is also, often the problem, that scientists can see there is a “problem” (something is obviously WORNG somewhere) … but what? Where does one start?
    The classic recent example is the revolution in the Earth Sciences 1960-80, with the realisation that Plate TEctonics answered (almost) all the questions as to “continental drift” and sepciation & distribution, with an underlying mechanism.
    But all the separate bits had to be integrated.
    Not an easy job

Comments are closed.