Is medical science built on shaky foundations?

More than half of biomedical findings cannot be reproduced – we urgently need a way to ensure that discoveries are properly checked

It appears as though "the truth will out" as Shakespeare penned.

For the longest time I have been trying to spread the word about the fallacies of Meta-research and the knee-jerk press release nature of commercial research. Apparently, scientist who expect actual science are starting to be heard.

One goal of scientific publication is to share results in enough detail to allow other research teams to reproduce them and build on them. However, many recent reports have raised the alarm that a shocking amount of the published literature in fields ranging from cancer biology to psychology is not reproducible.

Pharmaceuticals company Bayer, for example, recently revealed that it fails to replicate about two-thirds of published studies identifying possible drug targets (Nature Reviews Drug Discovery, vol 10, p 712).

Bayer's rival Amgen reported an even higher rate of failure - over the past decade its oncology and haematology researchers could not replicate 47 of 53 highly promising results they examined (Nature, vol 483, p 531). Because drug companies scour the scientific literature for promising leads, this is a good way to estimate how much biomedical research cannot be replicated. The answer: the majority.

For emphasis, The majority of reported biomedical research cannot be replicated!

Yet, it is based on this largely private research that our political leaders are forming policy. For example, deciding whether to 'mandate' vaccination, what the "approved" medical treatment for a disease or condition will be, etc. Based on this research, insurance companies "decide" what medicines will be prescribed, and what treatments are "covered." Imagine the profit potential when your company is a 'favored' company by an insurance firm? Imagine when the decision about how you will be treated to maintain or improve your health may or may not be based on research that some group 'decided' sounds better.

The reasons for this are myriad. The natural world is complex, and experimental methods do not always capture all possible variables. Funding is limited and the need to publish quickly is increasing.

There are human factors, too. The pressure to cut corners, to see what one wants and believes to be true, to extract a positive outcome from months or years of hard work, and the impossibility of being an expert in all the experimental techniques required in a high-impact paper are all contributing factors.

The cost of this failure is high. As I have experienced at first hand as a researcher, attempts to reproduce others' published findings can be expensive and frustrating. Drug companies have spent vast amounts of time and money trying and failing to reproduce potential drug targets reported in the scientific literature - resources that should have contributed towards curing diseases.

Failed replications also quite often go unpublished, thereby leading others to repeat the same failed efforts...

The author makes a case for a pilot program to address the problem...

But where profit is involved... how much weight does paltry scientific uncertainty hold with those who decide?

Edit to add source: www.newscientist.com...

there are numerous cases of people faking results,
there is an incentive to find the results favourable to the funders,
this creates sloppy or misleading results that tend to reinforce profits.
one month ago a pharma saftey tester was convicted of faking his entire carers worth of data.
they have to re test his 25 years worth of work in the industry.

i blame the copywrite and patent laws that incentivise the creation of "anything" that is then made to be scarce,
weather it works or not, that then puts legitimate researches out of pocket and incentives cutting corners to be first to market, instead of first to be safe and effective.



to me this is the source of the problem.

this system ensures cheaters prosper while blocking acual innovation,
with the passage of the ttp (trans pacific partnership)
this problem will intensify and the terms of copy right and patent will get longer and more onerous.

as a designer and inventor i found both systems acually stifle innovation,
not promote it

xploder

One classic example of this is the arguement about vaccines - where is the empirical evidence that shows researchers the results of double blind testing??? To my knowledge there are no such tests to be found in any archives. The same can be said for fluoride or many other pharmaceuticals that 'appear' suddenly on the market.

Unfortunately this article is about the money/paper trail. I've said it before and I will say it again - I was teaching a senior advanced maths class not long ago and the Students opened up a conversation about science. Essentially the Students stated to me that all science is theory. Even high school Students can see clearly that science is theory and all I had to add was the fact - the theories that make it to the media are the ones that are funded.

Just follow the money to see who, in the arena of science, is being supported. Control of medicine and healing are what this thread are all about. It will be interesting to see what people have to say about this topic.

Much Peace...

I think the title is misleading, and the data itself is being interpreted wrong.
They're questioning the FOUNDATIONS of medicine.....when what they are talking about is additions being made to it, i.e. new research. Sorry but new research isn't the FOUNDATION of current medical practices.
And just because experiments are published, doesn't mean the drugs will be approved by the FDA. The FDA does their own testing, and at the very least doesn't approve drugs based off of one study.

TLDR: Most of the drugs that are currently prescribed have been thoroughly tested. This is about research practices, not medical practices; they are different.

Nothing to do with the foundations of medical science. Everything to do with the rush to publish. The trouble is that when the shoddy research is discovered by those who would provide the funding the futility is exposed and no grant...no job. It's a waste of time which could be better spent.

It's not the science which is at fault. It is, in fact, science which reveals the failures. Science requires replication of results.

Yet, it is based on this largely private research that our political leaders are forming policy. For example, deciding whether to 'mandate' vaccination, what the "approved" medical treatment for a disease or condition will be, etc. Based on this research, insurance companies "decide" what medicines will be prescribed, and what treatments are "covered."

No. Such decisions are made on the basis of clinical trials. You are talking about "research" which occurs long before clinical trials are done.

Originally posted by Phage
Science requires replication of results.

That's what I was thinking. I'm not sure what the rate of failure to replicate is in areas outside of medicine, but we've all heard of the famous cold fusion experiment as a famous example and that too was apparently a rush to publish on the part of Pons and Fleischman, rather than a measured approach to publishing good science.

After I read the OP title, but before I read the OP story, I thought it might be about how many doctors have actually finished "practicing" medicine. I've made a few bad picks of MDs that were definitely still "practicing" in every sense.

But drugs that make it to market have a lot of data behind them, and even with all that work, some drugs still get pulled from the market when they get even more data from a larger population. So it shouldn't be too surprising that preliminary research based on a lot less data has more problems with replication.

Originally posted by Arbitrageur

But drugs that make it to market have a lot of data behind them, and even with all that work, some drugs still get pulled from the market when they get even more data from a larger population. So it shouldn't be too surprising that preliminary research based on a lot less data has more problems with replication.

More than half of biomedical findings cannot be reproduced..

It is difficult for me to understand the apparent confidence in what we are sold as science, when the odds are 50/50 or worse that they have actually practiced 'good science.'

reply to post by Maxmars

It is difficult for me to understand the apparent confidence in what we are sold as science, when the odds are 50/50 or worse that they have actually practiced 'good science.'

I guess you missed the point of the article in your OP. It not about good science. It is about bad science. Research which is rushed to be published. The article is not about us being "sold" anything because what it is talking about never reaches us, beneficial or not.

But where profit is involved... how much weight does paltry scientific uncertainty hold with those who decide?

There really is no profit involved with the shoddy research. Only wasted time. The article in the OP is about that wasted time and effort. That bad research detracts from good research because upon the application good science it is found to be lacking. That bad science benefits no one. Not the original researchers. Not the companies which see (at first look) a promising direction. Not the consumers.

Drug companies have spent vast amounts of time and money trying and failing to reproduce potential drug targets reported in the scientific literature - resources that should have contributed towards curing diseases.

www.newscientist.com...

The problem is broadly applicable to all scientific disciplines. See this article for a cursory glance at just how pervasive it is. Search a while online and you'll find numerous articles on this very thing. Science is as much faith as religion, and to many it is one and the same.

Compelling reading though I agree with Phage about the conclusions mostly.

From the Guardian:

It comes as no surprise, then, that to survive in academia, let alone thrive, scientists must now game the system in ways that would have appalled our forebears. Outright fraud is just the tip the iceberg. Beneath it churns an ocean of dubious practices that spans the physical, biological and social sciences.

Source

I remember my brother telling me about his time with journals and academic science, and more than once he was asked to lie or at least be overly optimistic about his own research. He no longer does research. The academic requirement to publish for results makes so many temptations, and grants even more so. (Is a minor point I disagree with Phage on, money is certainly involved at even the academic level.)

An unspoken rule among early-stage venture capital firms that “at least 50% of published studies, even those in top-tier academic journals, can't be repeated with the same conclusions by an industrial lab” has been recently reported (see Further information) and discussed

Source

Taking into account that drug companies are likely to attempt to follow up more ambitious leads, and have quite large numbers of reasons for considering a project a failure it might not be as bad .... Still many of them are due to negligence or even fraud.

It's not so much that the OP is wrong that this isn't an issue ... its been known for a long time that academia and journals have huge issues. It is the price we pay for mixing capitalism with education and 'first in' reward. Think about it, Francis Crick, James Watson and Rosalind Franklin discovered the double helix structure of DNA ... off the top of your head can you name who replicated their findings first?

reply to post by Amanda5

One classic example of this is the arguement about vaccines - where is the empirical evidence that shows researchers the results of double blind testing??? To my knowledge there are no such tests to be found in any archives.

In college, we were TOLD vaccines were proven not to be the cause of childhood brain defects. The "approved" research comes from those whose best interest is to forbid contradictory research. All contradictory evidence was considered not credible and not appropriate or acceptable to be used in our own research. There is a list of topics that are banned from research papers as well. 12 years ago in college, I only heard of this control in research I wasn't studying in the medical field. Recently though, I experienced it first-hand. The whole system is corrupt...just take a look at the socialist organization walking arm in arm with the teachers that are on strike....Our system is no longer about teaching people to think, they are taught what to think. Got to get your feet wet to learn this. There is no place for those who will not submit to the state. Keep that in mind , follow the money (paycheck).

If the medical foundation is based on biochemical research coupled with the use of pharmaceuticals to treat disease then yes, this is Bad Science. Too many factors are not taken into account to restore the body to an optimal state of health. That's not medicine, it's a quick fix with no lasting results. It's been proven time and time again the body becomes toxic when pharmaceuticals are present even in small quantities over large periods of time or in large quantities over small periods of time. Researchers need to stop looking at the biochemical pharmaceutical interaction and start taking all of the organs working together in synergy into account. This is simply not done in the real world. Sure some drugs can help in many cases but they are not meant to cure the person and bring them back to proper health. This is the core problem of today's modern medicine. Most of the time when people are prescribed pharmaceuticals over a long period of time, they develop other symptoms of disease for which they are given other drugs. These in turn cause other problems, and leads to a cascade effect of disease that could have been prevented if they just stopped feeding the body with harmful toxic chemicals. This is what doctors call Toxicity and it used to be normal that when people died in the hospital, you could see toxicity listed as the cause of death. All this meant was for some reason not explained, they body built up enough toxins to start shutting down the body. No one wants to admit it was the drugs they were feeding the patient all those years that caused the problem. You have a medical association that knows this and does not care because there is too much money to be made and when you couple that with the insurance companies backing their favorite so called " medicines" you have the potential for medical malpractice that's as epidemic as the Ebola plague can be itself. This has already happened years ago and is getting worse all the time. Yes, I'd say there is ample room to question the foundations of medicine, but not necessarily for the reasons the Op states.

Why is it that this idea of science being some kind of dark organization, where dark shadow figures control what is being published and destroy peoples careers based on what big phama wants to do. I am quite sorry but, this thing that people believe so much in just does not exist. And at worse doesn't even come CLOSE to what people claim.

Vaccines and brain disorders? you know how stupid the arguments are? cause and effect? i read a lovely article that most data presented by the anti vacs people is guilty of being hand picked and massaged to say what they want it to say.

Why? Well, statement that is most prevalent Vaccines cause autism, look at this data that shows the number of vaccines given against cases of autism, oh look they both go up!

Then they ignore the fact that cases of autism vs numbers of vaccines is not at all correlated when you split the data sample by country and or by ethnic groups. For something to be the absolute cause, data must be reproducible as you all said... what is presented by the Anti-vac people is absolute garbage at best and it makes me sad how much people swallow it because it has a few celeb figure heads who have been on TV screaming about it.


Medical research being conducted has many many stages, and a great deal of it is conducted at Universities. There is a degree of secrecy to these places because as you all point out, the next big breakthrough is worth a lot of money. Now unfortunately i will admit that possibility of fame and fortune does get the better of people sometimes. But do any of you know what peer review is about? or even like?

The issue is partly because peer review is very slow and very very expensive at times... publishing results of research in a renowned journal is a long and sometimes expensive process. It has resulted in a market opening for journals that perform little or no peer review and such publish anything that you give them for some money. THIS is the truth behind alot of medical research being of reduced quality or repeatability because research groups are often judged on how many papers they produce, rather than any merit of quality... the whole thing results in people who have no idea about how the system works in science making statements that "OH big pharma is doing x y and z and its a big cover-up omg omg end of the world" The truth is much more simple, and unfortunately much sadder.

Medical science is whatever big pharma says it is. $$$

reply to post by OldSchoolMom


I recently re-visited some information on the Amish people. They do not vaccinate their Children and only eat what they grow and no sitting in front of computers and televisions with ipods and gameboys going incessantly. I learned that there have been only three documented cases of autism within the Amish community - two of the Children were vaccinated and there was no information available regarding the third Child. That's a very good track record as far as I am concerned.

Three Children vaccinated and two confirmed with Autism and one with no data to include. Information that makes you wonder why there is still any debate. Thanks for the straightforward information that tells it the way it really is.

The Amish mind their own business and are an excellent control as far as any social/medical/science research is concerned. Hard to argue that they are an invalid example of how good uncomplicated living actually keeps you healthy!!!!! They are an excellent example of health - more power to them.

Much Peace...

reply to post by Amanda5

I learned that there have been only three documented cases of autism within the Amish community - two of the Children were vaccinated and there was no information available regarding the third Child.

Not that it has anything to do with the topic but where did you learn that?

From September 2008 to October 2009, 1899 Amish children were screened in the two Amish communities. A total of 25 children screened positive for ASD on either the SCQ or the DSM-IV-TR checklist. A total of 14 screened positive for ASD on both screeners. Of those 25 children, 14 were evaluated and seven children were confirmed as having a diagnosis of ASD using the ADI and/or ADOS, and clinical judgment. Interestingly, four of the seven only met ASD criteria on the ADOS but not the ADI. Three of the four who were not diagnosed by the ADI only missed criteria on the Behavioral Domain, which may be attributable to the reporting style of Amish caregivers.

www.opposingviews.com...

While the autism rate for the Amish is lower than that of the general population there are a number of significant differences between the two populations (lifestyle and genetic diversity, not to mention social factors which may affect reporting). To arbitrarily select vaccination as the reason for the difference is not valid.

If you think this doesn't make it to market...

The glaring relatively recent example is Scott Reuben, an anaestesiologist who had many papers retracted after being found out by an audit.

Pulled Vioxx was a drug included among his retracted studies.

Double standards are more exposed lately.

Walker-Smith won appeal, Wakefield persecuted / a pillar of vaccine-negative autism link, Thorsen from Denmark's Aarhus University charged for CDC grant fraud.

I've said it before and I'll say it again.

Pharmaceutical companies do not care about you, or helping you with your problems. They care about money and creating a situation where you need more of their drugs, to help treat the problems associated with other drugs you are taking.

When you take pharmaceuticals you are signing up for a trial.

This is just more proof of that.

~Tenth

reply to post by tothetenthpower

You too are missing the point. The article in the OP is not about pharmaceutical companies. It is about "research" which is done before pharmaceutical companies become involved.

Drug companies have spent vast amounts of time and money trying and failing to reproduce potential drug targets reported in the scientific literature - resources that should have contributed towards curing diseases.

www.newscientist.com...

reply to post by wujotvowujotvowujotvo

Reuben's fabricated research is another story but his studies were not involved with the approval of Vioxx.

The retracted studies aren't expected to affect the drugs' regulatory status because Dr. Reuben's studies weren't part of the packages that manufacturers submitted to the FDA or European authorities.

online.wsj.com...:a16:g2:r1:c0.0270612:b22894832