Fake Scientific Paper Fools 157 Journals!

This is just ludicrous!

A spoof scientific report was recently accepted for publication in 157 journals around the world.

In a sting operation conducted by the journal Science, contributing correspondent John Bohannon uncovered a "Wild West" landscape among fee-seeking publishers -- a portion of which use false addresses, false names, overseas bank accounts and superficial "peer reviews" on a routine basis.
"From humble and idealistic beginnings a decade ago, open-access scientific journals have mushroomed into a global industry, driven by author publication fees rather than traditional subscriptions," wrote Bohannon, a molecular biologist and science reporter.
"Most of the players are murky," he wrote. "The identity and location of the journals' editors, as well as the finacial workings of their publishers, are often purposefully obscured."
Hoping to test the academic rigor of these journals, Bohannon concocted a false and fatally flawed study on a wonder cure for cancer. Variations of the paper, which were sent to 304 journals, contained experimental blunders that should have been detected during a proper review.

The spoof study had at least three problems:

-The study drug killed cancer cells with increasing doses, even though its data didn't show any such effect.
-The drug killed cancer cells exposed to medical radiation with increasing effect, even though the study showed the cells weren't exposed to radiation.
-The study author concluded the paper by promising to start treating people with the drug immediately, without further safety testing.

"If the scientific errors aren’t motivation enough to reject the paper, its apparent advocacy of bypassing clinical trials certainly should be," Bohannon writes.

The spoof study was organised by Science Magazine

Additional links here and here.

I am sure there have been a fair few papers that slip through the net in terms of not being error free, or in some aspect, slight dubious. This is quite astonishing though, the sheer number it was published in. What becomes more worrying is the fact that the same exploit could of been used many of times for gain, whilst muddying certain areas of study and even further, promoting poor research or flawed non-science as fact based which could be putting people in danger, i.e medicinal publications.

Hopefully this will raise the alarm bells and at the very least help cast a more discerning eye over the publications, promotion of research and journals.

Were these same folks fooled by the praise heaped upon Vioxx too? I mean, since those studies and reports were all fake too...

And, of course, a peer reviewed article doesn't have to be an intentional hoax to be wrong either.
That's were the independent replication and verification part of science comes into it.

hmm, a traditional science publisher launches a sting only on open-access journals. Why not also include traditional publishers? I think you can find plenty of evidence that many traditional journals have reviewers who can't discriminate good science from bad.

As evidenced by this story from a few years ago:

www.theguardian.com...

The statistical error that just keeps on coming
The same statistical errors – namely, ignoring the "difference in differences" – are appearing throughout the most prestigious journals in neuroscience

We all like to laugh at quacks when they misuse basic statistics. But what if academics, en masse, deploy errors that are equally foolish? This week Sander Nieuwenhuis and colleagues publish a mighty torpedo in the journal Nature Neuroscience.

They've identified one direct, stark statistical error so widespread it appears in about half of all the published papers surveyed from the academic neuroscience research literature.

Phage
And, of course, a peer reviewed article doesn't have to be an intentional hoax to be wrong either.
That's were the independent replication and verification part of science comes into it.

edit on 10/3/2013 by Phage because: (no reason given)

If the process of 'peer review' is as fundamentally flawed and corrupt as shown, what makes you imagine replication and verification is going to be any less?

reply to post by MysterX

Because the flaw is one of carelessness. Sloppiness.

Using the hoaxed article as an example; the claim was that no clinical trials were going to be undertaken, that treatments were going to begin. An absurdity. The article was not reviewed at all.

Now, let's say another researcher, after publication reads, the article. Assuming that they didn't catch the obvious errors like the claims not matching the data. What are they going to do if the article arouses their interest because it doesn't sound quite right? Well, the first thing is to look closely at the data and the conclusions. That would bust the fraud right there but the next thing is to perform the same or similar experiments.

That's how it works. Science is actually more a process of proving something wrong than proving something right. Scientists take a great deal of pleasure in disproving a theory if they can. Peer review before publication is only the first step, it's a winnowing. The real test comes when other scientists deem a paper even worth reading. Next, when they study it. And finally, if they see that it may have merit, trying to duplicate the results.

So what's the problem? Why is it bad if the journal review panel does a crappy job? The problem is that it wastes the time of the scientists who pursue what turns out to be a dead end.

reply to post by Phage


The fact the hoax deliberately contained multiple absurdities, blatent and obvious errors is really the point.

It shows the peer review process, which is meant to be a form of professional self-regulation that lends professional credibility to a scientific endevour, has become rotten to the core. Or at least, is now being shown to be rotten.

Profits are being put before scientific honesty, accuracy and professionalism.

This discredits the entire process and while you may think it doesn't matter, there are many that will and do.

reply to post by MysterX

This discredits the entire process and while you may think it doesn't matter, there are many that will and do.

Please specify where I said it doesn't matter. You seem to have set up a straw man that you can knock down at your pleasure.

I said clearly that a sloppy peer review system is a hindrance. I said that it wastes time, and therefore effort and resources.

The entire "system" is self correcting. Bad science, whether intentional or otherwise, will fail.

The thing is you also have researchers using these journals to beef up their resumes, showing that they have been published, how many employers are going to fully investigate each claim? Not to mention lawyers who use scientific citations in trials and Government officials who draw on published research to set policies at times.

The more important thing is to actually look at the journals that apparently reviewed and accepted it. Now I am going to make a small leap and make a statement that the point of the whole thing was to expose these journals as being frauds.

My experience with publishing is that it is extremely bad practice to publish the same paper in more than one article. You might send it to 2 or 3, but the assumption is that the final publication will not be in more than 1 or 2 places. This happens in Physics, medicine, i am not sure of but I cannot see any logical reason why it would not be the same.

Now, because of the rather toxic culture of "Publish or Perish" in some areas of science, it has cultivated the "fake peer review" Journals. These journals will publish just about anything and are about as complicated as buying a fake PhD degree from some dodgy print shop for $50. It is a very well known problem in science and this article points out how BIG a problem it really is.

The other problem is the following. The number of these so called journals are outnumbering the credible ones, because it is great looking for someone's CV if they can publish 15 papers that are full of garbage and not written very well, rather than publishing 1 or 2 which have been through a real peer review process.

Peer review takes time, and I can guarantee you all that anything that is actually peer reviewed comes back with comments. And by the sounds of this, all of the fake journals basically accepted them as is. What you all need to realize while making the "Oh my god science and peer review doesn't work omg omg omg" is that as i said... those journals do not peer review and are just out to make money... they are frauds. There is one journal in India which is run by a professor for the soul purpose of publishing his own papers, because when he went to peer review, he found that no one in their right mind would publish the fundamentally flawed work he does... so he made his own journal, and invited people to submit papers for a "administration fee" for publication.

The problem is like having Oprah peer review a science paper... she doesn't know anything about science

MysterX
It shows the peer review process, which is meant to be a form of professional self-regulation that lends professional credibility to a scientific endevour, has become rotten to the core.

No, it doesn't.

It shows that at some journals, the peer review process didn't even take place.

And it's not like this is a total surprise...for this very reason, if you want to post at Physicsforums.com, you can only cite articles from certain journals in a list that they cite, which I'm guessing excludes a lot if not most or all of these journals that don't really do any peer review (or at least attempts to do so).

ErosA433
The more important thing is to actually look at the journals that apparently reviewed and accepted it. Now I am going to make a small leap and make a statement that the point of the whole thing was to expose these journals as being frauds.

I'm glad to see somebody gets it.

I don't think this is supposed to be a paper about whether peer review is working or not. It's an attack on open source science publishing. This is coming from an elite and expensive journal, just at the same time their business model is under attack from a governmental push towards open source publishing of publicly funded science.

It is obvious that there are some very low brow open access journals online. That is to be expected. But any decent scientist should quickly be able to work out the quality and prestige of the source and filter based on that. And with open access publishing - everyone gets the opportunity to judge whether the work is bad or not. It isn't hidden away, only accessible to elites.

I'd like to see Science repeat this experiment, with a slightly more believable but equally false paper (say, based on quantum physics), and send it to hundreds of paid journals. Then publish the names of which ones accepted it.

yampa
I'd like to see Science repeat this experiment, with a slightly more believable but equally false paper (say, based on quantum physics), and send it to hundreds of paid journals. Then publish the names of which ones accepted it.

That would be a better test of peer review, to send it to journals that actually do peer review, but the problem I see with sending it to hundreds of journals is that it would probably end up going to some of the same reviewers from different sources, wouldn't it?

This is a good response to the Science article:

bjoern.brembs.net...

Yesterday, Science Magazine published a news story (not a peer-reviewed paper) by Gonzo-Scientist John Bohannon on a sting operation in which a journalist submitted a bogus manuscript to 304 open access journals (observe that no toll access control group was used). Science Magazine reports that 157 journals accepted and 98 rejected the manuscript. No words on any control groups or other data that would indicate what the average acceptance rate for bogus manuscripts might be in general.

As Michael Eisen points out, this story is merely the pot calling the kettle black, when Science Magazine is replete with bogus articles (such as that on #arseniclife, for instance) and the magazine has one of the highest retraction rates of the entire industry. Which brings me to the main point of this post: it should come as no surprise that Science Magazine publishes a news story on an ill-conducted sting operation, an anecdote without proper controls – that’s what glamor magazines like Science, Cell or Nature do. The data that we have on this fact are quite unequivocal: hi-ranking journals like these retract many more papers than any other journal and a large fraction of these are retracted because of fraud. There is not even a single quality-related metric in the literature that would confidently express any advantage, quality-wise, of hi-ranking journals over others. However, there are a number of metrics which suggest that, in fact, the quality and reliability of the science published in these Glam Magz is actually below average.

To make things worse, when we submitted this data to Science Magazine, they rejected it with the remark that “we feel that the scope and focus of your paper make it more appropriate for a more specialized journal”. Obviously, Science Magazine values anecdotes more than actual data. No surprise their retraction rate is going through the roof: rejecting data that make them look bad and publish anecdotes that make them look good.