Originally posted by Klassified
I have not been through the process. Although the last time I had a hypothesis examined by a "peer", she found all kinds of errors.
Did you rework the hypothesis after that? That's what we have to do. I'm currently doing my PhD dissertation, and it's a hair-puller. I gave the
committee MY research design (and grand ideas) and they came after it with chainsaws and hatchets. I moaned and wailed because they whacked down my
Grand Idea into something that I thought was trivial -- BUT -- when I took their approach, I found that what they were pointing me at is a nice little
Then they told me that this would be one of the foundations of my "Life's Work." (oh sure. No pressure THERE, right??)
But when you're looking at days worth of reviewing time. I can only imagine someone thinking "how can I weed out some of this, and lessen this
mountain staring me in the face?"
What'll kill a paper first is Bad Reseach Design (or if it's not explained well.) After that, it's Bad Statistical Analysis.
Research Design transparency is CRITICAL. If we can't tell how you did the research, we can't prove that you really have something good, and we can't
confirm what you did by replicating your research. In a recent paper presented to the City of Dallas by the Centers for Disease Control (their
assessment of the West Nile Virus aerial spraying), the CDC went over how they gathered and measured data in very great detail before they gave their
conclusions. While the "science-y stuff" probably put everyone to sleep, it told me that the CDC didn't just show up and blow smoke at everyone. It
confirmed some of the data I had gathered independently and highlighted the real problems cities have in dealing with mosquitoes (they don't know how
to collect data or what to do with it when they get it.)
The reviewer doesn't need to know the authors name, or their background.
When a paper is passed for review, the names of the authors and their affiliations NEVER show up anywhere on the review copies.
Reviewers should have guidelines established, and personal opinions should be noted as such. I would like to say reviewers should be certified
by a board.
Magazines do have rubrics in place, yes, and where appropriate, most reviewers are board certified. Some disciplines don't have boards, though.
The problem is, so many who are not scientists, or academics, like myself (I'm self-employed, and I consult, as well as fix problems for
consumers and businesses relating to computers and Audio-visual.) rely on peer review as something akin to the "good housekeeping" seal of approval.
However, the more I read, and talk to a few folks I know, the more I have found that isn't what it is at all.
Perhaps it's just that the failures are so loudly talked about, and the millions (literally) of good papers are ignored?
Too much has been made of peer review to the public. And not enough measures have been taken among academia to insure that good theories don't
slip and fall through the cracks, because of the faults and limitations of the review process.
How do you know they are "good theories"?
Look around the board at all the theories you see here. None of them would get into any journals (including PLOS, and that's got VERY low standards
of "peer review" (I hesitate to call it that, frankly.)) There's no methodology, there's no background, it's just "wow! I got a great idea!"
My "great ideas" (I haz them) have to be well documented and explained before they can get into a paper. When I don't have enough data (example: the
"heat island effect" on ponds) then I will write an article for a magazine about the topic (because I'm retired, and all the science I do is funded
solely by my not-very-glorious pension and by social security (I'm older than dirt.))
I have seen a lot of self-made mathematicians on the Internet who explain how they've solved all sorts of grand mathematical things. I'm married to a
mathematician, so have gotten Math By Osmosis, and the people who are complaining that "academics are ignoring me" or "the government is suppressing
this" would fail your average senior level college math course. Most of them would fail a junior level math course, and some of them couldn't pass a
high school algebra course... yet they're telling us they've solved everything with math.
So... how do you know good ideas are suppressed?
Evidently, those on ATS who see peer-review as the gospel, don't read those letters, or go to conferences.
True. We talk about it, but it's not as exciting as mentioning Kim Kardassian's newest boyfriend or bathing suit. There's a zillion conferences with
exciting stuff and journals and all, but the latest moral outrage gets a lot more attention.
edit on 20-2-2013 by Byrd because: (no reason