Can computers be peer reviewers?

Before any academics start mashing on their keyboards, hear me out.

Websters dictionary describes a peer review as a-



I'm sorry; I have no idea how that happened. Anyway Websters dictionary describes a peer review as a...

process by which a scholarly work (such as a paper or a research proposal) is checked by a group of experts in the same field to make sure it meets the necessary standards before it is published or accepted.

The real Webster

It seems nowadays that a positive peer review is all that is needed to validate or dismiss any argument and I can admit that I've done the same in the past. But what if you took the human element out of the equation and let a machine be the peer?

Computers are not bias or capable of deceit. Who better to trust to crunch the numbers then a computer? IF both sides of an argument can remain honest and non duplicitous then a machine could determine what facts are right and which are wrong. A computer could become an independent 'jury' and decide what is right science and what is wrong science and the results could force us to act accordingly when it comes to the climate or social issues.

However that might not work as humans are deceitful. Every issue that has been studied has been scrutinized by the other side of the coin as it always has been, and it always will be. False reports, misinterpreted figures, vested interests-all could contribute to skewered results.

At the end of the day we can question the peer review system on one issue and embrace it, yet we can question the peer review system on another issue and deride it. Maybe an emotionless machine could one day unite us.

a reply to: Thecakeisalie

Well I can program bias into a computer program, much like I can review something with bias on paper.

So, the same questions about legitimacy would still arise IMO.

Company develops AI to 'peer review' papers, ends up showing favorable results for their own work vs others. Same thing as hiring a group of people to do a study, while looking for a specific result.

~Tenth

a reply to: tothetenthpower

Yep, that is an issue for sure.

Such a system would have to be truly independent, free of proxies or other involvements, no self regulation.

So as long as the computer gets it's data from the internet, we can be sure of it's clarity and fact checking?

(I mean, as long as it's on the internet, then it must be true, right?)

I think the short answer is no.

If you go looking for answers to most any question, you are likely to find some disinfo scattered around. And a computer may not be programmed to eliminate dubious sources, or recognize obvious BS. Remember, a computer is only as good as it's programmer made it.

a reply to: network dude

As the doctor said "a computer is a very sophisticated idiot"

When it comes to certain issues like climate change, there is need for consistency. I believe in it but there is so much convolution and inconsistency. The same applies to the deniers and their research.

If we had a supercomputer dedicated to the task of making sense of all the facts and figures from both sides on any subject, a shade of grey would emerge. But until then...

I think we need to clear up the terminology a bit:

Peer review for •publication • is the process of referees ensuring that the work is novel and original as well as presented in the correctvl format. These will be other researchers in your field who are qualified to make these calls. This is NOT the end of the process nor does it "all that is needed to validate or dismiss any argument". It is but the first step. Many papers are published that turn out to be wrong/dead ends. And thats fine.

After publication, the work is peer reviewed by the wider academic community. Attempts at replication will take place, methodology will be dissected and any issues with the research will need to be addressed. Only once it stands up to the scrutinity of the wider community can the work be considered to be valid.

A lot of papers never get past thid second stage which is why we get a lot of these " Scientists say.." Headlines in the media with no follow up because science reporting in the media is terrible. It's also why a lot of crappy studies get used for ideological purposes (eg anti GMO) because the average person assumes that a published study is the be all and end all when it's not.

a reply to: Thecakeisalie

Indeed.

Robert Earl Keen - Shades of Gray Lyrics

We made Oklahoma a little after 3
Randy, his brother Bob and my old GMC
We had some moonshine whisky
And some of Bob's homegrown
We were so messed up we didn't know
If we were drunk or stoned
Randy was a sad-sack, tall kinda frail
Bob was a raving maniac, crazy in the head
They been kicked out of high school several years ago
For pushin over port-a-cans at the 4-H rodeo
Since then they've done their little dance
Right outside the law
Popped twice in Oklahoma, once in Arkansas
And I don't know what possessed me
To want to tag along
Cause I was raised a Christian
And I knew right from wrong

CHORUS:
Right or wrong, black or white
Cross the line your gonna pay
In the dawn before the light
Live and die by the shades of gray
We stole two Charolais heifers from
Randy's sweetheart's paw
Sold them at the livestock sale
Outside of Wichita

We got $900 and never did suspect
The world of hurt we'd be in once
We cashed that check
Next day we heard the story
On the local radio
Made our plans that very night
To go to Mexico
I swear we would have made it
If it wasn't for that shine
I got sick about the time we crossed
That Kansas line

CHORUS

I was layin in the bar ditch
Prayin I would die
When a light come on above us
And a voice come from the sky
A half a dozen unmarked cars
Came screeching to a halt
They grabbed bob, he started screamin
It was all my fault

There were men and dogs
And helicopter buzzin all around
They had the brothers on the
Pickup hood and me down on the ground
Bob flew all to pieces but randy he
Held tight when a black man in a
Suit and tie stepped out into the light
He told his men to turn us loose
They put down their guns
He said these are just some
Sorry kids, they ain't the ones

CHORUS

They left us by the roadside
Down hearted and alone
Randy got behind the wheel
Said boys I'm going home
We turned around to face our fate
Downhearted but alive on that
Mornin in late April, Oklahoma, 1995

CHORUS

Then you will need peer reviewers to peer review the computer code that does peer reviewing.