Honestly, reducing the teaching + publish-or-perish + the constant need to apply for grants would go a long way towards fixing the review process. Academics have to spend a lot of time doing a lot of non-academic work that peer reviewing properly sometimes gets pushed down the list of priorities.
Maybe, just maybe, if editors did a hint of work with all the money they steal from public science funding, we could stabilise the system towards more integrity and less quantity of publication. Or also just get rid of editors to obtain the same result, but this is sadly utopic today. Peer reviewing is not the problem, and probably still is the best way to assess research quality. However, tendency towards quantity over quality, and applied research over fundamental are what skews the process and its results
if editors did a hint of work with all the money
Exactly. Why do authors need to pay for review/publication but the reviewers are volunteer and the journals paywalled? There is a fundamental mismatch between who gets vs deserves the money.
Wait, reviewers are not paid?!
Not typically, no. There have been exceptions ofc, but peer reviewing is typically volunteer.
I had no idea. That’s fucked up.
Eh, it’s not as bad as it sounds TBH. Paid reviewers would have ethical and economic pressures that hinder their effectiveness. Non-specialists in the same field would end up responsible for reviews of articles they are only rudimentarily familiar with (think astrophycisists working on exoplanet formation and composition having to review papers on black hole implications for dark energy. They ‘could’ but are not the best qualified to do so). Needing to review enough papers to earn a living means this dilution multiplied 100-fold to get enough done.
With volunteering at least scientists that are interested in that paper’s topic, and hence are likely a specialist in it, are the ones looking at it and doing so at their leisure instead of needing to do 100 by weeks end to put food on their table.
Personally, I think all privatization involved should be removed. Volunteer reviewers to public non-profit journals paid either by donations or tax dollars and freely accessible to all.
Why does the selection process for reviewers need to also disappear to provide wages? Journals still select reviewers for best fit. Ultimately, peer reviewers are performing labor for a corporation making profit from that labor. It is unethical for anyone to be put in a position to provide free labor in the pursuit of profit for a corporation.
I’m arguing that the solution to your concern is that the corporation and profit aspects be removed. The reason I prefer this angle is because science emphasizes the need to remove all or as much bias as possible and economic incentives inevitably induce bias’ and restrictions that increase the problems I’ve already pointed out earlier.
The question of whether peer reviewers ought to be paid and how is a complex one that has many ethical considerations on either side of the argument. I strongly recommend you research this debate yourself if you are interested in the subject.
Right, your take sounds convincing. Thanks for the insight!
The hell aren’t colleges publishing this stuff themselves? There isn’t an academic journal published by even one single university?
what we really need is funding for experiment duplication.
There is literally no other option than peer review for science.
Does peer review need to work the way that it does now with publishers as gatekeepers and an expectation that work will be reviewed for free? No, the process should absolutely change but it will still require peers to review new papers. Publishing before review and anonymous reviewers is a terrible idea.
If you read the article, they are suggesting a different approach to peer review, not doing away with it. They want to find ways to build in incentives for reviewers to make it worth their while to review rather than allowing it to continue as something that scientists do out of a sense of obligation.
They have an interesting approach but I think it doesn’t go far enough.
Couldn’t you have researchers who specialize in finding “bugs” in published papers (yes, researchers already do this to each other), like we have QA testers or bounties for finding exploits? Is this too aggressive an approach for science? Should work for hard sciences, though.
This is a great idea I think, but part of the problem with science versus programming is that they’re just very different social environments, so the expectations, norms, and demands on each are very different.
Dependent down a little bit more, most research is done by people with phds or other advanced degrees (or pursuing them) in an academic job, and one of the conditions of attaining or maintaining that job is publishing. And these are the same people doing the peer reviews.
I think what this creates, even aside from the overwhelming volume and complexity of work, is a certain amount of grace amongst academics. That is, I think a fair number of peer reviewers are not only failing to rigorously grapple with the material that they review, but because of the small social mileux and shared incentives, they are incentivized to not be very rigorous in many cases.
Not saying peer review is without value, but how harshly would you want to challenge or critique the work of someone whom you may work alongside or under in the future?
i heard about a woman a while back that did exactly that: she read papers across disciplines and found doctored results etc… she’d found something like 10 papers that had fabricated data
I had a thought and didn’t make it clear, added the last sentence that I’m referring to the new system allowing for anonymous reviews. That combined with publish before review is making a new system catered to malicious business interests. Tobacco companies would just love this system.
Now the idea of making the whole process more visible to a wider audience? Yeah, that could be a benefit.
Like paying them?
How about Ai?
You.are joking, right? I’m just missing that implied /s?
If my company can buy an AI powered proposal manager, someone is working on AI peer reviews. So while I was joking, i’m afraid that its coming.
There is always place for a system that could potentially help with pointing out issues, as long as it’s not the only one
