Kevin Dewalt's blog on the 19th January includes 10 criticisms of peer review. I've posted a comment on his blog with my response to each of the points, but I'll copy them here as well.
Kevin's original points are italicised, and I've made a couple of additional comments since I replied on his blog that are indicated by square brackets. I hope I've corrected some misconceptions about peer review.
---
1. Unstated real or perceived conflicts of interest. Reviewers and authors can have relationships with entities that have an ulterior motive in getting material published.
True, but many journals, such as mine, require authors and reviewers to declare their competing interests - in our medical journals, these interests are published with the article. Editors are used to watching out for this.
---
2. Peer-review process advances slower than scientific progress.
Yes, but peer review doesn't stop someone first posting their article on their own web-site, discussing their work at conferences, or posting their work on a pre-print server like ArXiv. Anyway, scientific progress isn't as rapid as people believe, and without the checks and balances that peer review gives, all sorts of rubbish would be published, and scientists would have to follow even more blind alleys than they do already after reading profoundly flawed research. Peer review adds some rigour into the process of communicating scientific research. Less haste, more speed is an apt concept here.
---
3. The current process does not provide authors and reviewers with basic collaborative web tools.
That's nothing to do with peer review, just the delays in the Web 2.0 revolution getting to publishers. PLoS ONE (published by Public Library of Science, another OA publisher) does now offer reviewers and authors interactive tools to annotate articles. Many journals, like mine and the BMJ, allow any reader to comment on a published article.
---
4. Authors lose copyright privileges when publishing yet are often forced to publish to continue career advancement.
Traditional journals insist on copyright transfer. Many open access journals, including those published by BioMed Central and PLoS, allow the authors to retain copyright. The article is published under a Creative Commons Attribution License.
---
5. Peer-review networks tend to form around cliques. Those “outside the club” of a particular discipline - where often the best ideas surface - cannot get published because new ideas are rejected by the current establishment. As a result great ideas are often lost.
I don't believe that this complaint is really that valid. The complaints I've read about were by top scientists who couldn't get their idea published in Nature, Cell or Science. Well, just publish it elsewhere. There are plenty of journals that aren't as picky as those journals, and if authors had a little more self-awareness they'd recognise that they aim too high sometimes. Besides, many journals don't use established lists of reviewers, but go straight to those publishing related work and ask them. So, yes, you usually have to be a published scientist to review, but then it is called *peer* review, isn't it? I doubt that "the best ideas" surface outside academic research, the lone researcher is more likely to be a kook than a genius. There are some geniuses out there, but they are the ones you read about in the news - there's a teensy bit of a selection bias going on...
---
6. Precedence is often establish by those with the best personal contacts and not those who first introduce new theories.
I don't see the basis for this argument. Precedence does go to those who first raised a theory, so long as scientists are aware of it [this is the idea of 'priority']. Those who publish in languages other than English are at a disadvantage, admittedly, but some journals allow republication of work in English that was previously published elsewhere in another language, so that gives authors the possibility to widen their audience. Peer reviewers go out of their way to alert authors to work that first demonstrated something, and I have also insisted that authors cite certain studies. Scientists are very attuned to giving due credit for the origin of ideas or techniques.
---
7. There is no medium for wider, instant dissemination. Doctors or researchers who prepare a presentation or speech cannot “publish it” to a wider audience.
Yes, they can. ArXiv and other pre-print servers allow the publication of non-reviewed work (see e.g. Public Knowledge Project). Theses and dissertations can be published electronically (e.g. NDLTD, MIT on DSpace). This Portugese university repository, for example, allows the publication of reports, presentation etc. If a university doesn't have a repository for this kind of material, then it should do! Staff and students can take the lead, rather than waiting for journals to do it for them - journals are traditionally for peer-reviewed research, why would we necessarily expect them to post presentations? That said, Nature has recently launched Nature Protocols, so publishers are making some effort to include material that is outside their usual range.
8. Participating in the review process has little benefit for the reviewer. Performing reviews can take an enormous amount of time and the written reviews are not themselves “published”.
Reviewing takes between 2-6 hours, according a survey I read [an average of 3 hours]. I've seen reviews done in 10 minutes...
Here are a few reasons for participating in peer review:
- Allows a researcher control over what is published in their field - they are the "gatekeepers of science".
- Allows researchers to ensure that what is published accurately reflects and acknowledges their field.
- Can help a scientist get promoted and get grants, as journals often list the names of their reviewers annually.
- In the case of the medical journals in the BMC-series, published by BioMed Central, we *do* publish the reports, along with the name of the reviewer.
- Reviewers are actually paid by a small minority of journals [the BMJ pays £25], and more commonly can get other perks such as discounts on reading or publishing in the journal.
- Reviewers get the opportunity to read their competitors' work months before it will be published, and unscrupulous reviewers can deliberately block publication.
- It's interesting! They're scientists, they enjoy critiquing science!
9. Reviews and reviewers are not “reviewed”. An author who receives a biased review or one based on poor critical thinking has no recourse to publicly respond or invite others to comment.
Not true. Editors assess the reviewer reports and qualifications. Authors who receive what they perceive to be a biased review can appeal to the editor, and request a further opinion. If they are badly treated and the journal is a member of the Committee on Publication Ethics (such as BioMed Central, BMJ, Lancet) then authors can even take a case to that body [currently only editors can submit a case, but often will in cases of a dispute]. Some journals (BMJ, Lancet) have an ombudsman.
10. Journals can be prohibitively expensive for some in the developing world.
Yes - this is one of the reasons why open access is a good idea! The research is free to read for anyone with Internet access. Traditional pay-to-view journals are also members of a scheme called HINARI, a WHO project that allows some people in developing countries to read the research for free (but it does have limitations, as they need to be connected to an institution).
