Elsevier experiments with peer review

Well I never. I've been advocating the adoption of open peer review and community peer review for a while now; I didn't expect one of the pioneers of community peer review to be Elsevier, but they've surprised me.

On 21 June, they announced a three-month trial of what they are calling PeerChoice on Chemical Physics Letters, which allows potential reviewers to volunteer to review papers. As Ida Sim points out, this doesn't open up peer review in the sense of making it more transparent, but it should help speed up peer review and it might avoid the bias caused by editors selecting from a limited pool of the same 'usual suspect' reviewers.

The devil is in the details: who gets to be in the pool of potential reviewers; how will you motivate reviewers to volunteer, when getting reviewers to agree when directly inviting them can be hard enough; will volunteers be vetted for suitability for that article; is this alongside or instead of editorial selection? These question aside, let's hope it's a success.

---
Edit: There's some answers on the hidden-away page about PeerChoice - PeerChoice is supplementary to editor-invited reviewers. Registered reviewers will see titles and abstracts and be allowed to download the manuscript if they agree to provide a "timely review." There doesn't appear to be a vetting/vetoing system, but the editor still makes the decision. The trial is on nanostructures and materials; the results might not be applicable outside that very narrow field as scholars in different fields react in very different ways to variations in the peer review process.

Medical Hypotheses' editor is sacked

So Bruce Charlton's editorship at Medical Hypotheses comes to an end, and I must raise a small cheer. Schadenfreude is an ugly thing, but this journal was a boon to fringe 'scientists' everywhere, giving them the apparent legitimacy of publishing in a 'proper journal' (owned by Elsevier, indexed in PubMed) without the pesky hurdle of peer review. It was no surprise that it favoured kooks, having been set up by David Horrobin, a pusher of evening primose oil.

The final straw was allowing AIDS denialists a platform, and the subsequent outcry from scientists and Charlton's inability to see what he did wrong led Elsevier to pull the plug. Charlton thinks that as an editor he has a perfect right to publish whatever papers he wishes, but unaccountable editorial control is no way to run a journal. Poor editorial decisions should have consequences, and the lack of any peer review or other quality control on Medical Hypotheses (the only criterion being that a paper was 'interesting') always doomed it to be derided by serious scientists and medics.

Will the new (and improved?) Medical Hypotheses see any more gems like too much sex causing RSI, kissing evolving to spread germs, cancer being caused by stopping smoking, masturbation being good for relieving a bunged up nose, or the origin of belly button fluff?

Reviewing Medical Hypotheses

This is a guest post by Joe Dunckley
Zoë Corbyn writes in the Times Higher this week that Elsevier have "started an internal review" of legendary journal Medical Hypotheses following its publication last year of two hiv/aids denialism papers (covered in Bad Science here and Respectful Insolence here). One of the offending papers, lead-authored by notorious aids denialist Peter Duesberg, took an entire two days from submission to acceptance by the peer review shunning "journal", and had already been rejected from all of the real hiv/aids journals for making such embarrassing claims as that Uganda's population increase proves that hiv can not cause aids.

It would be a shame to loose the journal that gave us Ejaculation as a potential treatment of nasal congestion in mature males and the equally entertaining response, Ejaculation as a treatment for nasal congestion in men is inconvenient, unreliable and potentially hazardous, but at the same time, we have to consider whether we are really comfortable continuing to humour the confused outbursts of Bruce Charlton.

It's interesting to note that the best defence for the journal's existence that Corbyn could find was this: "while peer review worked for 'normal science', it also had the power to suppress radical ideas." The defence comes from intelligent design creationist Steve Fuller, whose ideas I don't think even Med Hypotheses sunk as low as publishing.

Arms trade and publishing - strange bedfellows

Although the pen is mightier than the sword, involvement in publishing hasn't kept Reed Elsevier out of the defense industry.

The conflict between on the one hand being involved in advances to aid the treatment of patients and on the other arranging the sale of lethal weapons has garnered increasing criticism.

A letter organised by the Campaign Against Arms Trade pitted the Lancet against its own publisher, and the BMJ has waded in with a call for a boycott of the Lancet. This focus on the Lancet appears to be aiming to force a wedge between journal and publisher - imagine the fall-out if the Lancet left Elsevier?

A petition on Idiolect mentions that "DSEi's 2005 official invitees included buyers' delegations from 7 countries on the UK Foreign Office's list of the 20 most serious human rights abusing regimes, countries like Colombia, China and Indonesia... Reed Elsevier arms fairs have featured cluster bombs, depleted uranium munitions and torture equipment". Nice.

Another petition organised by Nick Gill has some real teeth as it calls for a boycott of Elsevier journals.

Richard Smith and some others have even been engaging in shareholder action at the AGM.

I won't sign either petition as I obviously have an ulterior motive, but I'm certainly not posting this simply to stick one in Elsevier's eye for their opposition to open access. Others might want to consider signing the petitions. The defence industry might be legal (but not always), but then as has been pointed out, so was slavery.

Publisher and journal nicknames

A previous handling editor of BMC Ear, Nose and Throat Disorders affectionately (?) referred to the journal as BMC Wax, Snot and Phlegm, and rumour has it that our independent journal Cough was almost called Cough and Phlegm. A blog going by the name wax, boogers, and phlegm actually exists.

The Public Library of Science (PLoS) is known by editors at the Lancet as SLoP, and I'm sure that the nicknames Evil Empire and Elseviley need no introduction. In the Pipeline lists the colloquial ways of referring to several journals in the field of organic chemistry, including the Journal of the American Chemical Society, known as "Jacks".

If anyone knows of more pejorative or affectionate names for publishers or journals, I'd be interested in hearing them.

p.s. A play with an anagram tool reveals that BioMed Central becomes melodic banter, metabolic nerd, atomic blender, dental microbe, clambered into, and mint or debacle.

The Evil Empire Strikes Back

Picture, if you will, Darth Vader. Big, black armour. Heavy breathing. Imagine that, tired of being right hand man to the Emperor, Darth Vader has decided to venture into scientific publishing. Imagine what his publishing house would be like.

Elsevier has a reputation as "evil" (as seen here, here, here , here, here, here, here and here, but not here or here). This is mainly due to making massive profits (total revenues for 2005 were £1.4 billion) while restricting access to scientific research. They have also been noted for an involvement in the arms trade, and the censorship of published work.

Open access has been a thorn in Elsevier's side, much as the Rebel Alliance were to the Imperials in Star Wars. Open access threatens the ability of publishers like Elsevier to maintain their hold over library budgets, which is why Elsevier has repeatedly criticised OA (several society publishers have joined in too as they perceive OA to be a threat to their society's income, drafting the Washington DC Principles).

However, Elsevier recently announced a Sponsored Article option on some of their journals. Their hand was effectively forced by CERN, doing in effect what PLoS' boycott failed to do. Elsevier refuse to call it open access, and authors do not retain copyright. They even allowed authors to post preprints, allowing self-archiving. It might have been thought that Elsevier had laid to rest their hostility to open access.

We'd have been wrong. In a brilliant piece of investigative journalism, Nature have revealed a fruitful relationship that Elsevier, Wiley and the American Chemical Society have with a PR guru, advising them on how to take on the open access movement. Not just any guru, but the same one who represented Enron and ExxonMobil. A director at Wiley is quoted as saying that "Media messaging is not the same as intellectual debate". This explains some of the outrageous spin we've seen about open access ("government censorship"; "no peer review").

The blogosphere is beginning to react. The IWR blog understates it somewhat when they say that this will "do little for the reputations of the publishers involved". Chris Leonard predicts that "they won't be able to use these arguments even if they wanted to", while Jonathan Eisen has confidently exclaimed that "Their ship is sinking and they are grabbing at the last little pieces of wood they can find".

Hopefully, this news thoroughly discredits the smears of these publishers against open access. Was this their open exhaust port?

Peer review lite at PLoS ONE?

PLoS ONE, the 'Open Access 2.0' journal trumpeted by the Public Library of Science, launched late last year. Editors and reviewers often make arbitrary decisions about importance, in a chase for the Impact Factor. The idea of removing the need for journals to select the most 'important' science, and instead concentrating on publishing solid, sound science is a good one. This is a philosophy already followed by the BMC-series journals, which I'm involved with, although we do frequently reject articles on the basis that they present no advance in the field. To assess the soundness of a manuscript you usually need to find at least two experts to judge the topic, methods and statistics, and when the journal was announced I was genuinely puzzled as to how PLoS ONE would run their peer review any differently to other journals. There has been debate in the blogosphere by some who were under the impression that PLoS ONE was doing way with peer review entirely. An example I have come across gives me cause to worry that rather than focussing on conducting solid peer review, the system PLoS ONE uses will indeed sometimes scrimp on peer review.

One of the really interesting features of PLoS ONE is the annotation and discussion system. It isn't the first journal to allow readers to post comments, but it is the first to allow them to attach comments to a certain part of the published article, like a post-it note. There is a list of the Most Annotated articles, and on the day I looked one of these was A Large Specific Deterrent Effect of Arrest for Patronizing a Prostitute.

Alongside a comment discussing the use of the term "prostitute" is the Academic Editor's viewpoint. PLoS ONE may be experimental, but they don't have open peer review as standard (i.e. named, rather than anonymous reviewing, with the reports published), so this is not standard practice (they do name the Academic Editor for each article). The editor commented that "Although this manuscript was quite far from my own field of expertise, I accepted to act as academic referee for this manuscript because I felt that it was important that this type of manuscript should be published in a open access mode, and that the possibility for further discussions offered by this new journal would be very positive. Although I am reasonably confident that the scientific content and the statistics performed have been conducted appropriately, this does not mean to say that I condone all that this manuscript contains".

This brought me up short. I handle the peer review of articles on which I am not an expert, but I never make a decision to publish based only on my assessment of the manuscript. This is what peer review is for. The Academic Editor was Etienne Joly, an immunogeneticist who is also a strong supporter of open access - he is also an editorial board member for Biology Direct, another experiment in peer review published by BioMed Central.

I respect Dr Joly, but there is something worrying about an article being accepted after only being assessed by someone who is not a peer of the authors. I'm not sure that an immunologist can assess the conduct and reporting of public health/social science research. This isn't peer review, it is editorial selection. Indeed, PLoS ONE states that:
"AEs [Academic Editors] can employ a variety of methods to reach a decision in which they are confident:

Based on their own knowledge and experience;
Through discussion with other members of the editorial board;
Through the solicitation of formal reports from independent external referees".

Chris Surridge, the Managing Editor of PLoS ONE, has said that "When papers are submitted they get assigned to one of these editors based on the content of the paper and the editor’s specific areas of expertise". Now whilst the board of Academics Editors is impressive, it is still only 200 people. As PLoS ONE has ambitiously opened itself to submissions from across all of science, not just biology and medicine, it is impossible that all submissions will find an Academic Editor who will be an expert. If an Academic Editor is pressed for time (as most academics are) might they not take the easy route and attempt to assess a manuscript themselves that they are not qualified to judge, rather than embarking on the process of selecting external peer reviewers?

Etienne Joly went on to say in his viewpoint that "I have little doubt that this subject will lead to active debates. But this is exactly what PlosOne is about: Open Acess, and open discussions". PLoS ONE appears to be genuinely aiming to replace the pre-publication review process with "community-based open peer review", while at the same time not quite admitting this publicly, arguing that "the pre-publication assessment of papers is definitely ‘peer-review’". What concerns me most about the discussion of peer review surrounding the launch of PLoS ONE is the perception that 'only' assessing the technical quality of a manuscript is somehow easy. It's not. If you don't have the fall back option of claiming that something is "out of scope", "not of interest to our readers", or "more suited to a more specialized journal", then the job of assessing manuscripts actually gets harder, not easier.

Editorial selection is the process already used by the Elsevier journal Medical Hypotheses, which states boldly that "The editor sees his role as a 'chooser', not a 'changer': choosing to publish what are judged to be the best papers from those submitted". It has been said of the journal that it "exists to let people publish their craziest ideas". I would not imagine that this is a reputation that PLoS ONE hopes to emulate.
---

The most annotated article on PLoS ONE (aside from the testing 'Sandbox') now has 10 comments. Wonderful! However, they are all from an author of the article, to external links such as PubChem. Likewise, 5 of the 6 comments on the next most annotated article are links or notes added by the author. An annotation on another article is a note about the correct orientation of a figure. Is that not the sort of thing that is integral to the manuscript and needs correcting in the production process?