Conflicts of interest in the open access debate

I can accept that some society publishers are concerned that open access may make it harder for them to fund their activities. I can accept that some people are confused about the implications of open access.

What I find very difficult to accept is that executives at the American Chemical Society appear to be raising spurious arguments against open access, such as calling it 'Socialized Science', and that they have clear, undeclared conflicts of interest in this debate, namely that they are paid bonuses depending on the profits of the publishing division of the ACS.

Just as we now expect medics to declare their competing interests in journal publications, anyone involved in the open access debate should declare their own competing interests. My own financial competing interest is a fixed salary received from BioMed Central. Let's see some more transparency from all concerned.

A case study in open peer review

Last year, BMC Anesthesiology published an article by Andrew Vickers and colleagues on the use of acupuncture for the pain caused by thoracotomy; it was a pilot study to examine whether a randomized controlled trial (RCT) was feasible. Thoracotomy is surgery to the chest to allow access to the lungs. Andrew Vickers is on the editorial boards for several of our journals, a respected statistician and trial methodology expert with an interest in testing complementary medicines. It was unlikely when he submitted the work that there would be any fatal flaws. However, we don't allow submissions from our editorial boards to escape peer review - and I've seen many manuscripts from editorial board members fail to pass muster.

Who would be suitable to review the manuscript? It's about acupuncture, so acupuncturists, right? Well, partly.

If we were to ask only acupuncturists to review, there are two potential drawbacks. Acupuncturists believe in the efficacy of acupuncture, otherwise they probably wouldn't be acupuncturists. If there were flaws in the study they might be inclined to give it the benefit of the doubt, whereas someone without a vested interest in the intervention under study might raise objections. The other issue is that they might be unfamiliar with pain relief in this particular setting. On the other hand, were we to only approach those who had never used acupuncture and who were otherwise experts in pain relief we would face two potential biases. For one, they might be skeptical that acupuncture works at all and thus be too picky, raising unreasonable objections to block publication. Another consideration is that they might themselves have a vested interest in the drugs used to relieve post-surgical pain, perhaps having received speaking fees or consulted for pharmaceutical companies.

To ensure rigorous and fair review, we needed someone who was familiar with acupuncture for pain relief, preferably with an additional experience of either randomized controlled trials, systematic reviews, or anesthesia for surgical interventions. Although it is not itself an RCT, the purpose of the trial was as a pilot to see if an RCT was feasible, and those who have conducted systematic reviews will have a good knowledge of critical appraisal. Secondly, we needed someone familiar with post-thoracotomy pain relief other than acupuncture, preferably with a knowledge of randomized controlled trials. Lastly, we needed someone with a familiarity with randomized controlled trials and anesthesiology for post-surgical pain, were either of the other two not themselves familiar with it.

BMC Anesthesiology is open access, but it is unusual among journals in another way. It has open peer review, as do all the medical journals in the BMC-series. By this we mean that the reviewers consent to their names being made to be known to the authors, and for their reports to be made public if the manuscript is accepted for publication. Open peer review allows me to sweep back the curtain, and reveal the peer review process, like a magician flaunting the secrecy of Magic Circle.

Among our reviewers were two complementary and alternative medicine experts, Betsy Singh and Hugh MacPherson. Betsy Singh has published on the use of several complementary medicines, including a systematic review of acupuncture for pain relief. Hugh MacPherson has published on ways to ensure the safety and accurate reporting of acupuncture trials, and is familiar with randomized controlled trials. We also had two reviewers who are researchers of pain relief, Deniz Karakaya and Jorge Dagnino. Deniz Karakaya is a thoracic surgeon has published on the use of conventional anesthetics in various procedures, including for post-thoracotomy pain. Jorge Dagnino has also published on the use of anesthetics for post-surgical pain, including thoracotomy. We had the full house.

What criticisms or points did they raise? Dr Singh had no complaints, and it isn't surprising to see a reviewer of a manuscript by Dr Vickers say this. Dr MacPherson was well-disposed to the study, but raised several points where the authors could better report details of their work, or better justify a statement. Dr Karayaka had only two relatively minor criticisms, asking for more detail on their procedures. Dr Dagnino raised the most objections, requiring many more methodological details, and questioning some of the conclusions. It is interesting to see that the two reviewers who asked the authors to make the most corrections were an acupuncturist and a traditional anesthesiologist. Both types of researchers applied their skills of critical appraisal to help the authors improve their work. Upon re-review, Drs Karayaka and Dagnino had some remaining questions, and the editorial staff determined that the authors' response to this second round of review was satisfactory and we proceeded with publication.

Although the study is limited in its scope and conclusions, inevitable for a pilot, uncontrolled study in only 36 patients, and although it would be easy to dismiss by many journals as not interesting enough to publish, we thought it necessary and valuable to have enough qualified reviewers to assess it, and we obtained the advice of four reviewers who together were qualified to judge all the main aspects of the work. Judging soundness (technical validity) isn't easy, and is more difficult than measuring the level of interest. It isn't that uninteresting though - more than 3,000 readers have accessed the article from our website in the past 10 months, and more will have read it on PubMed Central, a level of interest for which many blog writers would willingly give up a kidney.

Response to '10 Problems with the Peer-Review Publishing Process'

Kevin Dewalt's blog on the 19th January includes 10 criticisms of peer review. I've posted a comment on his blog with my response to each of the points, but I'll copy them here as well.

Kevin's original points are italicised, and I've made a couple of additional comments since I replied on his blog that are indicated by square brackets. I hope I've corrected some misconceptions about peer review.

---
1. Unstated real or perceived conflicts of interest. Reviewers and authors can have relationships with entities that have an ulterior motive in getting material published.

True, but many journals, such as mine, require authors and reviewers to declare their competing interests - in our medical journals, these interests are published with the article. Editors are used to watching out for this.
---
2. Peer-review process advances slower than scientific progress.

Yes, but peer review doesn't stop someone first posting their article on their own web-site, discussing their work at conferences, or posting their work on a pre-print server like ArXiv. Anyway, scientific progress isn't as rapid as people believe, and without the checks and balances that peer review gives, all sorts of rubbish would be published, and scientists would have to follow even more blind alleys than they do already after reading profoundly flawed research. Peer review adds some rigour into the process of communicating scientific research. Less haste, more speed is an apt concept here.
---
3. The current process does not provide authors and reviewers with basic collaborative web tools.

That's nothing to do with peer review, just the delays in the Web 2.0 revolution getting to publishers. PLoS ONE (published by Public Library of Science, another OA publisher) does now offer reviewers and authors interactive tools to annotate articles. Many journals, like mine and the BMJ, allow any reader to comment on a published article.
---
4. Authors lose copyright privileges when publishing yet are often forced to publish to continue career advancement.

Traditional journals insist on copyright transfer. Many open access journals, including those published by BioMed Central and PLoS, allow the authors to retain copyright. The article is published under a Creative Commons Attribution License.
---
5. Peer-review networks tend to form around cliques. Those “outside the club” of a particular discipline - where often the best ideas surface - cannot get published because new ideas are rejected by the current establishment. As a result great ideas are often lost.

I don't believe that this complaint is really that valid. The complaints I've read about were by top scientists who couldn't get their idea published in Nature, Cell or Science. Well, just publish it elsewhere. There are plenty of journals that aren't as picky as those journals, and if authors had a little more self-awareness they'd recognise that they aim too high sometimes. Besides, many journals don't use established lists of reviewers, but go straight to those publishing related work and ask them. So, yes, you usually have to be a published scientist to review, but then it is called *peer* review, isn't it? I doubt that "the best ideas" surface outside academic research, the lone researcher is more likely to be a kook than a genius. There are some geniuses out there, but they are the ones you read about in the news - there's a teensy bit of a selection bias going on...
---
6. Precedence is often establish by those with the best personal contacts and not those who first introduce new theories.

I don't see the basis for this argument. Precedence does go to those who first raised a theory, so long as scientists are aware of it [this is the idea of 'priority']. Those who publish in languages other than English are at a disadvantage, admittedly, but some journals allow republication of work in English that was previously published elsewhere in another language, so that gives authors the possibility to widen their audience. Peer reviewers go out of their way to alert authors to work that first demonstrated something, and I have also insisted that authors cite certain studies. Scientists are very attuned to giving due credit for the origin of ideas or techniques.
---
7. There is no medium for wider, instant dissemination. Doctors or researchers who prepare a presentation or speech cannot “publish it” to a wider audience.

Yes, they can. ArXiv and other pre-print servers allow the publication of non-reviewed work (see e.g. Public Knowledge Project). Theses and dissertations can be published electronically (e.g. NDLTD, MIT on DSpace). This Portugese university repository, for example, allows the publication of reports, presentation etc. If a university doesn't have a repository for this kind of material, then it should do! Staff and students can take the lead, rather than waiting for journals to do it for them - journals are traditionally for peer-reviewed research, why would we necessarily expect them to post presentations? That said, Nature has recently launched Nature Protocols, so publishers are making some effort to include material that is outside their usual range.

8. Participating in the review process has little benefit for the reviewer. Performing reviews can take an enormous amount of time and the written reviews are not themselves “published”.

Reviewing takes between 2-6 hours, according a survey I read [an average of 3 hours]. I've seen reviews done in 10 minutes...
Here are a few reasons for participating in peer review:
- Allows a researcher control over what is published in their field - they are the "gatekeepers of science".
- Allows researchers to ensure that what is published accurately reflects and acknowledges their field.
- Can help a scientist get promoted and get grants, as journals often list the names of their reviewers annually.
- In the case of the medical journals in the BMC-series, published by BioMed Central, we *do* publish the reports, along with the name of the reviewer.
- Reviewers are actually paid by a small minority of journals [the BMJ pays £25], and more commonly can get other perks such as discounts on reading or publishing in the journal.
- Reviewers get the opportunity to read their competitors' work months before it will be published, and unscrupulous reviewers can deliberately block publication.
- It's interesting! They're scientists, they enjoy critiquing science!

9. Reviews and reviewers are not “reviewed”. An author who receives a biased review or one based on poor critical thinking has no recourse to publicly respond or invite others to comment.

Not true. Editors assess the reviewer reports and qualifications. Authors who receive what they perceive to be a biased review can appeal to the editor, and request a further opinion. If they are badly treated and the journal is a member of the Committee on Publication Ethics (such as BioMed Central, BMJ, Lancet) then authors can even take a case to that body [currently only editors can submit a case, but often will in cases of a dispute]. Some journals (BMJ, Lancet) have an ombudsman.

10. Journals can be prohibitively expensive for some in the developing world.

Yes - this is one of the reasons why open access is a good idea! The research is free to read for anyone with Internet access. Traditional pay-to-view journals are also members of a scheme called HINARI, a WHO project that allows some people in developing countries to read the research for free (but it does have limitations, as they need to be connected to an institution).