Open access: the saviour for Chinese journals?

Discussing the announcement that the Chinese government is going to crack down on poor quality journals, a Nature editorial puts forward the welcome view that moving towards open access might be the best approach for Chinese publishers:

"The best opportunity to revive Chinese publishing, whether in Chinese or English, probably lies in an open-access platform — increasingly popular in Western journals. Many Chinese journals already charge authors a publication fee, so should be able to make a smooth transition to the open-access model, in which they are supported by fees rather than by subscription revenues."

Bioinformation Journal: making sure "open access" means open access

When looking at the journal Bioinformation, I spotted the tagline

"It is free to publish, open access and online immediately upon acceptance"

This is wrong, as like the BioMed Central journals this journal usually charges an article processing charge. This got me digging a bit further.

They state on the published articles that

"This is an open-access article, which permits unrestricted use, distribution, and reproduction in any medium, for non-commercial purposes, provided the original author and source are credited"

OK, although a non-commercial license isn't great, as Peter Murray-Rust has explained. Yet when I tried to right click on the page, a notice popped up:

"Sorry! Copying of these contents for potential editing and reproduction is not permitted. However, you have access to read, know and print contents for non-commercial processes".

This is not consistent with open access; it is a restriction of reuse. It also prevents 'fair use'.

The journal states that

"The authors of published articles in Bioinformation automatically transfer the copyright to the publisher upon formal acceptance. However, the authors reserve right to use the information contained in the article for non commercial purposes"

It is usual among open access journals that the authors retain copyright, and a Creative Commons Attribution license is applied. As the Budapest Statement on Open Access put it "The only constraint on reproduction and distribution, and the only role for copyright in this domain, should be to give authors control over the integrity of their work and the right to be properly acknowledged and cited".

I emailed the editors and received a brief reply from Prof Kangueane. Now, a couple of weeks later, the tagline reads

"It is open access and online immediately upon acceptance"

Excellent. The right-click restriction also appears to be gone. I will leave it up to their authors and editors to lobby them to change from a CC-BY-NC to a CC-BY license, but it seems that Biomedical Informatics Publishing Group and Bioinformation are now doing things right.

Funding for publication ethics research

The Committee on Publication Ethics (COPE) has established a Grant Scheme to fund research in the field of publication ethics. The Scheme is designed to provide financial support to any member of COPE for a defined research project that is in the broad area of the organisation’s interests, and specifically in the area of ethical standards and practice in biomedical publishing.

The project should have a specific goal and be intended to form the kernel of a future publication. A maximum sum of £5000 will be allocated to any one project, but applications for smaller sums are welcomed.

The terms and conditions of the Grant are as follows:
1. At least one of the applicants must be a member of COPE.
2. Calls for applications will be made twice a year with closing dates of 1 December and 1 June. An electronic version of the application form must be sent to the Administrator no later than 12 pm (noon GMT) on the closing date for consideration by COPE Council.
3. The application must contain a lay summary of the project, a definition of the question to be posed, sufficient methodological detail to allow assessment of the viability of the project, a clear timeline and a definition of the likely deliverables. A full justification for the sum requested must accompany the application.
4. A report on the progress of the research should be presented within one year of the award and at the end of the project. The grant must be used within two years from the date of award, and balance sheets must be forwarded annually. These should be sent to the Administrator. Any remaining funds after two years must be returned.
5. It is anticipated that the work stemming from the project will be presented at one of COPE’s annual seminar meetings within 2–3 years of the award. Such data may also be published in peer-reviewed journals. Any publications or related presentations at meetings by the recipient emanating in part or whole from COPE’s support should be duly acknowledged and copies sent to the Administrator.

Applications are reviewed by a COPE sub-committee. Applicants will be advised of a decision as soon as practicable after the deadline date.

An application form can be obtained by contacting Linda Gough, COPE administrator, at LGough@bmj.com or 020 7383 6602.

The closing date for receipt of applications is 1 June 2008.

---

And who made that banner ad? Yours truly.

Archivangelism - has the means become the end?

Stevan Harnad has always been insistent on the need for immediate, free access to academic research, and he sees self-archiving as the means to this end.

Now that he recognises that self-archiving may only be compatible with some publishers if there is a delay in access, Stevan (who is normally uncompromising, e.g. "OA itself is non-negotiable") seems to have accepted this fudge, which is not immediate free access: "Access to the immediate deposit can then either be set as Open Access immediately, or (in case of a publisher embargo), as Closed Access, provisionally". This is the "Immediate-Deposit & Optional-Access" (IDOS) policy. Even with a fancy name, and as Jan Velterop has noted, it's not open access.

Stevan has been adamantly against the crystal-ball-gazing that predicts a loss in subscriptions resulting from self-archiving, but his own crystal ball predicts that following a universal adoption of 'IDOS' repositories, "Embargoes will disappear very soon thereafter" [my emphasis].

Stevan has criticised advocates of open access journals who claim that open access journals will reduce costs, as he insists that it is access that is paramount and not costs, which is a fair point. Yet he now criticises open access journals and hybrid journals for the extra costs he says they impose, for example criticising CERN's decision to put up funds to pay for article processing charges as "diverting scarce research funds from research to paying publishers". It seems that if open access journals might save costs, that's a side issue; if they might cost more, we should take heed.

While there might be issues with double-payment in hybrid journals, that can be corrected by adjusting subscription costs - it isn't an insurmountable issue. Further, open access journals don't have this problem, but Stevan's criticisms of Springer Open Choice don't often allow for this distinction. How open access can be paid for is open to debate - the debate has usually focussed on the article processing charge, but as Peter Suber has noted not all open access journals charge author fees. Costs can also be met using advertising (though hardly an uncontroversial way for a medical journal to recover costs), grants from societies (this is how the Beilstein Journal of Organic Chemistry is funded), charitable and philanthropic donations, or even by cutting costs. An interesting aspect of article processing charges is that they can result in price competition on an article-by-article basis.

Rather than being a system with no barriers to access (the definition of open access as I understand it), under self-archiving each author (having signed over copyright to the publisher) needs to deposit their articles in their local repository (if one exists), then each reader needs to realise that the repository exists, find the article, and then possibly have to contact the author to get a copy, and then have the author respond and send it to them. Sadly, this seems to be laborious, incomplete and prone to failure. I had thought that the launch of Google Scholar opened up the possibility of self-archiving really being viable, but their practice of linking to all copies of an article that could be found on the web only lasted a few short months and links to free versions are now intermittent - possibly (probably), Google were sat on by publishers.

Libraries will apparently continue to subscribe to journals that their users can access at no cost, despite evidence that libraries are acutely attuned to cost, and an existing trend of university actions against high journal prices. Publishers will apparently be happy to have a business model that depends on their customers paying for a product that can be obtained for free. This business model is actually seen in shareware, though shareware is much rarer now than 10 to 20 years ago, and certainly the music industry isn't too keen on this business model. If self-archiving doesn't cause a collapse in subscriptions to closed access journals, I'd suggest that it will implicitly have failed to achieve its goal. Surely the aim is to provide immediate free access to peer reviewed academic research - if libraries and readers are unaware that they could get what they are still paying for at no cost, might that not imply that self-archiving doesn't provide universal access? Might there be those who aren't paying and who don't realise that the research is accessible, and therefore never read the article? We need to end this farcical situation of researchers not reading articles, and although I like the idea of self-archiving I don't believe that if offers a complete enough solution, or is sustainable.

Stevan's insistence on self-archiving has even extended to criticising central deposit in PubMed Central, arguing that articles must only be deposited in institutional repositories. What is BioMed Central to do; stop depositing in PubMed Central? It's hardly as though we block our authors depositing in their own local repositories.

Stevan lays claim to being the true voice of open access. In responding to an article by Ben Goldacre he argued that "It is not "two [Gold] OA publishing organisations" that have led the fight for OA, but one (Green and Gold) organisation -- the same one that first coined the term OA in 2002: the Budapest Open Access Initiative (BOAI)". Actually, it's not true that BOAI coined 'open access'.

Stevan at times appears to be entirely opposed to the idea of open access journals (despite apparently supporting the idea a decade ago), for example raising criticisms against BioMed Central as a publisher from the outset.

I can't agree with Stevan's insistence upon local institutional self-archiving to the apparent exclusion of other approaches to open access. It appears to me that with archivangelism the means has become the end.

Publisher and journal nicknames

A previous handling editor of BMC Ear, Nose and Throat Disorders affectionately (?) referred to the journal as BMC Wax, Snot and Phlegm, and rumour has it that our independent journal Cough was almost called Cough and Phlegm. A blog going by the name wax, boogers, and phlegm actually exists.

The Public Library of Science (PLoS) is known by editors at the Lancet as SLoP, and I'm sure that the nicknames Evil Empire and Elseviley need no introduction. In the Pipeline lists the colloquial ways of referring to several journals in the field of organic chemistry, including the Journal of the American Chemical Society, known as "Jacks".

If anyone knows of more pejorative or affectionate names for publishers or journals, I'd be interested in hearing them.

p.s. A play with an anagram tool reveals that BioMed Central becomes melodic banter, metabolic nerd, atomic blender, dental microbe, clambered into, and mint or debacle.

Journal bundling

On the SPARC Open Access Forum, Sally Morris responded to criticism of journal bundling by arguing that "I know of no publishers (please correct me if I’m wrong) who only offer their journals as part of a ‘big deal’ bundle. Is anyone taking food manufacturers to task for not offering the same value in individual servings as in family packs?".

The family pack analogy is flawed. Imagine if you wanted to buy food and groceries (let's say bread and milk), but when you handed over your money to get what you wanted the shopkeeper insisted on you also paying for some nappies (when you didn't have a child) and some denture cream (when you still have all your teeth)... well, you'd go to another shop. No shop has a monopoly on bread and milk (at least not yet). The difference with journals is that if you want access to particular articles in one particular journal, you *can't* go to another journal to read them. That's why bundling is criticised as being anti-competitive.

Billions of genes, billions of articles?

We receive, and sometimes publish, manuscripts that "clone and characterize" a new gene in a species. The gene is usually identified from a cDNA library and is confirmed as present in the genomic DNA by PCR. The expression is measured using RT-PCR, and perhaps a phylogeny is drawn that relates this gene to others within the same species and to its relatives in other species.

There are something in the order of 1.5 million known species. A reasonable guess of the average number of genes per species is 5,000 (bacteria are in the range 500-4,000, eukaryotes 5,000-40,000). This gives the number of genes waiting to be cloned and characterized in known species alone at 7.5 billion, more than one for every woman, man and child alive today.

I think this illustrates quite well Mark Gerstein's recent comment in an article in BMC Bioinformatics that "academic journals alone cannot capture the findings of modern genome-scale inquiry". Philip Bourne has similarly said in the pages of PLoS Computational Biology that "Clearly, no one perceives a database entry of, say, a sequence, or a specimen in a museum collection, as being as valuable as the journal paper that describes it. But, ironically ... the database entry may indeed be more valuable".

Does peer review work?

There are now a reasonable numbers of studies from journals such as the BMJ and JAMA on the factors affecting peer review. For example, we know due to a piece of work done by my colleagues that while author-suggested reviewers appear to return reports of equal quality to editor-suggested reviewers, they are significantly kinder to the authors in their recommendations on publication.

One of those authors, Pritt Tamber, regularly makes clear his belief that peer review doesn't work, most recently arguing in a BMJ Rapid Response that "Much of the research conducted at the BMJ [...] showed that there is little or no objective value to the process, yet journals and their editors persist with—and advocate—peer review; their only defence is that "there's nothing better," even though few have tried to find an alternative".

As Pritt notes, one alternative is the system used by Biology Direct, published by BioMed Central. The idea is that authors obtain reviews from three members of the reviewing board. If the author cannot find three members of the board to agree (or to themselves solicit an external review) the manuscript is considered to be rejected. If they can get three reports, then the manuscript will be published, no matter what the reviewers say. The twist is that the comments of the reviewers will be included at the end of the manuscript, as an integral part of the manuscript, and signed by the reviewers. The author can make revisions to the manuscript if they wish or even withdraw it, but equally they can ignore the comments and publish despite them. This is with the knowledge that readers will be able to see the reviewers' dissent. Other alternatives include the community peer review being tried by Philica, PLoS ONE and Nature (Nature's experiment appears to have been unsuccessful, but that is no reason to write-off the idea). More journals, publishers and researchers need to go out on a limb to explore new and better ways to assess and critique scientific research.

Before we go too far with condemning peer review, it is worth remembering that without an evidence base, we won't be able to work out where peer review works, where it doesn't and why, and how to improve it.

Much of the research done into the effects of peer review has been, in my opinion at least, quite superficial. Reading it has really only told me what I knew already from working as an editor.

My wish-list for studies of peer review are:

1. Creating a metric of "on-topicness" that editors can use to assess how relevant a reviewer's expertise is to a piece of research or an aspect of that work. This could be by simple similarity analyses, comparing their PubMed abstracts to the abstract of the submitted manuscript, or by more complicated semantic analyses.
2. Comparing manuscripts that were accepted to those rejected to examine the predictive factors. Some of these have been done, but the analyses always strike me as simplistic. The sample size needs to be greater, and the journals chosen need to not be so highly selective - is it really that interesting to see the factors that influence publication in journals like Nature, The Lancet or NEJM? I really want to see are the factors that affect whether a study is ever published in a reputable journal.
   
3. A side-by-side comparison of published articles with the original submitted version (before any peer review in any journal). This could be done by a paid panel who would be able to spend the time to do an in depth analysis; an alternative would be to invite journal clubs at universities worldwide to analyse manuscripts in this way (a sort-of Seti@Home for journalology). Did peer review noticeably improve the work?
   
4. Examine the fate of articles rejected by journals. Several studies of this nature have been conducted, but they mainly focus only on the journal it is eventually published in and the Impact Factor of the publishing journal. Why not examine whether any changes had been made since rejection? What about whether the rejected work is cited and read? Do a panel and journal clubs agree that the work is now sound, even if it might be uninteresting?
5. Compare the ability of different editors to assess a manuscript and select appropriate reviewers under time pressure, pitted against some of the new semi-automated tools available, like etBLAST. This would be like a peer review Olympiad.
   

It is tough to design and conduct good studies to examine peer review, but editors need to make the effort, else skeptics like Pritt will have a point. Now, just as soon as I have some spare time...

Why reviewers decline, and paying for peer review

"Reviewers are more likely to accept to review a manuscript when it is relevant to their area of interest. Lack of time is the principal factor in the decision to decline". This nugget comes from a study by Sara Schroter of the BMJ (Tite L, Schroter S: Why do peer reviewers decline to review? A survey. J Epidemiol Community Health 2007, 61(1):9-12).

Well, blow me down with a feather - it had never occurred to me that reviewers might decline if they were off-topic or busy...

What is more interesting is that their respondents doubt that paying reviewers would make them more likely to review when their time was constrained, and they suggest various other non-financial perks for reviewers, such as being publicly acknowledged, or joining the editorial board.

I'm not sure that I agree that payment would fail to act as an incentive, but I do have doubts that journals should move to making payments. Those journals that do pay reviewers, such as the Lancet sometimes does, find it easier to get people to agree than those that don't. This is confounded by the prestige of the Lancet, but the extra cash can't harm their chances. What the respondents appear to have forgotten is that every reviewer is asked to review by several journals on a regular basis. If one routinely offers financial compensation, and the others don't, the paying journal will be the more attractive choice. Once enough journals began to pay reviewers, those that didn't would begin to notice their declining success rate, and feel the need to switch to paying (this is classic Game Theory). Payment would no longer help journals to obtain reviewers more easily than their competitors, but no journal could opt out for fear of losing reviewers.

Another issue with paying reviewers is that quite often reports are returned late, and may be of low quality. Payment could be tied to the report being delivered on time, but if reviewers were used to receiving payment, the incentive to then return the report once already late and without payment would be diminished. Payment could be tied to review quality, but using the Review Quality Instrument on every report would be laborious, and from speaking to someone who has used this rating tool it appears to be less than perfect. Currently editors send invitations to some reviewers who reply that they are off-topic or not qualified to review. Would the promise of payment fog the memory of some as to whether they were a suitable reviewer?

The payment of reviewers is also connected to a promise to fast-track. The Journal of Medical Internet Research offers authors the option of paying a fast-track fee (currently $350), of which part is used to pay reviewers to return reports rapidly. My concern about such a promise of speed is that it conflicts with the job of an editor to ensure a high-quality review process. Although the standard number of reviewers is 2, quite often because editors invite more than 2 reviewers at a time more will agree to review. Normally an editor will be glad of the extra advice (authors may be less keen). If each peer reviewer needs to be paid or a very rapid decision needs to be made, editors will be less inclined to keep more agreed reviewers. If there is the need to seek further advice to resolve a certain issue, an editor might rather simply reject the manuscript rather than pay for and wait for another report.

The House of Commons Science and Technology Select Committee when reporting on open access in 2004, stated their belief that “the introduction of modest incentives for peer reviewers is an imaginative way of rewarding the contribution of peer reviewers to scientific endeavour. By carrying out reviews, researchers add value to the services provided by publishers. Whilst it would be inappropriate to pay reviewers personally, some recognition, made to their department, of the value of their contribution would be welcomed, particularly in view of the fact that many researchers are paid from public funds". I agree that, if it comes to it, a payment to the reviewer's institution would be preferable to direct payments to individual reviewers.

Peer review lite at PLoS ONE?

PLoS ONE, the 'Open Access 2.0' journal trumpeted by the Public Library of Science, launched late last year. Editors and reviewers often make arbitrary decisions about importance, in a chase for the Impact Factor. The idea of removing the need for journals to select the most 'important' science, and instead concentrating on publishing solid, sound science is a good one. This is a philosophy already followed by the BMC-series journals, which I'm involved with, although we do frequently reject articles on the basis that they present no advance in the field. To assess the soundness of a manuscript you usually need to find at least two experts to judge the topic, methods and statistics, and when the journal was announced I was genuinely puzzled as to how PLoS ONE would run their peer review any differently to other journals. There has been debate in the blogosphere by some who were under the impression that PLoS ONE was doing way with peer review entirely. An example I have come across gives me cause to worry that rather than focussing on conducting solid peer review, the system PLoS ONE uses will indeed sometimes scrimp on peer review.

One of the really interesting features of PLoS ONE is the annotation and discussion system. It isn't the first journal to allow readers to post comments, but it is the first to allow them to attach comments to a certain part of the published article, like a post-it note. There is a list of the Most Annotated articles, and on the day I looked one of these was A Large Specific Deterrent Effect of Arrest for Patronizing a Prostitute.

Alongside a comment discussing the use of the term "prostitute" is the Academic Editor's viewpoint. PLoS ONE may be experimental, but they don't have open peer review as standard (i.e. named, rather than anonymous reviewing, with the reports published), so this is not standard practice (they do name the Academic Editor for each article). The editor commented that "Although this manuscript was quite far from my own field of expertise, I accepted to act as academic referee for this manuscript because I felt that it was important that this type of manuscript should be published in a open access mode, and that the possibility for further discussions offered by this new journal would be very positive. Although I am reasonably confident that the scientific content and the statistics performed have been conducted appropriately, this does not mean to say that I condone all that this manuscript contains".

This brought me up short. I handle the peer review of articles on which I am not an expert, but I never make a decision to publish based only on my assessment of the manuscript. This is what peer review is for. The Academic Editor was Etienne Joly, an immunogeneticist who is also a strong supporter of open access - he is also an editorial board member for Biology Direct, another experiment in peer review published by BioMed Central.

I respect Dr Joly, but there is something worrying about an article being accepted after only being assessed by someone who is not a peer of the authors. I'm not sure that an immunologist can assess the conduct and reporting of public health/social science research. This isn't peer review, it is editorial selection. Indeed, PLoS ONE states that:
"AEs [Academic Editors] can employ a variety of methods to reach a decision in which they are confident:

Based on their own knowledge and experience;
Through discussion with other members of the editorial board;
Through the solicitation of formal reports from independent external referees".

Chris Surridge, the Managing Editor of PLoS ONE, has said that "When papers are submitted they get assigned to one of these editors based on the content of the paper and the editor’s specific areas of expertise". Now whilst the board of Academics Editors is impressive, it is still only 200 people. As PLoS ONE has ambitiously opened itself to submissions from across all of science, not just biology and medicine, it is impossible that all submissions will find an Academic Editor who will be an expert. If an Academic Editor is pressed for time (as most academics are) might they not take the easy route and attempt to assess a manuscript themselves that they are not qualified to judge, rather than embarking on the process of selecting external peer reviewers?

Etienne Joly went on to say in his viewpoint that "I have little doubt that this subject will lead to active debates. But this is exactly what PlosOne is about: Open Acess, and open discussions". PLoS ONE appears to be genuinely aiming to replace the pre-publication review process with "community-based open peer review", while at the same time not quite admitting this publicly, arguing that "the pre-publication assessment of papers is definitely ‘peer-review’". What concerns me most about the discussion of peer review surrounding the launch of PLoS ONE is the perception that 'only' assessing the technical quality of a manuscript is somehow easy. It's not. If you don't have the fall back option of claiming that something is "out of scope", "not of interest to our readers", or "more suited to a more specialized journal", then the job of assessing manuscripts actually gets harder, not easier.

Editorial selection is the process already used by the Elsevier journal Medical Hypotheses, which states boldly that "The editor sees his role as a 'chooser', not a 'changer': choosing to publish what are judged to be the best papers from those submitted". It has been said of the journal that it "exists to let people publish their craziest ideas". I would not imagine that this is a reputation that PLoS ONE hopes to emulate.
---

The most annotated article on PLoS ONE (aside from the testing 'Sandbox') now has 10 comments. Wonderful! However, they are all from an author of the article, to external links such as PubChem. Likewise, 5 of the 6 comments on the next most annotated article are links or notes added by the author. An annotation on another article is a note about the correct orientation of a figure. Is that not the sort of thing that is integral to the manuscript and needs correcting in the production process?