What is WebmedCentral?

Loosely following the style of Jeffrey Beall's assessment in The Charleston Advisor of various OA startups, here is an assessment of WebmedCentral, a new post-publication review biomedical journal.

DESCRIPTION
WebmedCentral is a post-publication review biomedical web portal launched in July 2010. It aims to "eliminate bias, increase transparency, empower authors, improve speed and accountability, and encourage free exchange of ideas." There is no pre-publication screening, although the instructions for authors imply some oversight for issues such as patient consent. Authors may submit revised versions. Articles can be read for free on the website, where they may be reviewed both by reviewers solicited by the authors and by readers. There is a list of "Scholarly Reviewers" on the site. Readers may also rate articles. Biomedical videos are also published. The journal has ISSN 2046-1690, but articles do not appear to have DOIs. It is not indexed in PubMed, but the articles are indexed on Google Scholar. The site aims to host other open access, open peer reviewed journals.

SUMMARY
Content: Primary scientific research, case reports, and reviews make up the bulk of the articles, alongside opinion, hypotheses, and outright fringe science. None have been peer reviewed before publication.

Usability: The site has a category listing, browse by date, featured articles, popular articles, most reviewed articles, RSS feeds, basic and advanced search, latest reviews. The PDF is only available via a Javascript link.

Cost: Free to read and publish, unless the author pays the US$50 Premium Upload fee.

Licensing: Authors retain copyright. Personal non-commercial use, digital archiving and self-archiving are allowed, though no standard license is used and details are confusing

Contact:

Address: Suite 250, 162-168 Regent Street, London W1B 5TD, United Kingdom
Phone: None given
Fax: None given
Email: contact@webmedcentral.com or http://www.webmedcentral.com/Contact_Us
URL: http://www.webmedcentral.com

COST
Free to read and publish, the journal aims to receive income from advertising and sponsorship. They offer a "premium upload service" for $50 per article that allows authors to simply email their submission to the journal. Scholarly Reviewers who post three reviews can obtain a free "premium upload".

LICENSE
The simplest formulation is that "Authors keep copyright to the article but our readers will be freely able to read, copy, save, print and privately circulate the article." However, the details are less clear. At one point they say authors "are free to publish it elsewhere" but also say elsewhere that "we require ... an exclusive license". They also say that users have a "free, irrevocable, worldwide, perpetual right of access for personal non-commercial use, subject to proper attribution of authorship and ownership of rights" but then say users may "view or download a single copy of the material on this website solely for your personal, non-commercial use". But they allow self-archiving: "WebmedCentral allows the final version of all published research articles to be placed in any digital archive immediately on publication. Authors are free to archive articles themselves." The precise freedom all this gives to users to reproduce the text is unclear, but calling WebmedCentral "open access" would be misleading.

EVALUATION
The approach of WebmedCentral is reminiscent of Google Knol, which is where PLoS Currents is hosted, or of a preprint server, except there is an active post-publication peer review system.

Open peer review and community peer review are not new ideas. A similar approach to that of WebmedCentral was tried by Philica in recent years without great success; the site rapidly filled with crank publications. Another was 'E-Biomed', which was stifled and instead became PubMed Central. Although anticipated a decade ago, biomedical publishing has been wary of preprints and other proposals to remove or reduce pre-publication peer review. BioMed Central's Genome Biology had a preprint server, but it closed in January 2006. A humanities institute is experimenting with community review on Shakespeare Quarterly, though they are using a hybrid model rather than abandoning invited pre-publication review. More generally, MediaCommons argue for community peer review in their book "Planned Obsolescence". They are far from naïve, noting that

'Too many digital publishing experiments, like Philica, have lagged due to an assumption that might be summed up as "if you build it, they will come."'

Ethics:
The journal requires appropriate ethical approval for human and animal studies and will remove studies if they find that they fail to meet ethical standards. Articles may also be removed in cases of scientific misconduct or plagiarism. They suggest that authors use statistical advice, and ask authors to adhere to reporting standards such as CONSORT. They ask authors of clinical trials to adhere to the Good Publication Practice guidelines, but do not specifically mention trial registration. They endorse the ICMJE criteria for authorship and the use of medical writers should be declared. Funding and competing interests should be declared, though there is no definition of a competing interest. They ask authors to suggest at least three reviewers and to not only pick "friendly reviewers", and say they may invite further reviewers. How these policies are enforced and who enforces them is not clear.

Technical issues:
Previous versions of an article should be linked to, but this fails. The journal allows digital archiving and digital preservation by LOCKSS members. Some test articles can be found as Word documents that are not visible via the search, which raises questions about site security. The presentations of figures is in a sidebar and sometimes without even a thumbnail, though the pop-up view is user friendly. The referencing could be improved, with clearer formatting and hyperlinks down to the references. Some of the formatting of the reviews is poor, with changes in font and font size, and several reviews are double posted.

Publication volume:
There are 366 published articles as of 30/01/2011. Submission rates appear to have peaked following publicity in August, and have since declined (see figure).

Content:
There is currently no indication on the articles that they have received no pre-publication review. As might be expected given the lack of pre-publication review, some of the articles are fringe science: aliens, homeopathy, prayer, and telepathy are all represented. There is an account of chiropractic care of a patient with fibromyalgia, an opinion article on the evidence for homeopathy in acute upper respiratory tract infections by Peter Fisher and colleagues, a study linking 'emotional quotient' and telepathy that has the obligatory mention of quantum theory, an article on the hunt for alien life that takes in the Higgs Boson, the Bermuda Triangle, and alien implants , a virtually content-free account of acupuncture in rats, and an intercessory prayer study. The latter is, thankfully, a deliberate satire.

When you get this kind of opportunity of publishing without a filter, sex always seem to come to the fore: step forward, a hypothesis on why women don't sleep with the first man they see when they ovulate, two case reports of priapism, an institutional review of Peyronie's disease, and a case report of penile fracture. As pointed out by two reviewers, it contained the unfortunate typo in the title of the corpus callosum (in the brain) rather than the corpus cavernosum, hence it was republished (demonstrating that the article version system is not working).

Many of the articles are unpublishable in any biomedical journal: a rant about academic exploitation; a review of the biological activities of a herb that the author seems to have forgotten to write; an account of a trauma registry that is confused and sketchy; a review of oral health and inequality for which the recommendations section appears to be lifted verbatim from Nunn et al. 2008, who are not cited. How many more of the articles will contain plagiarism would be interesting to see.

On the more positive side, there are a series of interesting articles by three authors: Leonid Perlovsky has published a series of mainly hypothetical papers, e.g. on language and cognition; William Maloney, a New York dentist, has published a series of overviews and historical accounts, e.g. the medical legacy of Babe Ruth; Uner Tan has published a series of articles of his observations and theories of quadrapedal locomotion in humans, e.g. these two cases.

Other interesting reads are a survey of the role of hairdressers and bartenders as informal emotional support following the 9/11 attacks and their responses to this role, a study by Robert Dellavelle on how journals don't require ethics approval for meeting abstracts, and a series of witty anecdotes by an Israeli psychiatrist of cases of "curing demons" in his patients.

Around a quarter of the articles are case reports. The insatiable demand of hospital doctors to publish case reports has clashed with a reluctance of medical journals to publish what are often "me too" publications offering little generalisable insights, and which are often poorly presented and incomplete. The recent trend of open access case report journals - BMJ Case Reports, Cases Journal, Journal of Medical Case Reports, Clinical medicine insights. Case reports, Case Reports in Ophthalmology etc. from Karger, Case reports in Medicine from Hindawi and the American Journal of Case Reports (free, not OA) - doesn't appear to be matching demand.

There are also 58 reviews, 31 opinion articles, and at least 15 of the "original articles" are not research articles; less than half of the articles on WebmedCentral are primary research.

Reviews:
Some of the reviewers are published researchers, but they usually have only a handful of publications and they would be unlikely to be selected as peer reviewers by a mainstream biomedical journal editor – this could be seen as a positive or a negative. There are pages listing reviewer details, but the reviews by a single reviewer are not listed.

Relatively few articles have received an insightful review or comment. Around 55% (201 articles) have received a review of some kind, and the most any article has received is six reviews (see right hand panel of the figure). 138 reviews were unsolicited and 211 were solicited by the authors. The quality of the reviews is usually low. Just over half of both solicited and unsolicited reviews contain critical analysis, i.e. at least some mention of improvements the authors could make to their article, meaning that probably less than 25% of all articles receive any degree of critical analysis. Many reviews are sycophantic, for example one case report is said to be "the best ever article publishe[sic] so far". Many merely state what the articles is about - one author-invited reviewer spends 358 words reiterating what the article says and telling us that it is a "must read" - or give the views of the reviewer on the subject rather than the article - another reviewer devotes a mere 23 words of a 430 word review to even mentioning the paper. Most reviews are very short: the average is only ~115 words for both author-suggested and unsolicited reviewers; the longest is just over 1500 words (see left hand panel of the figure for the length distribution). Comments with critical analysis are much longer (~175 words) than those without (~50 words). If I were to see reviews like most of those on WebmedCentral during standard peer review, I would never use that reviewer again.

Some of the reviews include comments such as "this is suitable for publication" or "I hope it is accepted", which indicate a lack of awareness of the publishing model. One author has even reviewed his own paper. An article I consider unpublishable received the reviews, and I quote them in full, "good" and "No comment".

There are some examples where robust review has taken place. The concerns raised by the reviewers on this paper, including a lack of mention of ethics or consent, would lead most editors to reject such a paper – but WebmedCentral has no routine mechanism for doing this. Authors responded to reviews only on a handful of papers. A lively debate developed around a physician's self-case report, but this was a rare exception. I found one example of what appears to be functional peer review, with the authors revising their work and the reviewer stating that they are happy with the revisions.

In Bambi, Thumper's parents taught him that "If you can't say something nice... don't say nothing at all", but I think that the opposite applies in peer review. If you can't come up with critical comments about a paper, you're probably missing something: every paper has something wrong with it. The sycophantic nature of many of the reviews in WebmedCentral might be inherent to open (named) peer review, but in my experience and according to published studies, open peer review increases the length of reviews and makes them more polite, but has no effect on review quality. Another factor may be that many of the authors and reviewers of WebmedCentral are from India: R. A. Mashelkar argued in Science that "India must free itself from a traditional attitude that condemns irreverence", and Nikhil Kumar and Shirish Ranade argued in Current Science that "it is a preponderance of obsequious reverence and sycophancy that has placed the science in the country on a downhill slope." Are we seeing this unwillingness to criticise in action?

Overall assessment:
This is an interesting experiment in post-publication peer review, which both indicates the possibilities – instant publication, open community review – and the perils – unsound science, unbalanced opinion, and substandard writing being presented as part of the scientific literature.

Building a functioning publishing platform from scratch is no easy matter, and several hundred publications in seven months is an impressive figure. There has been a noticeable engagement from the community, with over 365 submissions and a total of nearly 350 reviews in seven months, 40% of them by reviewers not suggested by the authors. However, the submission rate is declining and the coverage and quality of reviews is not nearly high enough to functionally replace pre-publication review.

The onus is on the authors to obtain reviews: the journal states that it will obtain reviews, but this is not in evidence - just under half of the papers have no reviews, and 30% have only one review. More effort needs to be put into gaining reviews from qualified experts.

Reviews are essentially worthless if nobody pays any attention to them, be that an editor, the authors or the readers. Pre-publication peer review is not merely a filter, but it also acts to improve articles. On WebmedCentral there is no pressure for articles to be revised in accordance with any critical reviews, perhaps other than author embarrassment. As reviewers see a lack of response to their comments, they may lose enthusiasm.

Without a clear indication that reviewers have criticised an article and no indication that the articles are not peer reviewed, readers may view the work uncritically. If reviewers state for instance that the work is not sound, this should be clearly flagged up to readers near the top of the page, and articles should be sortable based on the answers given in the review from and the rating given by reviewers and readers. Another layer should be added, allowing articles to be promoted by agreement from their 'Scholarly Reviewers' to a "publication standard" level, giving authors an incentive to revise their work. "Featured articles" do exist, but the criteria used are not revealed. WebmedCentral are forming an "Advisory Board" of "eminent scientists"; perhaps this board will increase the rigour of the site.

Without the oversight of an editor choosing diverse reviewers and because most scientists are unaware of the site, it may become a closed community of the same authors positively reviewing each others' work – the precise opposite of the aim of the journal. Unless the process is reformed, WebmedCentral is likely to remain a "Cargo Cult science" journal, which in the main publishes articles that only superficially resemble the peer-reviewed literature, and that are reviewed in a manner that is only a pale imitation of pre-publication peer review.

Other commentary on WebmedCentral:

WebmedCentral and ginger pee on Pleion, a discussion on Ramy Aziz's FriendFeed, a blog review, a response to receiving an email from them, a brief welcome, a mention by Jenny Rohn in a Guardian comment thread, a comment by an author, an assessment in a sports science magazine, a mention in Lab Times, a critical appraisal of an article by PJ of Pyjamas in Bananas, a blog reaction in French, and a few Twitter comments.

But who runs this site?
"We are a group of medical and management professionals with no affiliation to any major biomedical publishing group" is all they say, but who runs the site shall be revealed in my next post: "Who are WebmedCentral?".

Publisher and journal nicknames

A previous handling editor of BMC Ear, Nose and Throat Disorders affectionately (?) referred to the journal as BMC Wax, Snot and Phlegm, and rumour has it that our independent journal Cough was almost called Cough and Phlegm. A blog going by the name wax, boogers, and phlegm actually exists.

The Public Library of Science (PLoS) is known by editors at the Lancet as SLoP, and I'm sure that the nicknames Evil Empire and Elseviley need no introduction. In the Pipeline lists the colloquial ways of referring to several journals in the field of organic chemistry, including the Journal of the American Chemical Society, known as "Jacks".

If anyone knows of more pejorative or affectionate names for publishers or journals, I'd be interested in hearing them.

p.s. A play with an anagram tool reveals that BioMed Central becomes melodic banter, metabolic nerd, atomic blender, dental microbe, clambered into, and mint or debacle.

Declaration of Pomposity, and a Declaration of War?

Ahead of the EC Conference on Scientific Publishing in the European Research Area, a group of publishers have released what they dub the 'Brussels Declaration', which shamelessly echoes the Berlin Declaration on Open Access, the Budapest Open Access Initiative and the Bethseda Statement on Open Access Publishing. Richard Charkin, who works for Macmillan, has irreverently nicknamed it the Bognor Declaration, which will hopefully take some of the wind out of their sails.

They have stated that "we have decided to publish a declaration of principles which we believe to be self-evident". Oh, for pity's sake! Using language that apes the US Declaration of Independence makes them sound very pompous. Is it really 'self-evident' that "Copyright protects the investment of both authors and publishers"? Insisting on transfer of copyright to the publisher certainly helps maintain publisher profits, but how does that help the authors? The statement that "authors should be free to choose where they publish in a healthy, undistorted free market" is plainly a dig at PLoS (who have received philanthropic funding), but since when was journal publishing an 'undistorted free market'? The many mergers over the years have led towards the creation of monopolies.

The real target of this declaration is self-archiving. Stevan Harnad, the "archivangelist", optimistically commented on the IWR Blog that "There will be no war in Brussels. The meeting is about online access to European research findings. The European research community is meeting to decide how to maximise access, usage and impact for its research findings. The answer -- the very same answer -- has already been proposed by the European Commission [...] As a condition for receiving public research funding, all funded researchers should self-archive the resulting research publications online in an Open Access Repository, free for all would-be users".

In other news, Prime Minister Neville Chamberlain has declared that there will be "peace for our time". Um, perhaps not. The ninth "principle" in this declaration is that "Open deposit of accepted manuscripts [self-archiving] risks destabilising subscription revenues and undermining peer review [...] Free availability of significant proportions of a journal’s content may result in its cancellation and therefore destroy the peer review system upon which researchers and society depend". I sense that hostilities have started... Perhaps soon the publisher tanks will be parked on the University of Southampton's driveway?

Overall this declaration makes these publishers look self-satisfied and a bit silly. I'd have expected better from the BMJ Group, who are one of the signatories.

Peer review lite at PLoS ONE?

PLoS ONE, the 'Open Access 2.0' journal trumpeted by the Public Library of Science, launched late last year. Editors and reviewers often make arbitrary decisions about importance, in a chase for the Impact Factor. The idea of removing the need for journals to select the most 'important' science, and instead concentrating on publishing solid, sound science is a good one. This is a philosophy already followed by the BMC-series journals, which I'm involved with, although we do frequently reject articles on the basis that they present no advance in the field. To assess the soundness of a manuscript you usually need to find at least two experts to judge the topic, methods and statistics, and when the journal was announced I was genuinely puzzled as to how PLoS ONE would run their peer review any differently to other journals. There has been debate in the blogosphere by some who were under the impression that PLoS ONE was doing way with peer review entirely. An example I have come across gives me cause to worry that rather than focussing on conducting solid peer review, the system PLoS ONE uses will indeed sometimes scrimp on peer review.

One of the really interesting features of PLoS ONE is the annotation and discussion system. It isn't the first journal to allow readers to post comments, but it is the first to allow them to attach comments to a certain part of the published article, like a post-it note. There is a list of the Most Annotated articles, and on the day I looked one of these was A Large Specific Deterrent Effect of Arrest for Patronizing a Prostitute.

Alongside a comment discussing the use of the term "prostitute" is the Academic Editor's viewpoint. PLoS ONE may be experimental, but they don't have open peer review as standard (i.e. named, rather than anonymous reviewing, with the reports published), so this is not standard practice (they do name the Academic Editor for each article). The editor commented that "Although this manuscript was quite far from my own field of expertise, I accepted to act as academic referee for this manuscript because I felt that it was important that this type of manuscript should be published in a open access mode, and that the possibility for further discussions offered by this new journal would be very positive. Although I am reasonably confident that the scientific content and the statistics performed have been conducted appropriately, this does not mean to say that I condone all that this manuscript contains".

This brought me up short. I handle the peer review of articles on which I am not an expert, but I never make a decision to publish based only on my assessment of the manuscript. This is what peer review is for. The Academic Editor was Etienne Joly, an immunogeneticist who is also a strong supporter of open access - he is also an editorial board member for Biology Direct, another experiment in peer review published by BioMed Central.

I respect Dr Joly, but there is something worrying about an article being accepted after only being assessed by someone who is not a peer of the authors. I'm not sure that an immunologist can assess the conduct and reporting of public health/social science research. This isn't peer review, it is editorial selection. Indeed, PLoS ONE states that:
"AEs [Academic Editors] can employ a variety of methods to reach a decision in which they are confident:

Based on their own knowledge and experience;
Through discussion with other members of the editorial board;
Through the solicitation of formal reports from independent external referees".

Chris Surridge, the Managing Editor of PLoS ONE, has said that "When papers are submitted they get assigned to one of these editors based on the content of the paper and the editor’s specific areas of expertise". Now whilst the board of Academics Editors is impressive, it is still only 200 people. As PLoS ONE has ambitiously opened itself to submissions from across all of science, not just biology and medicine, it is impossible that all submissions will find an Academic Editor who will be an expert. If an Academic Editor is pressed for time (as most academics are) might they not take the easy route and attempt to assess a manuscript themselves that they are not qualified to judge, rather than embarking on the process of selecting external peer reviewers?

Etienne Joly went on to say in his viewpoint that "I have little doubt that this subject will lead to active debates. But this is exactly what PlosOne is about: Open Acess, and open discussions". PLoS ONE appears to be genuinely aiming to replace the pre-publication review process with "community-based open peer review", while at the same time not quite admitting this publicly, arguing that "the pre-publication assessment of papers is definitely ‘peer-review’". What concerns me most about the discussion of peer review surrounding the launch of PLoS ONE is the perception that 'only' assessing the technical quality of a manuscript is somehow easy. It's not. If you don't have the fall back option of claiming that something is "out of scope", "not of interest to our readers", or "more suited to a more specialized journal", then the job of assessing manuscripts actually gets harder, not easier.

Editorial selection is the process already used by the Elsevier journal Medical Hypotheses, which states boldly that "The editor sees his role as a 'chooser', not a 'changer': choosing to publish what are judged to be the best papers from those submitted". It has been said of the journal that it "exists to let people publish their craziest ideas". I would not imagine that this is a reputation that PLoS ONE hopes to emulate.
---

The most annotated article on PLoS ONE (aside from the testing 'Sandbox') now has 10 comments. Wonderful! However, they are all from an author of the article, to external links such as PubChem. Likewise, 5 of the 6 comments on the next most annotated article are links or notes added by the author. An annotation on another article is a note about the correct orientation of a figure. Is that not the sort of thing that is integral to the manuscript and needs correcting in the production process?