31 Jan 2011

What is WebmedCentral?

Loosely following the style of Jeffrey Beall's assessment in The Charleston Advisor of various OA startups, here is an assessment of WebmedCentral, a new post-publication review biomedical journal.

WebmedCentral is a post-publication review biomedical web portal launched in July 2010. It aims to "eliminate bias, increase transparency, empower authors, improve speed and accountability, and encourage free exchange of ideas." There is no pre-publication screening, although the instructions for authors imply some oversight for issues such as patient consent. Authors may submit revised versions. Articles can be read for free on the website, where they may be reviewed both by reviewers solicited by the authors and by readers. There is a list of "Scholarly Reviewers" on the site. Readers may also rate articles. Biomedical videos are also published. The journal has ISSN 2046-1690, but articles do not appear to have DOIs. It is not indexed in PubMed, but the articles are indexed on Google Scholar. The site aims to host other open access, open peer reviewed journals.

Content: Primary scientific research, case reports, and reviews make up the bulk of the articles, alongside opinion, hypotheses, and outright fringe science. None have been peer reviewed before publication.

Usability: The site has a category listing, browse by date, featured articles, popular articles, most reviewed articles, RSS feeds, basic and advanced search, latest reviews. The PDF is only available via a Javascript link.

Cost: Free to read and publish, unless the author pays the US$50 Premium Upload fee.

Licensing: Authors retain copyright. Personal non-commercial use, digital archiving and self-archiving are allowed, though no standard license is used and details are confusing

Address: Suite 250, 162-168 Regent Street, London W1B 5TD, United Kingdom
Phone: None given
Fax: None given
Email: contact@webmedcentral.com or http://www.webmedcentral.com/Contact_Us
URL: http://www.webmedcentral.com

Free to read and publish, the journal aims to receive income from advertising and sponsorship. They offer a "premium upload service" for $50 per article that allows authors to simply email their submission to the journal. Scholarly Reviewers who post three reviews can obtain a free "premium upload".

The simplest formulation is that "Authors keep copyright to the article but our readers will be freely able to read, copy, save, print and privately circulate the article." However, the details are less clear. At one point they say authors "are free to publish it elsewhere" but also say elsewhere that "we require ... an exclusive license". They also say that users have a "free, irrevocable, worldwide, perpetual right of access for personal non-commercial use, subject to proper attribution of authorship and ownership of rights" but then say users may "view or download a single copy of the material on this website solely for your personal, non-commercial use". But they allow self-archiving: "WebmedCentral allows the final version of all published research articles to be placed in any digital archive immediately on publication. Authors are free to archive articles themselves." The precise freedom all this gives to users to reproduce the text is unclear, but calling WebmedCentral "open access" would be misleading.

The approach of WebmedCentral is reminiscent of Google Knol, which is where PLoS Currents is hosted, or of a preprint server, except there is an active post-publication peer review system.

Open peer review and community peer review are not new ideas. A similar approach to that of WebmedCentral was tried by Philica in recent years without great success; the site rapidly filled with crank publications. Another was 'E-Biomed', which was stifled and instead became PubMed Central. Although anticipated a decade ago, biomedical publishing has been wary of preprints and other proposals to remove or reduce pre-publication peer review. BioMed Central's Genome Biology had a preprint server, but it closed in January 2006. A humanities institute is experimenting with community review on Shakespeare Quarterly, though they are using a hybrid model rather than abandoning invited pre-publication review. More generally, MediaCommons argue for community peer review in their book "Planned Obsolescence". They are far from naïve, noting that
'Too many digital publishing experiments, like Philica, have lagged due to an assumption that might be summed up as "if you build it, they will come."'
The journal requires appropriate ethical approval for human and animal studies and will remove studies if they find that they fail to meet ethical standards. Articles may also be removed in cases of scientific misconduct or plagiarism. They suggest that authors use statistical advice, and ask authors to adhere to reporting standards such as CONSORT. They ask authors of clinical trials to adhere to the Good Publication Practice guidelines, but do not specifically mention trial registration. They endorse the ICMJE criteria for authorship and the use of medical writers should be declared. Funding and competing interests should be declared, though there is no definition of a competing interest. They ask authors to suggest at least three reviewers and to not only pick "friendly reviewers", and say they may invite further reviewers. How these policies are enforced and who enforces them is not clear.

Technical issues:
Previous versions of an article should be linked to, but this fails. The journal allows digital archiving and digital preservation by LOCKSS members. Some test articles can be found as Word documents that are not visible via the search, which raises questions about site security. The presentations of figures is in a sidebar and sometimes without even a thumbnail, though the pop-up view is user friendly. The referencing could be improved, with clearer formatting and hyperlinks down to the references. Some of the formatting of the reviews is poor, with changes in font and font size, and several reviews are double posted.

Publication volume:
There are 366 published articles as of 30/01/2011. Submission rates appear to have peaked following publicity in August, and have since declined (see figure).

There is currently no indication on the articles that they have received no pre-publication review. As might be expected given the lack of pre-publication review, some of the articles are fringe science: aliens, homeopathy, prayer, and telepathy are all represented. There is an account of chiropractic care of a patient with fibromyalgia, an opinion article on the evidence for homeopathy in acute upper respiratory tract infections by Peter Fisher and colleagues, a study linking 'emotional quotient' and telepathy that has the obligatory mention of quantum theory, an article on the hunt for alien life that takes in the Higgs Boson, the Bermuda Triangle, and alien implants , a virtually content-free account of acupuncture in rats, and an intercessory prayer study. The latter is, thankfully, a deliberate satire.

When you get this kind of opportunity of publishing without a filter, sex always seem to come to the fore: step forward, a hypothesis on why women don't sleep with the first man they see when they ovulate, two case reports of priapism, an institutional review of Peyronie's disease, and a case report of penile fracture. As pointed out by two reviewers, it contained the unfortunate typo in the title of the corpus callosum (in the brain) rather than the corpus cavernosum, hence it was republished (demonstrating that the article version system is not working).

Many of the articles are unpublishable in any biomedical journal: a rant about academic exploitation; a review of the biological activities of a herb that the author seems to have forgotten to write; an account of a trauma registry that is confused and sketchy; a review of oral health and inequality for which the recommendations section appears to be lifted verbatim from Nunn et al. 2008, who are not cited. How many more of the articles will contain plagiarism would be interesting to see.

On the more positive side, there are a series of interesting articles by three authors: Leonid Perlovsky has published a series of mainly hypothetical papers, e.g. on language and cognition; William Maloney, a New York dentist, has published a series of overviews and historical accounts, e.g. the medical legacy of Babe Ruth; Uner Tan has published a series of articles of his observations and theories of quadrapedal locomotion in humans, e.g. these two cases.

Other interesting reads are a survey of the role of hairdressers and bartenders as informal emotional support following the 9/11 attacks and their responses to this role, a study by Robert Dellavelle on how journals don't require ethics approval for meeting abstracts, and a series of witty anecdotes by an Israeli psychiatrist of cases of "curing demons" in his patients.

Around a quarter of the articles are case reports. The insatiable demand of hospital doctors to publish case reports has clashed with a reluctance of medical journals to publish what are often "me too" publications offering little generalisable insights, and which are often poorly presented and incomplete. The recent trend of open access case report journals - BMJ Case Reports, Cases Journal, Journal of Medical Case Reports, Clinical medicine insights. Case reports, Case Reports in Ophthalmology etc. from Karger, Case reports in Medicine from Hindawi and the American Journal of Case Reports (free, not OA) - doesn't appear to be matching demand.

There are also 58 reviews, 31 opinion articles, and at least 15 of the "original articles" are not research articles; less than half of the articles on WebmedCentral are primary research.

Some of the reviewers are published researchers, but they usually have only a handful of publications and they would be unlikely to be selected as peer reviewers by a mainstream biomedical journal editor – this could be seen as a positive or a negative. There are pages listing reviewer details, but the reviews by a single reviewer are not listed.

Relatively few articles have received an insightful review or comment. Around 55% (201 articles) have received a review of some kind, and the most any article has received is six reviews (see right hand panel of the figure). 138 reviews were unsolicited and 211 were solicited by the authors. The quality of the reviews is usually low. Just over half of both solicited and unsolicited reviews contain critical analysis, i.e. at least some mention of improvements the authors could make to their article, meaning that probably less than 25% of all articles receive any degree of critical analysis. Many reviews are sycophantic, for example one case report is said to be "the best ever article publishe[sic] so far". Many merely state what the articles is about - one author-invited reviewer spends 358 words reiterating what the article says and telling us that it is a "must read" - or give the views of the reviewer on the subject rather than the article - another reviewer devotes a mere 23 words of a 430 word review to even mentioning the paper. Most reviews are very short: the average is only ~115 words for both author-suggested and unsolicited reviewers; the longest is just over 1500 words (see left hand panel of the figure for the length distribution). Comments with critical analysis are much longer (~175 words) than those without (~50 words). If I were to see reviews like most of those on WebmedCentral during standard peer review, I would never use that reviewer again.

Some of the reviews include comments such as "this is suitable for publication" or "I hope it is accepted", which indicate a lack of awareness of the publishing model.
One author has even reviewed his own paper. An article I consider unpublishable received the reviews, and I quote them in full, "good" and "No comment".

There are some examples where robust review has taken place. The concerns raised by the reviewers on this paper, including a lack of mention of ethics or consent, would lead most editors to reject such a paper – but WebmedCentral has no routine mechanism for doing this. Authors responded to reviews only on a handful of papers. A lively debate developed around a physician's self-case report, but this was a rare exception. I found one example of what appears to be functional peer review, with the authors revising their work and the reviewer stating that they are happy with the revisions.

In Bambi, Thumper's parents taught him that "If you can't say something nice... don't say nothing at all", but I think that the opposite applies in peer review. If you can't come up with critical comments about a paper, you're probably missing something: every paper has something wrong with it. The sycophantic nature of many of the reviews in WebmedCentral might be inherent to open (named) peer review, but in my experience and according to published studies, open peer review increases the length of reviews and makes them more polite, but has no effect on review quality. Another factor may be that many of the authors and reviewers of WebmedCentral are from India: R. A. Mashelkar argued in Science that "India must free itself from a traditional attitude that condemns irreverence", and Nikhil Kumar and Shirish Ranade argued in Current Science that "it is a preponderance of obsequious reverence and sycophancy that has placed the science in the country on a downhill slope." Are we seeing this unwillingness to criticise in action?

Overall assessment:
This is an interesting experiment in post-publication peer review, which both indicates the possibilities – instant publication, open community review – and the perils – unsound science, unbalanced opinion, and substandard writing being presented as part of the scientific literature.

Building a functioning publishing platform from scratch is no easy matter, and several hundred publications in seven months is an impressive figure. There has been a noticeable engagement from the community, with over 365 submissions and a total of nearly 350 reviews in seven months, 40% of them by reviewers not suggested by the authors. However, the submission rate is declining and the coverage and quality of reviews is not nearly high enough to functionally replace pre-publication review.

The onus is on the authors to obtain reviews: the journal states that it will obtain reviews, but this is not in evidence - just under half of the papers have no reviews, and 30% have only one review. More effort needs to be put into gaining reviews from qualified experts.

Reviews are essentially worthless if nobody pays any attention to them, be that an editor, the authors or the readers. Pre-publication peer review is not merely a filter, but it also acts to improve articles. On WebmedCentral there is no pressure for articles to be revised in accordance with any critical reviews, perhaps other than author embarrassment. As reviewers see a lack of response to their comments, they may lose enthusiasm.

Without a clear indication that reviewers have criticised an article and no indication that the articles are not peer reviewed, readers may view the work uncritically. If reviewers state for instance that the work is not sound, this should be clearly flagged up to readers near the top of the page, and articles should be sortable based on the answers given in the review from and the rating given by reviewers and readers. Another layer should be added, allowing articles to be promoted by agreement from their 'Scholarly Reviewers' to a "publication standard" level, giving authors an incentive to revise their work. "Featured articles" do exist, but the criteria used are not revealed. WebmedCentral are forming an "Advisory Board" of "eminent scientists"; perhaps this board will increase the rigour of the site.

Without the oversight of an editor choosing diverse reviewers and because most scientists are unaware of the site, it may become a closed community of the same authors positively reviewing each others' work – the precise opposite of the aim of the journal. Unless the process is reformed, WebmedCentral is likely to remain a "Cargo Cult science" journal, which in the main publishes articles that only superficially resemble the peer-reviewed literature, and that are reviewed in a manner that is only a pale imitation of pre-publication peer review.

Other commentary on WebmedCentral:
WebmedCentral and ginger pee on Pleion, a discussion on Ramy Aziz's FriendFeed, a blog review, a response to receiving an email from them, a brief welcome, a mention by Jenny Rohn in a Guardian comment thread, a comment by an author, an assessment in a sports science magazine, a mention in Lab Times, a critical appraisal of an article by PJ of Pyjamas in Bananas, a blog reaction in French, and a few Twitter comments.
But who runs this site?
"We are a group of medical and management professionals with no affiliation to any major biomedical publishing group" is all they say, but who runs the site shall be revealed in my next post: "Who are WebmedCentral?".


Joe Dunckley said...

"WebMed Central"? What losers. Haven't they learnt from Manuscript Central? If they wanted to be cool they would have called themselves "WebMed ONE".

When this new high speed railway gets built, I want it to take two hours to get to Glasgow ONE railway station.

And they think they they're the future?

pj said...

I notice a lot of the commentators take the 'bah, it'll only get the rubbish stuff people can't publish elsewhere' which is likely true but also an attitude that really annoys me.

Coming from the medical field it is amazing how much research which it would be really useful to know the results of gets buried, and even worse, how many clinical trials get buried or sketchily mentioned in a conference abstract. There is real money and time that could be spent doing something useful that gets pissed away because this information is not available - and all because academics don't want to waste their time or sully their publication record with dull or negative results, or even because they don't want to publish stuff that contradicts the narrative they're trying to convey.

Matt Hodgkinson said...

"it is amazing how much research which it would be really useful to know the results of gets buried".

Absolutely, which is why the advent of "mega-journals" like PLoS ONE (wot I work for) that focus on scientific merit rather than "impact" can only be a good thing.

pj said...

Well yes, but I would argue that, with all due respect, a publication fee of US$1350 is likely to provide quite a disincentive to publish for people who are already disinclined to publish their data anyway. At least things like Nature Precedings (do people still send stuff there?) provide a repository for lector emptor data that might be useful.

I think pre-registration and compulsory publication (whether peer reviewed or not) ought to be required by funding bodies for pre-clinical studies and for clinical studies trial data should be legally required to be made public within a pretty tight timeframe.

Matt Hodgkinson said...

Comment on behalf of Kamal Mahawar of WebmedCentral:

1. We have not overlooked issues such as patient consent. The article submission process includes the step to upload a patient consent form. Our articles do not currently have DOIs but in future it is our aim to get it. There is a cost involved in this and somebody will have to bear that cost.

2. Articles are not indexed by pubmed but that is mainly because pubmed and other databases insist on the content being peer reviewed. It is our hope that in near future we will be able to persuade organisations to index some if not all articles on WebmedCentral.

3. Hodgkinson found licensing details confusing. We have since updated our copyright policies. We now adhere to Creative commons license.

4. The term “open access” is open to interpretation. However there is a broad consensus as on http://en.wikipedia.org/wiki/Open_access. Since all content on WebmedCentral is free for readers/visitors to the site, it is fair to call it “open access”.

5. Generating a debate around their article and obtaining reviews is authors’ responsibility. In near future, our faculty team will help in this task. Our admin team will periodically invite reviewers for articles as and when we feel the need to reinforce post publication peer review.

6. It is our aim that all the versions of the articles will be interlinked. Please point out any technical flaws to our technical team and they will be happy to resolve the matter as soon as they can.

7. Some of the remarks made here, dare I say with due respect, are judgemental. We do not pretend to judge the importance of a manuscript or even technical accuracy. We publish everything that users submit to us as long as the content is scientific. History is replete with instances when the true value of an observation becomes obvious only much later.

8. Plagiarised content will be removed as soon as it is brought to our attention and authors asked to explain. We will maintain a record of these authors and such authors will be barred from further submissions to our portal. In near future, we hope to use technology to minimise such occurrences.


Matt Hodgkinson said...

[continued comment on behalf of Kamal Mahawar of WebmedCentral]

9. In future, users will be able to see reviews carried out by a single reviewer by clicking on his/her profile.

10. The quality of reviews on WebmedCentral will improve with the quality of reviewers. We are currently working to create a team of faculty who will be helping with the review work. This is an ongoing work and one that we pay a lot of attention to as our portal relies on post publication peer review. Generating a post publication debate is WebmedCentral’s priority. We are working on mechanisms to reinforce it further. In future, we will have a mechanism of rewarding authors who can get some good reviews and then modify their manuscripts by moving them to a higher level of website. This would be WebmedCentral indicator of article quality. However we do not intend to move away from our central philosophy of publishing without peer review just because some authors are not taking it seriously.

11. Yes a lot of our users are from India and we will not apologise for the fact that a country with more than a billion people wants to participate in academic discussions. However Hodgkinson ignores the fact that we have articles and users from all over the globe. A look at our recently formed advisory board and faculty team (both in development) will confirm that fact. Moreover our portal routinely gets maximum hits from USA.

12. Open reviews can be less critical but closed reviews can be overcritical and judgemental. Reviewing is a powerful act and with power comes always the responsibility to use it adequately. One should not use closed peer review as a veil to hide behind. Transparency in communications is surely a good thing and WebmedCentral believes in it. We are confident that when serious scientific issues will be debated, scientists will not confirm for the sake of doing it. To accuse a whole culture and race of sycophancy on the basis of isolated comments is arrogant and disrespectful.

13. Though we encourage authors to respond to reviews and modify their manuscripts accordingly, they are never worthless. They always provide readers with an additional insight into the work in question. Pre publication peer reviewers force authors into agreeing with them. Post publication peer review lets authors retain their independence and they ignore any negative review at their own peril.

14. We are developing new search functionalities when a user will be able to select articles with a certain reviewer rating etc.

15. Featured Articles are decided by our management team as articles most likely to generate interest. In near future, faculty team will own up this task as they will be responsible for leading the agenda of each speciality page.

16. We did not initially think that WebmedCentral was about us but there was a demand from some scientists to find out who was behind this venture even though such information is always available freely from company house. Hence we have now publicly declared ourselves on our web portal and also on social media circles.

17. Finally one should not overlook what we provide. We provide any biomedical scientist with an ability to say what he/she wants to say without any cost, without any significant delay (48 hours at the most), without any bias, and without any fear of being drowned by opposing scientific views.

Kamal Mahawar
CEO, Webmed Limited UK

Matt Hodgkinson said...

This post was referred to by Trey Lathe, Data is not in the papers (nor is analysis): WebMedCentral & post-publication review. The OpenHelix Blog. 14 March 2011

Matt Hodgkinson said...

Richard Poynder has an interview with Kamal Mahawar on his Open and Shut blog, which discusses WebmedCentral and specifically my comments and criticisms. See http://poynder.blogspot.com/2011/08/open-access-interviews-webmedcentrals.html.

There is also a discussion of WebmedCentral on LinkedIn at http://www.linkedin.com/groupAnswers?viewQuestionAndAnswers=&discussionID=65948470&gid=38278&commentID=48889110&trk=view_disc

Richard said in one of his questions that "One could argue that PLoS ONE is a competitor to WebmedCentral: perhaps Hodgkinson should have added a statement to his review indicating his affiliations?".

I had thought it was already clear that I was an editor at PLoS as this is mentioned in my Blogger profile: http://www.blogger.com/profile/02376788922895957748.

My blog posts are in a personal capacity and are not representative of the views of the Public Library of Science as an organisation. I am a full-time professional Associate Editor at PLoS ONE, an online-only, open access, peer-reviewed journal.

Personally, I am very much in favour of post-publication and community peer review, transparency, and a diversity of approaches and funding models in open access publishing. My comments on and criticisms of WebmedCentral were made in the hope of improving this and future ventures, not to try to stifle them.

My interest in WebmedCentral was sparked by a comment on my blog: http://journalology.blogspot.com/2010/10/editing-wikipedia-for-scientists.html?showComment=1295906374536#c6611060046856494980.