by George O'Toole and Roberto
Publishing the results of research in journals remains the most important lasting record of scientific work. Yet, if you ask almost any scientist about the process of publishing itself they will likely bemoan the current situation. Perhaps this has always been the case, it's true that people tend to long for "the good old days." But perhaps scientific publishing has changed in fundamental ways that negatively affect the scientific pursuit. I (Roberto) feel that we, as practicing scientists, need to address these recent trends and take actions that might help improve the process of scientific publishing. Therefore, I was delighted to read a few days ago, an editorial opinion paper entitled "Support Science by Publishing in Scientific Society Journals" by Patrick Schloss (Chair of ASM's Journals Board), Mark Johnston (Editor in Chief, Genetics) and Arturo Casadevall (Editor in Chief, mBio). The article is open access and I encourage everyone to download it, read it and discuss it with scientific colleagues. A few weeks ago I had an email "conversation" on the subject of scientific publication with my friend and colleague George O'Toole with the intention of posting here in STC. It seems the timing was right and we hope that after reading it you will be stimulated to read, discuss and act on the mBio paper.
Roberto: George, based on some very short exchanges that we've had, I know that you and I – along with many others in science these days – have a great sense of frustration when we see how the entire process of publishing the results of research has evolved in the last couple of decades. I'd like to dig a little deeper into what we perceive are the current problems, perhaps think about how we got to this point and, hopefully, talk some about what steps we, as a community of scientists, should be taking (or might already be taking) to turn the situation around. But as a start, let me hear from you in a little bit more detail, what are some of the key problems you see with the publication process today.
George: Let me start with for-profit scientific publishers. As I see it, the process of peer review of our colleague's work grew out of a sense of responsibility to the field – scientists who worked and published in the field had a responsibility to also do their part to review the work of others. Society journals were launched to provide the infrastructure (find a printer, hire the copy editors, organize the process) and page charges supported the operation. A great example was the launching of the Journal of Bacteriology in 1916 as a forum for microbiologists to publish their work. Somewhere along the line, publishing houses decided they could make a buck in this space, and somehow scientists just kept reviewing… for free! Maybe when the first such for-profit journal was launched most folks didn't think twice about this model – but once we think about it, it doesn't make sense. Ask yourself this – how much time have you taken away from your own science or from spending time with your family to work for free for some CEO making six figures? That's just nuts, and personally, I'm done with it – I don't review for any for-profit journals. In my opinion, no one should.
Wow! You come out swinging out against the for-profit publishers. I guess it is safe to assume we are not talking at all about the "predatory" publishing firms – let's take those off the board for now, because frankly, they are so bad they shall not even enter into the discussion. I assume you mean bona fide publishers, among them the Nature group, Elsevier, Blackwell, Cell Press, to name but a few. And I also assume that you recognize there are non-society publishers out there that are non-profit, for example PLOS. I see your point very clearly with the for-profits but have a problem coming up with a solution. In the spirit of full disclosure, I am currently Associate Editor for one of the Nature group journals. The point is that, as such, I get paid. And often, I act as "reviewing editor." So, in response to the issue you raise, at least the scientific editors of some for-profits are paid. But not the reviewers. Let's say, for the sake of argument, that the for-profits were to begin to pay reviewers. Might not that have the unwanted effect of making it very difficult for the editors of non-profit journals such as the Journal of Bacteriology, to get people to review for free? Many of the society journals are already struggling as it is. So, while I might welcome some system that pays reviewers for their work in reviewing manuscripts – as do most book publishers – I tend to think it will only work if it becomes universal. Might that come to pass? I guess, maybe, if enough prominent scientists such as you begin a grassroots movement in that direction. But, frankly, I believe the problems with publishing these days go well beyond compensation of reviewers. Let's focus for now on the review process. As an editor I have noticed over that last few years a steadily increasing difficulty in getting people to agree to review. Although, I guess, that a whole lot of those people who are "too busy" would all of the sudden find themselves much less busy if each manuscript they review meant a little extra cash in their pockets.
I agree that the issues with for-profit journals are just part of a changing landscape of publishing (and yes, I do mean for-profit as a distinct category from predatory publishing). Let me be clear – I think paying reviewers is the LAST thing we should be doing. I think the system worked well without financial incentives for a long time and we should keep it that way. Adding money to the system that way will not help – I shudder to think how such a system would be corrupted! Back to your point about busy reviewers. Back when I was a graduate student and postdoc, everyone (at least in my world!) read the Journal of Bacteriology, Applied Environmental Microbiology and later on Molecular Microbiology. One could keep up with the output of those three journals. Now as the number of journals in all disciplines multiply, people are bombarded with review requests. You just can't say yes to everyone, and that's my point above. Given that time is the most limiting resource for most people, reviewers have to pick and choose. My point is that society journals take any revenue from journals and plow it back into the field – running meetings, sponsoring travel grants and awards, advocating for the field, and helping to fund reports that speak to critical issues for which they have expertise. Societies give back to their members in real ways. Can societies have issues? Yes, but I do believe they have the best interests of the membership at heart. In my view, for-profit operations simply pull money out of the research endeavor to enrich stockholders and CEOs that have little or no investment in the scientists doing the work.
Seems like we agree, perhaps for different reasons, that paying reviewers is not a good idea. We also both agree that society journals are both a great resource and a way to get benefits for the publishing process back to the scientists. I respect and even support your decision not to review for for-profit journals though I am not sure your edict that "no one should" review for them is practical. Finally, I do not have numbers on this but the issue with people being inundated with review requests may be a vicious circle. People begin to say no, so editors need to send out more requests which means people will feel more inundated. I wonder if the number of papers to be reviewed per practicing scientist has really increased in the last twenty years. Yes, there are many more journals and more manuscripts submitted nowadays, but there are also many more scientists. As I said, I just don't have the numbers. But it is undeniable that the mood is not happy among reviewers. Now… moving on… After all, reviewing is only a small part of the process of publishing. If the reviewing process seems rather awry, what can we say about the feeling among the authors? I certainly don't sense great joy around the process of writing and submitting manuscripts. I personally think that while the process was always somewhat stressful, things have gotten out of hand over the last twenty years.
I also want to bring up this current sense that everyone seems to be "too busy." Here's one example that I think illustrates the diseased state of the publication process. Scientists need to be evaluated all along, for hiring, for promotion, for obtaining funding, etc. Evaluators are "too busy" to carefully read the work of the scientists they are evaluating so they look for easy ways to evaluate how "good" someone is. Along comes a digit associated with a journal, the infamous impact factor. Now busy people need only look at the impact factor of a JOURNAL mind you and determine the worth of someone's publication. Nonsense! But it's easy for people who are "too busy." So now, those being evaluated – particularly early career people – buy into the trap and want to publish, at all costs, their work in the journals with the highest impact factor they can hope for. Forget about appropriateness or fit for a given journal. Every result has to be "spun" into a story that can be "sold" to the highest possible impact factor. You see how the whole process begins to devolve into the current madness?
I do see. OK, we've complained, now can we think about some solutions? We talked about supporting society journals, that's great. What else? Individuals on search committees and writing tenure review letters can do their best to ignore impact factors and look at the work on its face (easier said than done!). Having discussions such as this one can help – we need to talk with our trainees about the costs of going for the prestige journal – yes, the impact factor is high, but all the people hours of work that end up generating data in the supplement, never to be viewed, and the submission, a long list of new experiments, another year to collect data, resubmit – oops rejected. Resubmit again to another journal, and repeat. That is a loss in terms of precious financial resources and time, and under-appreciated data, so there is a trade-off for going this route that one needs to consider. The current publishing landscape, in my view had two disruptive events – the advent of for-profit journals and the open access model. Anyway, perhaps we need another disruption. For my part, I am trying our first foray into the preprint model of publishing. It seems to work well in the physical sciences and computer science, and reminds me much more of the older society model. I'm not sure how the review process will work. Perhaps the author picks reviewers, or the people in the field who are impacted by the work review the paper. A whole new can of worms here, but a can of worms run by the community of scientists for scientists, so that means something.
Let me respond to one theme at a time. On the issue of research assessment, I am a big supporter of DoRA (Declaration of Research Assessment) initiated at the 2012 meeting of the American Society for Cell Biology. The general recommendation that DoRA makes is that journal-based metrics, such as the journal impact factor, should not be used as a "surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions, or in hiring, promotion, or funding decisions." DoRA nicely describes how every individual involved at all levels of research assessment and the publishing process can contribute to improve the current situation. Anyone and any institution or publishing entity that supports DoRA can become a signatory. While DoRA is non-binding, it is a very good step towards reversing the ongoing trend.
On your second theme, when you point at the two main disruptive events as being the rise of for-profit journals and the open access model, I will beg to differ. There were very successful and highly ethical for-profit journals for many decades. I would say it's the more recent rampant proliferation of rather dubious for-profit journals along with the increased distancing of practicing scientists from the editorial work among the better-known journals that has created disruption. Regarding the open access model, here again I have only good things to say about such open access journals as the PLOS family and ASM's own mBio, and eLife, to mention but a few. I have to add that the entire review process at eLife I find to be an extremely good model to follow: reviewers reach a consensus through online discussion and there is a very strict sense of not asking for additional experiments. I've experienced this process as reviewing editor, and as author of both rejected and accepted manuscripts and I have found it a "breath of fresh air" relative to many others. In the open access arena, it's the great proliferation of journals, and most concerning those with predatory practices, that I find disruptive. But I do find lots of merit in many open access journals. Finally, I do like your suggestion that the preprint model is worth trying. As you say, it has worked for decades in physics and more recently computer science. I will love to hear how that experience goes for you. As you know, I will not be involved in publishing primary research papers for too much longer since I am retiring and closing the lab soon. But I would have been interested in trying out that model.
Thanks Roberto, for clarifying my point. I don't want to lump all for-profit journals together. I think we agree the predatory journals really have no redeeming qualities. I do feel like using non-practicing scientists in roles as editors has resulted in what seems like an increasing growing trend of asking for a "tidy story" – which does not help science as a community. So, from my point of view I think there are a few things we can do as a community: (i) support society journals – the mission of societies is a positive one for our community broadly, (ii) from my point of view while reputable, high-profile for-profit journals are not on their face bad, but they do drain scientists' time and grant money out of the system, and I will personally not be reviewing for said journals, (iii) worry more about your personal impact factor than valuing your science based on how the other papers in the journal are cited, (iv) consider the preprint model going forward as a way to get your work out to a broader community. Anything else Roberto?
George, you nicely summarize important points in this last statement about what each one of us, as a member of a community of scientists, can do to help bring about some of the much needed change in scientific publishing. Clearly, we have our work cut out for us but I remain hopeful that we'll see changes, be they gradual, for the better. Wonderful "talking" with you!
George O'Toole is Professor in the Department of Microbiology & Immunology in the Geisel School of Medicine at Dartmouth in Hanover, NH.