Where to submit your paper. Or “If at first you don’t succeed, fail fail again … then try open access”

The confluence of two experiences motivated this post. First, I was involved in a conversation on Twitter (below) that was reacting to suggestions (in a commentary in Nature) that the high volume of open-access papers was the cause of the reviewer fatigue that so often bedevils journals and editors (such as myself). At one point in this thread, someone pointed to a blog post titled “Why I Published in PLoS ONE. And Why I Probably Won’t Again for Awhile.” The main point of that post was to contrast the desire of young scientists to better the world by publishing in open-access journals with the perception that senior scientists don’t view a paper published in open-access journals as equivalent to a paper published in a more traditional journal. This latter sentiment was similar to my own experience on search committees in which candidates would be considered less impressive if they published too much in open-access journals.

.@BrunaLab @EcoEvoEvoEco @PLOS @NatureNews Ummm...in 2013 PLOS One published over 31K articles. More than Elsevier's entire output?
— Adam P. Summers (@Fishguy_FHL) November 27, 2014

@Fishguy_FHL @BrunaLab @EcoEvoEvoEco@PLOS @NatureNews Elsevier published 250,000 in 2013 http://t.co/78RYgVoB22
— Trevor A. Branch (@TrevorABranch) November 27, 2014

@Fishguy_FHL @BrunaLab @PLOS @NatureNews #OA jour. (note: many more than just PLOS) decrease reviews b/c they don't reject = less re-review
— Andrew Hendry (@EcoEvoEvoEco) November 27, 2014

The second motivation came from our weekly lab meetings. Near the start of each meeting, we go around the room asking “Who had a paper or proposal accepted or published this week?” And then, after a hopefully long discussion, we ask, “Who had a paper or proposal rejected this week?” I kind of like this two-part question because it enables us to get excited about our successes while also making the failures seem more acceptable. (“Oh, it happened to her too, so my own rejection is OK.”) And we can also complain about reviewers and can discuss how we will make our papers better in response. It just so happens that, over the past few months, no one has been able to speak up for the first question and pretty much everyone has spoken up for the second. One lab member even noted that rejection seemed to be a recent trend in the lab.

These two experiences led me to consider the question: “Should you – as a young scientist – take the easy route and publish in open-access journals, or the hard route (likely entailing multiple rejections) of trying more traditional journals, either the big boys or the classic society-based journals?” First, let’s consider the benefits of open access. The basic idea is, of course, that everyone will see the paper and you won’t waste your time cycling through journals that don’t think your paper is “important enough.” Moreover, citation rates are pretty decent for open-access journals, right? At least, that’s what everyone says. I would like to put this presumption to the test based on my own experiences.

I have published three papers in PLoS ONE (and several in other open-access journals). I quite liked all three papers and first tried traditional journals, but the papers were rejected a few times and the students wanted to move on with their lives and research, so we sent them to PLoS ONE, which accepted them quickly. So I decided to ask: How well have these papers been cited relative to papers I published in the same year in other journals? It turned out that I had a decent sample size because my two early PLoS ONE publications (2007 and 2009) happened to occur in years when I published a good number of papers (10 in 2007 and 17 in 2009). So I simply tallied the cumulative number of citations (here always from Web of Science simply for convenience; Google Scholar tells the same story) for each paper I published in those years, ranked them in order of citations, and asked where the PLoS ONE papers fell in relation to the others.

My previous two PLoS ONE papers are cited (Web of Science) least among all my papers published in those two years.

The graph tells the whole story. For my papers published in each of the two years, the PLoS ONE paper ranked DEAD LAST in terms of citations. I did have an a priori expectation that these papers hadn’t been heavily cited, but I had no idea it would be this bad. Moreover, I want to reiterate that I felt these two papers were interesting, well conducted, and potentially important – here and here they are for your citation convenience. Indeed, both were long included in a list of my favorite 15 papers. Of course, the alternative is simply that they really weren’t that good and I just can’t see it. Perhaps so but it remains clear that publishing in PLoS ONE will not enhance your citation rate for a given level of paper quality. So much for one of the supposed benefits (or at least lack of costs) of open access – at least in my case, which I am sure you will agree is what matters here. Of course, this general point is also clear intuitively: How many of us routinely check the papers coming out in Evolution or Ecology versus Evolution and Ecology (or PLoS ONE or PeerJ or Scientific Reports or anything starting with International Journal of …)?

OK, so clearly I am going to argue that you should forego open access and publish in traditional journals – but what of the annoyance, time consumption, and stress of rejection? Well, it is certainly true that rejection is common – for everyone. Many young scientists are stressed out when a paper gets rejected and feel that this somehow reflects on the quality of the work. They also often feel they are getting rejected more often than other scientists. The reality is that even established scientists get rejected all the time. A first proof of this maxim comes from a review paper by Cassey and Blackburn (2004) in Bioscience (cited only 15 times!) that surveyed the most successful ecologists (those with many publications in major society-based journals) and asked them how often their papers were rejected. The answer is often - as shown in the graph below. Indeed, many of these successful ecologists have had the same paper rejected multiple times and some of them still had at least one paper that they had failed to publish despite many tries.

Acceptance rates for papers submitted by the most successful ecologists. From Cassey and Blackburn (2004).

The above data are for the most successful ecologists and are from the good old days when rejection rates were not so high. How about some more recent data from a more mediocre (but established) ecologist (or, more precisely, evolutionary biologist)? 

I am a compulsive record-keeper. In this context, I have recorded every single submission of a paper on which I have been an author, as well as the outcome of those submissions. The sample size is pretty large now and allows me to illustrate the frequency of rejection and some factors that influence it, as summarized in the table below. I have been involved in a total of 275 submissions to journals (including multiple submissions of the same paper to different journals), of which 148 led to acceptance – an acceptance rate of 54%. However, some of those submissions were invited papers or commentaries, or appeared in special issues for which I was an editor. Taking away those near-sureties, my acceptance rate decreases to 43%. On the other hand, I have submitted 45 manuscripts to “big” journals (Science, Nature, PNAS, Current Biology, PLoS Biology) and only 1 was accepted – the first one I submitted. (I realize this looks crazy – 45 such submissions – but more about that later.) Removing these submissions from the tally as well, the acceptance rate jumps back up to 53% for the remaining “real submissions.” Finally, considering only “real submissions” on which I was first author, the rate jumps up again, to 68%. Whew, lots of stats that all simply say: rejection is an ever present companion in science.

Rejection/acceptance stats for my own submissions. 

I am not ashamed of these numbers, nor my relatively low number of open-access publications – because both facts reflect sending papers to the very best journals with low acceptance rates that then subject them to extremely rigorous and critical review, not just for the methods but also for their importance. Of course, it means that one needs to develop a mechanism to cope with rejection. My own mechanism – and the one that I try to tell my students – is that as soon as I submit a paper, I ask myself “OK, where will I submit this paper when it gets rejected.” (Hope for the best, prepare for the worst.) That way, a rejection simply means that I am days away from submitting a paper – turning a bad feeling into a good one. I also tell my students (and myself) that journals reject papers for all sorts of arbitrary reasons and it really doesn’t mean the work isn’t any good. 

While letting this post mature for a few days, I came across a great video of famous failures by people that went on to become wildly successful. This video also reminded me of a story about Tim Mousseau failing (or at least not immediately passing) his qualifying exam at McGill and having to write a remedial paper that has gone on to be cited more than 1000 times. How’s that for turning failure into success? I have heard of many other instances of papers getting rejected from a journal only to be greatly improved, some so much so that they end up getting published at a much “better” journal, such as Science/Nature.

So how high should one shoot? I have been a part of 45 submissions to big journals and all but one failed. Yet I don’t regret them (at least not all of them) – for several reasons. First, analyses (published in Science, of course) have shown that papers submitted to, and rejected from, Science/Nature end up receiving more citations than those that were first submitted elsewhere. Perhaps these were good studies to begin with and were written in a general way, and perhaps the review process improved them. Second, I think a number of my own papers submitted to Nature/Science were very good studies. (Perhaps better than many other papers published there – but then this is the sentiment of everyone that gets rejected from those journals, otherwise they wouldn’t have submitted there in the first place.) Indeed, at least five of my papers rejected from Nature/Science (some from both) have been cited more than 50 times. Three of the rejected papers ended up in Molecular Ecology and each is doing well: one published in 2012 already has 67 citations and two published in 2014 have received considerable attention (one of these was previously rejected from 8 other journals). So, if you have a great study, it is fine – good even – to submit it to Nature/Science. You will write it better and more succinctly, and you might even get some great reviews.

Articles published on their first submission (first intents) are cited less often than articles published in the same journal/year that were first rejected from elsewhere (resubmissions). Data from Calcagno et al. (2012 - Science.)

My favorite route, though, is society-based journals, like Ecology, Evolution, American Naturalist, Proc Roy Soc B, J Evolutionary Biology, J Animal Ecology (I still haven’t cracked the last of these nuts), and so on. These journals are where I want all of my work to end up (and you should too) – it looks good on your CV, and many more people see it and cite it. But, wait, I hear you saying: those papers won’t be accessible to the rest of the world because it requires an expensive subscription. Nonsense. Anyone can get access to any paper from any journal – many papers are posted on someone’s website and, for those that aren’t, all you have to do is email the author to ask for a copy! (I admit getting papers is harder – but certainly not impossible – without institutional access.) Moreover, you can pay for open access in those journals at a cost that isn’t much higher than at PLoS ONE or many other open access journals.

In summary, I suggest you work toward publishing in traditional and well-respected general or society-based journals as your goal, learn to deal with rejection, and only when you are so sick of the paper that you vomit (actually vomit, that is, not just feel nauseous) send it to PLoS ONE or another open access journal. (Or if you need really quickly publications to graduate.) Someone is bound to cite it someday – probably anyway. With this in mind, perhaps you might like to cite the cool new paper we just published this year in PLoS ONE.

-----------------------------------------------------

Note added Dec 3: see my follow-up post (Where to submit your paper - response to reviews.) that responds to post-publication comments on the present post. 

-----------------------------------------------------
Just for fun:

Invited by journal to review your own paper? More common than you think. Here is 1 way to respond. via @danielbolnick pic.twitter.com/Qxq8qTyImz
— Andrew Hendry (@EcoEvoEvoEco) November 24, 2014

@PLOSONE No apology needed, as long as you aren't annoyed by the tongue-in-cheek response.
— Daniel Bolnick (@DanielBolnick) November 25, 2014

Speaking of tongue-in-cheek, see my parody of open access journals here: 

MYScience: A newer faster cheaper easier BETTER open access journal