Monday, August 6, 2018

On second thought...

I make a point of not writing too much about the behind-the scenes details of being an Editor, to respect the anonymity of authors, reviewers, and the process. But sometimes there are things which just aren't talked about openly but which prospective authors, especially students, need to know. So I'm going to spill some beans.

Let's talk about Editorial declines for a sec.




This is inspired by a particular paper, and I'm going to so totally change & obfuscate & rearrange the details that even the author in question probably won't realize this is about their manuscript.

An author submitted a paper to a journal I edit, which we will call Journal X.  I skimmed the paper in about 5 minutes (it was one of perhaps 4 new papers I was processing just that day). I was struck by two things. The author(s) were pushing an interesting conceptual idea I hadn't seen before, putting a twist on a familiar question of sympatric speciation (remember, I'm obscuring/changing the facts), by fusing that familiar topic with a new one (thermodynamics). And they did this with a really big-scale long-term experimental evolution dataset with unicorns. Impressive scope of the study. So, I passed it on to an Associate Editor. I confess I hadn't read it in detail, I just thought the dataset and question/answer sounded interesting, and worth a more careful look by someone who knows the detailed subject area better than I do, and has more time per paper than I do.  (Note: had I rejected it offhand, I certainly would have spent more than 5 minutes with it).

Now, we have a policy at Journal X. We want to give every paper a fair shake. But we also can't stretch our Associate Editors or reviewers too thin by assigning them too many papers. They get exhausted, they stop contributing decision recommendations or reviews, and we fall apart. So we try to avoid the situation where we send out a paper and get back an upset "why did you take up my time with this?"  Our policy is that the job of the Editor is to write helpful constructive reviews for Editorial Declines, for the papers that we shouldn't take our AE's time with. As my predecessor once told our AEs, "we Editors review some papers, so you don't have to". And our AEs should write reviews and recommend Editorial Declines for papers that aren't ready for reviewers' time. This is a favor to our reviewers, but also gives authors faster feedback than sending out a paper for full review. Sometimes my Editorial Decline letters can be pages long. The Journal X editors get thank-you letters for their detail, and for the fast turn-around (often just a few days). If you aren't going to get your paper accepted, wouldn't you rather get constructive feedback very quickly, so you can improve your paper for elsewhere?

The point is, we are trying to strike a balance between being fair to authors, and being considerate of AE and reviewers' time. That requires some triage, and a balancing act.

In the case of this particular manuscript about the thermodynamics of unicorn sympatric speciation, the Associate Editor gave a strong "Decline" recommendation, but acknowledged that I might disagree, leaving the door open a crack. That recommendation was supported by two full pages of the AE's own constructive review (that's not uncommon in the Journal X tradition). I was, frankly, surprised by the negative response because I thought the many-generation experimental evolution of unicorn mating behavior was pretty impressive. So I reread the paper, quickly but more thoroughly than last time. 30 pages in 20 minutes maybe. And I found myself generally agreeing with the AE. I wasn't 100% sure, but I want to respect the time and thought our AEs put in, so I took the AE suggestion and wrote a "Decline" letter and added my own two cents worth in to justify that on top of the AE's suggestions. I still had second thoughts, but I trusted the AE. (Note, at this point the paper had received about half an hour of my attention, and several hours of detailed examination by the AE, so this was not a casual decision by either of us).

Soon, I get a direct email from the Unicorn expert herself, thanking me for the detailed feedback but expressing her frustration. Not an appeal. Not a complaint. Just... disappointed.  She really wanted to publish with us.


I hate that part of the job, I don't like making people sad. I'm an author too of course, so I know how rejections feel. I had been rejected by Journal Y just that same day. So I was sympathetic.

A few days later, I did something I've not done before. Without an author's formal appeal, I went back and revisited my decision (I didn't tell the author, so as to not raise hopes). Remember, I had looked over the paper twice, once in about 5 minutes and once taking 20 minutes. So I hadn't fine-toothed combed through it (though the AE had). I can't, I often handle multiple papers per day (my record so far is 9). But this time, I had second thoughts. So I re-read the paper with the detail of a reviewer. I spent about 2-3 hours on it, and wrote down dozens of detailed and big-picture comments.  Had I erred with my Editorial Decline (with AE support)? Were we really wrong not to send the paper out to expert Unicorn biologists for review?

The short answer is, the paper really had a solid core. Unicorns really do undergo sympatric speciation. But we knew that already. The additional ideas that formed the centerpiece of the abstract, introduction, and discussion: the innovative fusing of sympatric speciation with thermodynamics? That was reaching too far beyond that solid core, and didn't have the data to back it up. Unsolicited, I sent the author an additional batch of my own detailed comments in an attempt to be constructive and further justify our decision, so they can really strengthen their submission for another journal.

<Note inserted later: The author replied, thanked me for the detail, commented it was more than she expected from an Editor. The author acknowledged that I had identified several weak points they had tried to gloss over, and that my assessment was correct and fair.>

The moral of the story here, if there is one, is this: we Editors do make knee-jerk decisions, but we do so in good faith trying to find papers we can support. And sometimes that makes you authors feel like we Editors are jerks. We can't read every word of every paper you submit, though we try to look through in enough detail to understand what you've done and what it implies. Consequently, we can indeed make mistakes. This fabricated case I refer to here is unusual because I double-checked myself in great detail, to see if I had erred.

That Editorial Decline? That turned out to be the right decision. Not because the author was a bad person or a bad scientist, its just that the paper needed work. It was claiming more than it had grounds to claim. With very substantial revision (reframing the point of the paper) that work would produce a fine paper, but not to the level of novelty our reviewers and readers expect of Journal X. The exciting integrative part had been more speculation than substance. So, my job was to help the author improve their paper to be a solid and successful submission to the Journal of Unicorn Biology. And the faster I could provide that feedback, the better for the author.

So, when you get an Editorial Decline, you may feel poorly served. Why didn't it at least get a review? I've certainly felt that way as an author. The answer is, I'd like to think that we Editors have a good  feel for what will fly with reviewers and readers, and what won't. As a junior faculty member I reviewed 7 papers for Journal X. As an Associate Editor I handled about 150 papers for Journal X. I've written 300 decision letters for Journal X since I started as Editor 7 months ago. I am overseeing about 35 active papers at the moment. All this gives me a clear snapshot of what reviewers tend to say, and where a given paper stands relative to other submissions in the competition for limited page space. I won't claim to be always right. But  I think I have a good basis for those decisions and will do my best to convey the justification. I emerged from this with a bit more confidence in the decision-making process. And I hope this story serves as a reminder to you, the readers, that Editorial declines (at least, with Journal X) aren't just arbitrary decisions, they represent the judgement of someone who has seen a lot of papers, and a lot of reviews, about where your paper falls in the discipline-wide multivariate distribution of papers (for clarity, quality, rigor, and novelty).

And if I have a second thought about a decision in the future? I'll still double-check to be sure I've been fair, but I'm growing (slowly) in my confidence about my judgements and slowly outgrowing the impostor syndrome of being Editor from Journal X.










1 comment:

  1. Nice post - as a fellow Editor I totally get it and think you really deserve some form of Editorial Oscar - Keep up the excellent work.

    ReplyDelete

Predicting Speciation?

(posted by Andrew on behalf of Marius Roesti) Another year is in full swing. What will 2024 hold for us? Nostradamus, the infamous French a...