How to respond to reviewers

Previous posts in this "How to" series

1.     How to pick a study system

2.     How to do statistics

3.     How to write/present your science 

4.     How to choose a journal (+ part 2)

5.     How to be a reviewer/editor
6.     How to get into graduate school
7.     How to succeed in graduate school - Part 1
8.     How to succeed in graduate school - Part 2

Now its time for: How to respond to reviewers

http://evol-eco.blogspot.ca/2011_06_01_archive.html

Students starting in science might imagine that decisions on papers submitted to journals are rather straightforward outcomes such as “accept” or “reject” or seemingly "strong reject". The reality, however, is that most decisions are more on the order of “reconsider after revision” or “reject without prejudice,” or in their more modern forms, “reject with resubmission allowed” or “rejected in present form.” These more ambiguous decisions necessitate a careful revision of your paper in response to reviewer [more properly “referee”] comments, and a cover letter or “response to reviewers” that explains how you dealt with each comment. These revisions and responses are critical to the future of your paper, and the manner in which you implement them will make all the difference between whether your paper ultimately falls into the “accept” or “reject” category. In most of the above categories other than “reject,” your chance of acceptance is actually reasonably high provided you do a good job of the revisions and responses.

Having now published more than 150 papers, having reviewed more than 350 papers, and having been an Associated Editor (AE) for five journals, I have performed and encountered seemingly endless revisions and responses. Some work well and others don’t, and these alternatives typically translate into ultimate accept or reject decisions. This post is an attempt to distill those experiences down to a set of guidelines that can help you to optimize your revisions/responses so as to maximize your chances of acceptance and minimize the number of rounds of review. It is modified from a talk that Jonathan Davies and I give to grad students in Biology at McGill.

Image credit: Nick of http://www.lab-initio.com/.

(By complete coincidence, a related post appeared today at Dynamic Ecology. And here is another recent one at Scientist Sees Squirrel.)

1. The response letter is critical

Most editors and reviewers will make their decision, or at least form a strong initial opinion, based entirely (or mostly) on the response letter, your “response to reviewer/editor comments”. This letter will be the first thing they read and – if they are satisfied with what you say – they might not even re-read the manuscript (MS) itself. Thus, you want to make sure that the editors and reviewers have all of the information optimally organized and explained in one place. Stated another way, the response letter is often just as important (maybe more so) than the changes you make to the MS itself. In preparing your letter, repeat all comments by the Associate Editor and the reviewers and make sure you respond to each immediately below. Repeat not only the negative comments you have to address but also all the positive comments. Repeating the latter is valuable because it can influence the other reviewers: “Hmmmm – the other reviewer quite likes this paper, maybe I am being a bit too harsh.”

As an aside, I often have my students write the response letter even before they modify the paper. This sequence helps to see in advance how their intended actions are likely to play out in the response letter, and doing so helps the students and coauthors to settle on the optimal set of changes. Of course, the letter will need to be modified as changes are made to the MS but it helps to settle the core elements first.

2. You need to convince the reviewers not the editor

Some people attempt to argue to the editor that the reviewer comments are not valid and should be ignored. The thought is that the editor will invalidate those comments and thereby let you proceed without addressing them. In the vast majority of cases, however, the editor will want to get the reviewers to agree to publish the paper and will be unlikely to overrule them (although it does happen). I therefore strongly suggest not trying to argue to the editor that the reviewer comments are invalid. Instead, you need to convince the reviewers, who will then help you convince the editor. In my experience, once you get a reviewer on your side, they will often then help to convince the other reviewer/editor too.

Further to the above, you should not say in your letter to the editor that the reviewer is unqualified or wrong or stupid or sloppy or anything like that. The editor usually selected the reviewers based on who they thought (or who you said) would be good reviewers – often people they know and respect and who could well be their friends. Thus saying the reviewers are unqualified is the same as insulting the judgement of the Editor. Moreover, your response letter – even if written only for the editor – will usually be provided to the reviewers. Thus, something you think you are saying in confidence to the Editor will often make it to the very person whom you are criticizing, which will only further bias them against your paper. Of course, there are exceptions when a reviewer really is personally insulting or overtly biased, in which case you should politely notify the Editor that the reviewer’s comments can be interpreted to be inappropriate. Of course, the other, more modern, strategy is to berate the journal on social media, which can lead them to issue a formal apology. However, I would not advocate this except in extreme cases, such as #addmaleauthorgate.

How editors select reviewers - Grod et al. (2010) Front. Ecol. Evol. 

3. (Re)define the problem

Reviews are often very extensive and different reviewers want different things – often many different things – some of which you can do and some of which you can’t. This can lead to very long and tedious response letters that serve to annoy and alienate the editor and reviewers. In such cases, it helps to define the critical problems at the outset of the response letter (sometimes the editor has helpfully defined them for you.) In essence, you write – just after the editor provides the key points – a short section that explains what you perceive to be the main criticisms, often rephrased in a way that best matches what you intend to do/argue about them. You then provide a short and focused explanation of how you have solved that problem or how, fortunately, the problem does not exist or isn’t too critical. The idea here is get the editor and reviewers focused on just a few key issues and show in a succinct and clear way how you have dealt with them. The implication (and truth) is then that the rest of the comments are really just minor things that didn’t influence the reviewer’s hesitation in accepting your MS. Thus, by showing right up front how you deal with the critical issues that you (or the Editor) think influence acceptance of the paper, you can set them to thinking right away that all will be well.

Types of reviewers - for more details see the awesome page:
http://matt.might.net/articles/peer-fortress/

4. Make actual changes to the MS whenever possible

You can respond to reviewer comments in two ways. First, you can change the MS to accommodate/ameliorate/fix the suggestions or criticisms. Second, you can try to argue your way out of making any changes. Take the first of these two options whenever possible. That is, whenever possible, DO something to the MS that helps to address each comment; and make sure to state that you have done so in your responses. However, it is likely that some reviewer suggestions are either impossible to implement or would actually make the MS worse or would totally change your intended meaning in a way you feel inappropriate. In such cases, you will need to argue your point. However, choosing to argue a specific point also means that you will want to have addressed as many of the other reviewer comments as possible with actual changes to the MS. Making these changes builds good credit with the reviewer and can give you some “free passes” on things you can’t (or don’t really want to) change. In essence, you should be careful to pick your battles, as the more you fight the more likely you are to lose the war. In some of the more difficult cases, you can meet the reviewer halfway by making some partial change (altering a graph, adding a new analysis in the supplementary materials, adding a qualifying statement), and I recommend you do this whenever possible.

As an aside , it is good to do these things even if your paper is rejected and then submitted elsewhere as it is reasonably common to get the same reviewers, who are annoyed at having to say the same thing they said previously and you seemingly ignored. And they really like it if you did what they suggested even if your paper was rejected. This happened to me once. I made many comments on a paper that got rejected from Journal A. I was then asked to review the same paper for Journal B. The author had - even though under no obligation to do so given the switch of journal - implemented essentially all of my suggestions, which were not trivial. I was very impressed and pleased and had many positive things to say about the MS, which as published in Journal B.

5. “I have now made this more clear in the MS”

If the reviewer is clearly wrong about something or if they missed something or misinterpreted something, never say so in as many words. Doing so can seem insulting or condescending (the reviewer missed something that you say is obvious) or it can imply that you think the reviewer is not doing a good job (as indeed they might not be).  Moreover, if the reviewer misunderstood or misinterpreted things, then other readers likely will too, and so you should change it. In essence, a reviewer’s misinterpretation of your study is YOUR fault, not the reviewer’s fault; or at least you should view it that way. In such cases, I first explain the reality and apologize for my mistake (“Fortunately, we did actually do XXXX but it was not sufficiently clear in the MS.”) and then state something like “We have now made this more clear through revisions to the text.” Of course, this means that you do indeed have to make it more clear in the MS – even if you thought it was clear to begin with.

6. Stop whining and just do the new analyses

With ever-increasing statistical sophistication (many would say over-complication), reviewers are likely to recommend some new analysis – no matter how hard you thought out and optimized your analysis in the first place. These new analyses will very rarely change any of your conclusions, and yet the reviewer thought they were important so it is not wise not to ignore them or try to argue them away. By far the simpler solution is to just do the new analysis (or graph or table) and place it in supplementary materials (if you don’t want to change the MS itself). You can then refer to this supplementary analysis in the text with a single sentence referring to the alternative analysis. Although doing the new analysis even when you are confident it won’t change the outcome can seem time-consuming and wasteful, it is even more time-consuming and wasteful to have the reviewer insist again that you do them, thus necessitating another round of revision. Just do the analyses the first time they are requested, which also lets the reviewer know you aren’t trying to weasel your way out of things because you are too busy/lazy or because you have actually done the analyses and found they go against your preferred conclusion. 

Real reviews compiled from my colleagues.

7. Be polite and respectful but not sycophantically so

Sometimes reviewers are insulting (see the above). Sometimes they clearly didn’t see something you had already put in the MS. Sometimes they appear to be complete idiots. Sometimes they are just wrong. However, you should never say any of these things in your response as reviewers will interpret them as being personal critiques. (And, as noted above, the reviewers will likely see things even if you intend them only for the editor.) Thus, no matter how annoying or rude or clueless reviewers are, you can never even hint that such is the case.

Balancing the above point, you should also avoid the temptation to repeatedly say “This is a great comment” or “We thank the reviewer for their comment” and so on. If you say it for every comment, then it means nothing. Instead, save such thank yous for key places, especially where you added an entirely new analysis or, seemingly paradoxically, where you can’t make a change. In this case, you are acknowledging that it is a good point but that you are able to make the change for logistical (or other) reasons.

8. Other procedural points

(a) Don’t paste the exact revised text (or even list the line numbers*) from your paper in the response letter. First, this wastes time as multiple changes are often needed and so the revised text and line numbers keep changing as the revision proceeds. Second, it greatly expands the length of the letter because you still need to explain it. Third, cutting out a single sentence and pasting it into the response letter means that it is out of context (which leads to the next point). Instead, simply explain the change you made and refer to the general part of the paper where you made the change. (Your supervisor or coauthor might have a different opinion here and you should obviously follow theirs in such cases.) *[It has been pointed out to me since writing this that Editors do often like line numbers]

(b) A great temptation is to simply insert a sentence into the MS to address a reviewer comment. Sometimes this works but more often it is out-of-place and out-of-context and awkward – so much so that people reading the paper can clearly see sentences that were added after the fact in response to reviewer comments. Instead, you need to re-read the entire MS (or at least the changed parts) with great care to make sure that any changes you make are seamlessly and effectively integrated into the whole such that they appear to have been there from the beginning. 

Coda

Of course, the above won't always work and, indeed, I have had papers rejected on the second round of review – although rarely. However, I do think the above steps will help you out. In closing, I will leave you with the famous video of Hitler responding to a peer review of his manuscript (which is apparently a lot less funny if you can speak German.)

Other resources:

Improving the reviewing process in Ecology and Evolutionary Biology