Previous posts in this "How to" series
4. How to choose a journal (+ part 2)
5. How to be a reviewer/editor
6. How to get into graduate school
7. How to succeed in graduate school - Part 1
8. How to succeed in graduate school - Part 2
6. How to get into graduate school
7. How to succeed in graduate school - Part 1
8. How to succeed in graduate school - Part 2
Now its time for: How to respond to reviewers
http://evol-eco.blogspot.ca/2011_06_01_archive.html |
Students starting in science might imagine that decisions on
papers submitted to journals are rather straightforward outcomes such as “accept”
or “reject” or seemingly "strong reject". The reality, however, is that most decisions are more on the order
of “reconsider after revision” or “reject without prejudice,” or in their more
modern forms, “reject with resubmission allowed” or “rejected in present form.”
These more ambiguous decisions necessitate a careful revision of your paper in
response to reviewer [more properly “referee”] comments, and a cover letter or “response
to reviewers” that explains how you dealt with each comment. These revisions
and responses are critical to the future of your paper, and the manner in which
you implement them will make all the difference between whether your paper
ultimately falls into the “accept” or “reject” category. In most of the above
categories other than “reject,” your chance of acceptance is actually reasonably
high provided you do a good job of the revisions and responses.
Having now published more than 150 papers, having reviewed
more than 350 papers, and having been an Associated Editor (AE) for five journals, I
have performed and encountered seemingly endless revisions and responses. Some
work well and others don’t, and these alternatives typically translate into
ultimate accept or reject decisions. This
post is an attempt to distill those experiences down to a set of guidelines
that can help you to optimize your revisions/responses so as to maximize your
chances of acceptance and minimize the number of rounds of review. It is modified from a talk that Jonathan Davies and I give to grad students in Biology at McGill.
Image credit: Nick of http://www.lab-initio.com/. |
(By complete coincidence, a related post appeared today at Dynamic Ecology. And here is another recent one at Scientist Sees Squirrel.)
1. The response letter is critical
Most editors and reviewers will make their decision, or at least
form a strong initial opinion, based entirely (or mostly) on the response letter,
your “response to reviewer/editor comments”. This letter will be the first
thing they read and – if they are satisfied with what you say – they might not
even re-read the manuscript (MS) itself. Thus, you want to make sure that the editors and
reviewers have all of the information optimally organized and explained in one
place. Stated another way, the response letter is often just as important
(maybe more so) than the changes you make to the MS itself. In preparing your
letter, repeat all comments by the Associate Editor and the reviewers and make
sure you respond to each immediately below. Repeat not only the negative
comments you have to address but also all the positive comments. Repeating the
latter is valuable because it can influence the other reviewers: “Hmmmm – the other
reviewer quite likes this paper, maybe I am being a bit too harsh.”
As an aside, I often have my students write the response
letter even before they modify the paper. This sequence helps to see in advance
how their intended actions are likely to play out in the response letter, and doing
so helps the students and coauthors to settle on the optimal set of changes. Of
course, the letter will need to be modified as changes are made to the MS but
it helps to settle the core elements first.
2. You need to convince the reviewers not the editor
Some people attempt to argue to the editor that the reviewer
comments are not valid and should be ignored. The thought is that the editor
will invalidate those comments and thereby let you proceed without addressing
them. In the vast majority of cases, however, the editor will want to get the
reviewers to agree to publish the paper and will be unlikely to overrule them
(although it does happen). I therefore strongly suggest not trying to argue to
the editor that the reviewer comments are invalid. Instead, you need to
convince the reviewers, who will then help you convince the editor. In my
experience, once you get a reviewer on your side, they will often then help to
convince the other reviewer/editor too.
Further to the above, you should not say in your letter to
the editor that the reviewer is unqualified or wrong or stupid or sloppy or
anything like that. The editor usually selected the reviewers based on who they
thought (or who you said) would be good reviewers – often people they know and
respect and who could well be their friends. Thus saying the reviewers are unqualified
is the same as insulting the judgement of the Editor. Moreover, your response
letter – even if written only for the editor – will usually be provided to the
reviewers. Thus, something you think you are saying in confidence to the Editor
will often make it to the very person whom you are criticizing, which will only
further bias them against your paper. Of course, there are exceptions when a reviewer
really is personally insulting or overtly biased, in which case you should
politely notify the Editor that the reviewer’s comments can be interpreted to
be inappropriate. Of course, the other, more modern, strategy is to berate the journal
on social media, which can lead them to issue a formal apology. However, I
would not advocate this except in extreme cases, such as #addmaleauthorgate.
How editors select reviewers - Grod et al. (2010) Front. Ecol. Evol. |
3. (Re)define the problem
Reviews are often very extensive and different reviewers
want different things – often many different things – some of which you can do and
some of which you can’t. This can lead to very long and tedious response
letters that serve to annoy and alienate the editor and reviewers. In such
cases, it helps to define the critical problems at the outset of the response
letter (sometimes the editor has helpfully defined them for you.) In essence,
you write – just after the editor provides the key points – a short section
that explains what you perceive to be the main criticisms, often rephrased in a
way that best matches what you intend to do/argue about them. You then provide
a short and focused explanation of how you have solved that problem or how,
fortunately, the problem does not exist or isn’t too critical. The idea here is
get the editor and reviewers focused on just a few key issues and show in a
succinct and clear way how you have dealt with them. The implication (and
truth) is then that the rest of the comments are really just minor things that
didn’t influence the reviewer’s hesitation in accepting your MS. Thus, by
showing right up front how you deal with the critical issues that you (or the
Editor) think influence acceptance of the paper, you can set them to thinking
right away that all will be well.
Types of reviewers - for more details see the awesome page: http://matt.might.net/articles/peer-fortress/ |
4. Make actual changes to the MS whenever possible
You can respond to reviewer comments in two ways. First, you
can change the MS to accommodate/ameliorate/fix the suggestions or criticisms.
Second, you can try to argue your way out of making any changes. Take the first
of these two options whenever possible. That is, whenever possible, DO
something to the MS that helps to address each comment; and make sure to state that
you have done so in your responses. However, it is likely that some reviewer
suggestions are either impossible to implement or would actually make the MS
worse or would totally change your intended meaning in a way you feel
inappropriate. In such cases, you will need to argue your point. However, choosing
to argue a specific point also means that you will want to have addressed as
many of the other reviewer comments as possible with actual changes to the MS.
Making these changes builds good credit with the reviewer and can give you some
“free passes” on things you can’t (or don’t really want to) change. In essence,
you should be careful to pick your battles, as the more you fight the more
likely you are to lose the war. In some of the more difficult cases, you can
meet the reviewer halfway by making some partial change (altering a graph,
adding a new analysis in the supplementary materials, adding a qualifying
statement), and I recommend you do this whenever possible.
As an aside , it is good to do these things even if your paper is rejected and then submitted elsewhere as it is reasonably common to get the same reviewers, who are annoyed at having to say the same thing they said previously and you seemingly ignored. And they really like it if you did what they suggested even if your paper was rejected. This happened to me once. I made many comments on a paper that got rejected from Journal A. I was then asked to review the same paper for Journal B. The author had - even though under no obligation to do so given the switch of journal - implemented essentially all of my suggestions, which were not trivial. I was very impressed and pleased and had many positive things to say about the MS, which as published in Journal B.
As an aside , it is good to do these things even if your paper is rejected and then submitted elsewhere as it is reasonably common to get the same reviewers, who are annoyed at having to say the same thing they said previously and you seemingly ignored. And they really like it if you did what they suggested even if your paper was rejected. This happened to me once. I made many comments on a paper that got rejected from Journal A. I was then asked to review the same paper for Journal B. The author had - even though under no obligation to do so given the switch of journal - implemented essentially all of my suggestions, which were not trivial. I was very impressed and pleased and had many positive things to say about the MS, which as published in Journal B.
5. “I have now made this more clear in the MS”
If the reviewer is clearly wrong about something or if they
missed something or misinterpreted something, never say so in as many words. Doing
so can seem insulting or condescending (the reviewer missed something that you
say is obvious) or it can imply that you think the reviewer is not doing a good
job (as indeed they might not be). Moreover,
if the reviewer misunderstood or misinterpreted things, then other readers
likely will too, and so you should change it. In essence, a reviewer’s misinterpretation
of your study is YOUR fault, not the reviewer’s fault; or at least you should
view it that way. In such cases, I first explain the reality and apologize for
my mistake (“Fortunately, we did actually do XXXX but it was not sufficiently
clear in the MS.”) and then state something like “We have now made this more clear
through revisions to the text.” Of course, this means that you do indeed have
to make it more clear in the MS – even if you thought it was clear to begin
with.
6. Stop whining and just do the new analyses
With ever-increasing statistical sophistication (many would say
over-complication), reviewers are likely to recommend some new analysis – no matter
how hard you thought out and optimized your analysis in the first place. These
new analyses will very rarely change any of your conclusions, and yet the
reviewer thought they were important so it is not wise not to ignore them or
try to argue them away. By far the simpler solution is to just do the new
analysis (or graph or table) and place it in supplementary materials (if you
don’t want to change the MS itself). You can then refer to this supplementary
analysis in the text with a single sentence referring to the alternative
analysis. Although doing the new analysis even when you are confident it won’t
change the outcome can seem time-consuming and wasteful, it is even more time-consuming and wasteful to have the reviewer insist again that you do them, thus
necessitating another round of revision. Just do the analyses the first time
they are requested, which also lets the reviewer know you aren’t trying to weasel
your way out of things because you are too busy/lazy or because you have
actually done the analyses and found they go against your preferred conclusion.
7. Be polite and respectful but not sycophantically so
Sometimes reviewers are insulting (see the above). Sometimes they clearly
didn’t see something you had already put in the MS. Sometimes they appear to be
complete idiots. Sometimes they are just wrong. However, you should never say
any of these things in your response as reviewers will interpret them as being
personal critiques. (And, as noted above, the reviewers will likely see things
even if you intend them only for the editor.) Thus, no matter how annoying or
rude or clueless reviewers are, you can never even hint that such is the case.
Balancing the above point, you should also avoid the
temptation to repeatedly say “This is a great comment” or “We thank the
reviewer for their comment” and so on. If you say it for every comment, then it
means nothing. Instead, save such thank yous for key places, especially where
you added an entirely new analysis or, seemingly paradoxically, where you can’t
make a change. In this case, you are acknowledging that it is a good point but
that you are able to make the change for logistical (or other) reasons.
8. Other procedural points
(a) Don’t paste the exact revised text (or even list the line
numbers*) from your paper in the response letter. First, this wastes time as
multiple changes are often needed and so the revised text and line numbers keep
changing as the revision proceeds. Second, it greatly expands the length of the
letter because you still need to explain it. Third, cutting out a single
sentence and pasting it into the response letter means that it is out of
context (which leads to the next point). Instead, simply explain the change you
made and refer to the general part of the paper where you made the change. (Your
supervisor or coauthor might have a different opinion here and you should
obviously follow theirs in such cases.) * [It has been pointed out to me since writing this that Editors do often like line numbers]
(b) A great temptation is to simply insert a sentence into the
MS to address a reviewer comment. Sometimes this works but more often it is
out-of-place and out-of-context and awkward – so much so that people reading
the paper can clearly see sentences that were added after the fact in response
to reviewer comments. Instead, you need to re-read the entire MS (or at least
the changed parts) with great care to make sure that any changes you make are seamlessly
and effectively integrated into the whole such that they appear to have been
there from the beginning.
Coda
Of course, the above won't always work and, indeed, I have had papers rejected on the second round of review – although rarely. However, I do think the above steps will help you out. In closing, I will leave you with the famous video of Hitler responding to a peer review of his manuscript (which is apparently a lot less funny if you can speak German.)
Other resources:
Improving the reviewing process in Ecology and Evolutionary Biology
Coda
Of course, the above won't always work and, indeed, I have had papers rejected on the second round of review – although rarely. However, I do think the above steps will help you out. In closing, I will leave you with the famous video of Hitler responding to a peer review of his manuscript (which is apparently a lot less funny if you can speak German.)
Other resources:
Improving the reviewing process in Ecology and Evolutionary Biology