Publish - or it perishes.

This post was inspired by the following line in Lord of the Rings read to my kids while on “vacation” at my cabin in BC. “’Follow what may, great deeds are not lessened in worth,’ said Legolas. Great deed was riding the Paths of the Dead, and great it shall remain, though none in Gondor be left to sing of it in the days to come.”

If a tree falls in the woods, and no one is there to hear it, does it make it sound? NO.

I have always maintained that – no matter the research you did and its quality and results – it never existed if you don’t publish it. Unpublished research only enriches (or deriches, I suppose) you and the people that did it – it has no influence outside that limited sphere. Given that no one outside of the researchers know the results, it ceases to exist as research – in the sense that research is conducted for the benefit not just of the researchers that did it but for the wider world. So – always publish your research – even if you don’t like the results, even if you move on to other jobs, even if you lose interest. If you don’t publish it, it is a waste of taxpayer dollars and en(de)riches no one but yourself.

Exceptions exist, of course. If you know that the experiment or research was BAD – that is, all the fish died or the field assistant mixed up the data irretrievably or all the camera traps failed or whatever – then you obviously don’t want to publish it. But, importantly, the decision to NOT publish something should never be a function of the actual RESULTS of the study. If the study was conducted well, then the results are the results and should be published regardless of what the actual result is. If you predicted a positive correlation between x and y and you felt like you did the study right BEFORE you saw the results, then you need to publish it regardless of whether the correlation is positive or negative or non-existent. Stated another way, your perception of the quality of the research should not change AFTER you see the outcome of the research – otherwise what is the point of conducting the research in the first place.

If you don’t publish your results, you run the risk of a confirmation bias (only publishing results if the conform to prior expectations), a file-drawer problem, wasting taxpayer money, wasting future researcher’s time and resources, and so on.

If a tree falls in the woods, and people are there to hear it – but then they all die afterward and leave no descendants, did it make a sound? YES.

Many people are disappointed when they publish a study but it ends up in a “lower-tier” journal after they first tried top-tier journals where they thought it belonged; or they publish the paper and few people cite it. These are reasonable feelings for a researcher to have, of course. You poured your heart and soul – and blood and sweat and tears – and money and time and resources – into your study; and the results were cool and graphs are engaging and you did an awesome job writing it up and so on. If it doesn’t shake the foundations of your field, or at least cause them to quiver, then you feel let down; like your research wasn’t that good after all. Critically, however, what matters – once your work is published – is how YOU feel about it. Are you proud of it? If so, then external validation is nice, but not important in the end.

My daughter has a t-shirt that reads “Don’t let the number of likes define your art.” – to which I like to jokingly add “unless you get a lot of likes.” The sentiment of the original saying is what I am talking about above, of course; if you like your research, then it is good! The tongue-in-cheek addition, however, also acknowledges that perhaps you don’t see the worth of your own work – but that others do. [See my post on "Should I be Proud of my H-index?"]This can happen if you have spent so much time and endless revising and reanalyzing on a study that you are simply deathly sick of it – and just want to be done with it so you can move on to good research in your future (or some non-research endeavor).

So, be proud of your research if others like it – or even if only you like it. But, regardless, you need to publish it first.

What Associate Editors (ideally) do

The following is a cross-post by Dan Bolnick, copied verbatim from his blog platform as Editor-In-Chief of The American Naturalist. The blog was written aimed at new Associate Editors (AEs), to articulate the journal's aspirations for the role of AEs. Unlike many journals, AmNat AEs often write more extensive and helpful comments than the reviewers. I have seen many cases where papers were fundamentally changed for the better, really transformed, by careful Associate Editor feedback. While some authors find this extent of feedback off-putting (not to name any names, Andrew), many authors respond very positively to the care that we put into our decisions. This becomes a cultural norm, as repeat authors may become new Associate Editors who pay it forward. The following essay is meant to articulate these norms. Although it is aimed at AmNat AEs, I am reposting it here to reach a broader audience of authors, reviewers, and AEs for other journals.

-Dan Bolnick












_________________________________________________________________________________________________________________________________________________________________

Associate Editors that join The American Naturalist's editorial board are given a set of detailed guidelines for the practical side of being an AE: how to use Editorial Manager, for instance, to select reviews or return a recommendation to the Editors once reviews are in. But there are also parts of this job that we assume Associate Editors know, without necessarily articulating it clearly. We assume that new AEs know the journal's editorial style from having been reviewers for us, and having received reviews and decisions when submitting to us. But, it always helps to make some expectations explicit, so the Editors of the journal decided we should more clearly articulate our vision of the role of Associate Editors at The American Naturalist.

There is a lot of negativity about the peer review process today. In person and on social media, scientists love to complain about the reviews that seem unreasonable, the decisions that felt cursory. In contrast, our journal office regularly receives ‘thank you’ emails, even from authors whose papers we declined. For example, last week I received a thank-you from an author whose paper I declined without even sending to an AE. I had spotted what I considered a fundamental logical flaw in the paper, and tried to kindly but firmly explain why it was a problem, the author couldn’t claim their data was evidence for the phenomenon they claimed. The authors used that feedback to rethink their approach to interpreting the data, collected additional data, and ultimately generated a stronger paper at another equally-good journal. I’ve seen papers we rejected end up in Ecology Letters and other high-end journals. I don’t see that as a failure on our part, but rather a success: we helped authors improve their papers, contributing to the quality of the published literature, even if it ended up at a competing journal. Of course, this is only a helpful contribution on our part if our reviews and decision letters pushed the authors to go an extra step to improve the clarity of their writing, accuracy of their analysis and interpretation, or add key data.

Authors recognize this value-added. At the Evolution meeting this summer I had numerous people praise the review process at the American Naturalist. I’ve seen similarly complimentary comments on Twitter. And I think this is a key part of our brand. Yes, we aim to publish conceptually innovative cross-cutting work. But we also aim to provide a positive review and decision-making experience, whatever the outcome. So, what does that mean for our expectations for you, our Associate Editors?

1)      Look at the paper carefully before review, and decide if it is worth using reviewer time. At this stage, I normally take notes that help me later with writing a decision letter.

2)      If you believe it is not a sufficiently important advance, or has flaws that will raise serious barriers to publication, write a review that clearly explains your logic. Editorial declines without review, by you or by me, should come with enough of a review that authors are convinced we read the paper carefully, and thought about it. It should contain enough feedback that authors feel the submission was worthwhile even if they did not get accepted, because it helps them adjust their approach for the next journal.

3)      Sometimes, a paper has real but unrealized potential: there’s a gem of an idea, but it is buried in shoddy writing, a flawed model or statistical analysis, poor graphics.  You know it’s not going to survive peer review very well, but you think it mightstand a good chance if the authors fix it up first. This is where the Editorial Decline Without Prejudice (DWOP) comes into play. You have a chance to send this back to the authors with a clear statement of what the value of the paper could be, and what needs to happen to realize that potential. You are doing everyone a favor here: the authors have a better shot at success, you save the reviewers by helping them see a better paper first. I’ve found that authors are shocked and thrilled to have an editor or AE give them pre-review feedback to improve their chances with reviewers. This strategy shouldn’t be applied to all papers, but to those with really high value but which will almost surely encounter serious but avoidable reviewer resistance.

4)      When reviews come in, write a review of your own. I tended to first read the reviews, then I would revisit the paper’s text, figures, and tables and any notes I took on my first look through. I would then write a review that includes:

a.      My own feedback on the paper, on points that the reviewers may have missed.

b.      A summary of the essential points from reviewers (and my own reading) that either preclude further consideration, or that must be dealt with in a revision.

c.       If you disagree with the reviewers on some point, or need to arbitrate between conflicting reviews,  do so while being respectful to the reviewers. This is important when the reviewers ask for something unreasonable, or especially when they express themselves in an overly aggressive or negative manner. You are an arbiter who can provide a buffer between the authors and a mean or thoughtless reviewer. Luckily, I rarely feel like this is an issue, which speaks well of our reviewers, but it does happen sometimes.

5)      Keep a lookout for ‘diamonds in the rough’: a paper with a great underlying idea, a unique dataset, that might get negative reviews in its current form, but which might be truly great with work. Sometimes one is tempted to just knee-jerk decline papers that get negative reviews. But look closely. Some of our best papers were met with initially very critical reviews, or editorial DWOPS that might have been declines. I want to especially draw your attention to this blog post by Meg Duffy (now one of our AEs):

https://dynamicecology.wordpress.com/2012/10/16/the-paper-that-ecology-rejected-that-later-won-the-mercer-award/

In this post, Meg describes a paper that was initially rejected from Ecology, and might easily have been rejected from AmNat next. But, then-AE Yannis Michalakis delved deeply into the paper. Through a series of revisions, his careful recommendation letters led Meg and co-author Spencer Hall (also now an AE) to hone the paper into a publication that went on to win the Ecological Society of America’s Mercer Award. I think we’d all aspire to be the AE who helps hone an initially rough submission into a citation classic. 

6)      Note, of course, that this doesn’t mean you should never decline papers. Not everything is a citation-classic in the making. We should also avoid endless cycles of revision and re-review, and you should feel free to recommend "decline" for papers that don't improve after resubmission or revision. Authors are likely to be upset if their paper ultimately gets declined after multiple rounds of revision, so it’s best to try to assess how likely the authors are to be able to address the editors’ and reviewers’ major concerns after the first round of review or first revision, and decline papers that are unlikely to improve sufficiently to meet our requirements. Again, it is rare for us to hear from an irate author, but when we do it is almost always because their paper got declined after two or more reviews.

7)      Keep in mind that, at Am Nat, the three Editors also play an active role in assessing papers and reviews, and that the Editor might occasionally disagree with your recommendation or might feel the need to seek additional advice on a paper. Please don’t feel offended when this happens. Editors see many manuscripts and try to ensure that all submissions are treated fairly while also monitoring our overall acceptance rate. Editors may often change a recommendation of “Major Revision” to “DWAP” or vice versa, or might change a “DWOP” to a “Decline”. Sometimes this is because we see something differently than you do. In cases of more serious disagreement, the Editor may discuss the manuscript with you and try to reach a consensus. More often, it is because we are trying to manage the bulk flow of papers into print. If we accept too many papers, we get a backlog and authors get upset about delayed printing. If we accept too few papers per month for a while, we might have too few articles to physically bind together into a print issue. So, one of the Editors’ jobs is to quietly manage the overall acceptance rate. The most effective way for us to do this is to nudge papers from Decline to DWOP or vice versa, or from DWOP to major revision or back.

8)   Find the right balance between pushing authors to improve their writing, while allowing them to retain their own voice as writers. It sometimes happens that a great scientific idea comes wrapped in hard-to-read prose. It is okay to guide authors through the difficult process of improving their writing. It can make all the difference between a great idea that nobody reads, and a great idea that moves the field in a new direction. But, try not to push authors out of a perfectly sound and readable writing style, that might differ from your own.

9)  Make a decision.  You don't always need to take reviewers' time, especially when receiving a revision. If the authors have made substantial technical changes, that you are not in a position to evaluate, then of course send a revision back out for review. But in general our first instinct should be to read the response letter and manuscript ourselves to decide whether the authors satisfied the reviewers' concerns. If so, check the manuscript for any remaining concerns, and try to reach a final decision. For more about re-review, and also the DWOP / Decline distinction, see a previous editors blog post here: http://comments.amnat.org/2017/11/an-open-letter-from-incoming-amnat.html 

In sum, handling a paper as AE for AmNat requires a bit more than a typical review that you might do for another journal.  I want to articulate these expectations explicitly to discourage the temptation to write relatively cursory decision letters, that mostly just summarize a couple of key points from the reviews. I understand that temptation. We editors sometimes succumb it ourselves by just pointing to your recommendations or the reviews, without adding much (especially on days with a half dozen decision letters to write). But I always strive to have at least a few insights of my own to add. When authors know that the AE and Editor have read their paper carefully enough to have their own opinions (as opposed to echoing reviewers’ opinions), they feel like their paper has been given a fair evaluation. 

Dan Bolnick

Editor-in-Chief

American Naturalist

July 2019