Over-citation: my papers that should be cited less often

Ever write a great paper – an important one – and publish it to great expectations? “Surely everyone will love this paper,” you think. It is going to be a barn-burner. It is going to bust Web of Science – maybe even Google Scholar – with citations. Then, as the weeks and months and years go by, pretty much nothing happens. The paper gets a few citations (mostly from your own group), a few people seem to have read it, but not much else. And you think, “How did this happen”? “That was one of my best papers ever – it should be more widely cited.” Perhaps you start to think, “Maybe folks just missed it. If I could only get it in front of them again, people would recognize its greatness and it would go viral.” So you write a blog about “hidden gems” or you emphasize the paper on your website or you send out a few tweets or all of the above. And …. nothing happens. So you carry a (mild) resentment to your retirement, where you give your “exit seminar” and talk about your great work that just didn’t get the attention it deserved. (Yes, I have seen this happen.) Well, this post is about the exact opposite situation – papers that get way more attention than they deserve.

When one applies for a research grant, one usually has to talk about how wonderful one is – at least partly in relation to publications and citations. This need usually takes one to Web Of Science or Google Scholar to find out numbers of citations and H-indices and so on. Whenever I do this (such as yesterday while preparing a grant application), I see my top cited papers. I look at some of them and think, “Well, yeah, that paper was indeed useful and influential” but, about the same amount of time, I think “What the hell, why does THAT paper have so many citations?” So, I thought I would here take the opposite tack to the usual “papers of mine that should be cited more” and write about “papers of mine that should be cited less.” In doing so, I first need to point out that there isn’t anything wrong with these papers, they simply seem to have received more attention (or at least citations) than their content might deserve – or that we, as authors, expected.

One choice for an over-cited paper might be a short note we published in Conservation Biology about how species distribution models that predict massive extinction under climate change generally ignore evolution and are therefore probably often wrong. Models of this sort look at the abiotic conditions where a species is currently found, ask how the geographical distribution of those conditions is expected to change into the future, and then – if the conditions currently occupied by a given species in a given area shrink excessively – make a prediction of likely extinction. The problem, of course, is that species might evolve to occupy the changing abiotic conditions as selection forces them to do so – which is the only point we made in this paper. This point is certainly correct and many papers have now shown that such modelling is likely to be wrong much of the time, partly because of evolution. Yet it just seems so obvious as to not warrant a citation and – really – all our note did was point out that evolution could be rapid and that it could cause a mismatch between predicted and realized future species distributions. Does this rather obvious insight in a very small note really deserve 200+ citations in 7 years?

And the third most cited paper on Eco-Evolutionary Dynamics is ....
(coauthors redacted to protect the innocent)

Another choice for an over-cited paper might be the introduction we wrote to a Philosophical Transactions of the Royal Society special issue on Eco-Evolutionary Dynamics. The introduction simply pointed out that evolution could be rapid and that evolution could influence ecological process, before then summarized the papers in the special issue. Again, nothing wrong with the paper, but a summary of papers in a special issue is hardly cause for (soon) 300+ citations, nor is that typical of such a summary. I here assume that people are citing this paper mainly for the first two general points we make as listed above. This is fine, but excellent papers that treat eco-evolutionary dynamics as a formal research subject, rather than a talking point, are out there and should be cited more. Indeed, several papers in that special issue are precisely on that point, and yet our introduction is cited more. Similar to this example of over-citation, I could also nominate the introduction to another special issue (in Functional Ecology) – which is my fourth most cited paper (437 citations).

WTF?

Why are these “OK, but not that amazing” papers so highly cited? My guess is that two main factors come into play. The first is that these papers had very good “fill in the box” titles. For instance, our PTRSB paper is the only one in the literature with Eco-Evolutionary Dynamics being the sole words in the title. Thus, any paper writing about eco-evolutionary dynamics can use this citation to “fill in the citation box” after their first sentence on the topic. You know the one, that sentence where you first write “Eco-evolutionary dynamics is a (hot or important or exciting or developing) research topic (REF HERE)” The Functional Ecology introduction has much the same pithy “fill in the box” title (Evolution on Ecological Time Scales) and, now that I look again, so too does the Conservation Biology paper (Evolutionary Response to Climate Change.) The second inflation factor is likely that citations beget citations. When “filling in the box”, authors tend to cite papers that other authors used to fill in the same box – perhaps partly because they feel safe in doing so, even if they haven’t read the paper. (In fact, I will bet that few people who cite the above papers have actually read them.) One might say these are “lazy citations” – where you don’t have to read anything but can still show you know the field by citing the common-cited papers.

Of course, I too sometimes take the lazy citation strategy. Sometimes when I am busting out an introduction and initially write “This [topic here] is a (hot or important or exciting or developing) research area (REF HERE)”, I simply fall back to my usual set of citations that I haven’t looked at for years and years. Doing so is a quick, easy, and safe way to simply move on to the more interesting stuff that really requires reading papers. Or, if I don’t know what to cite, but I know I am stating a well-known fact, I will simply search for the topic on Google Scholar to see what is most cited and then check the title and abstract to make sure citing it is safe. Perhaps this is a bad scholarship – or perhaps it is clever efficiency in the sense that these citations don’t really matter. They are generally known phenomena that have been discussed before and for which detailed additional reading would simply be a waste of time – so I am not exactly condemning “lazy citations” here.

My final closing point is that numbers of citations to a paper don’t always reflect the originality, importance, and quality of the paper. Sometimes papers are dramatically under-cited given their quality and potential importance. Sometimes papers are dramatically over-cited given their quality and importance. Of course, this point isn’t a new one but perhaps I am making it in a slightly novel way.

---------------------------------

Notes:

1.       Patrick Nosil first pointed out to me the “fill in the box” citation-inflation phenomenon.

2.       While writing this post, I noticed that the Google Scholar link for the Conservation Biology paper doesn’t even list me as an author – irony!

3.       No disrespect to my co-authors on the papers discussed above. In fact, my favorite part of all of the above papers was the collaborative writing efforts they involved. Clearly, we did a great job in the writing!

4.       Of course, I have my own papers that I think are way under-cited, particularly several awesome ones published in PLoS ONE (an analysis here). Check it how Humans are less morphologically variable (within populations) than are other animals and Bear predation drives the evolution of salmon senescence in unexpected ways. (And, no, I didn’t write this post simply to plug these under-cited papers.)