Nearly every context for evaluation in academia (or, really,
anything) focuses on recent progress. What grants have your received in the last
five years? What papers have you published in the last five years? How many
students have you supervised in the past five years? I understand the logic behind
this approach and I am not necessarily criticizing it overall. After all, why
should a formerly influential faculty member get to keep hanging onto funding
and space for decades after they became “dead wood” in the department.
However, a few years ago, I encountered a pretty weird version of this “what have you done lately” approach to evaluation. I was an external member of a committee seeking to hire a professor. During the meetings to evaluate candidates, one other member of the committee used, as a key criterion, the trajectory of citations in Google Scholar to judge whether a person was “declining in influence”. I found this criterion really weird as a person with few citations in recent years could rank higher than a person with tons of citations in the same year. (Of course, other valid reasons exist to not weight citations highly in evaluations in general.)
And then I looked at my own Google Scholar profile. Lo and behold, I was “declining in influence.” OK, fine, whatever: my happiness does not depend on the trajectory of my citations.
Now, a few year’s later, however, it looks like this.
Now that I am on the upswing again, I am more convinced than ever of the uselessness of the trajectory of citations for evaluation. Here are two key reasons:
1. As citations always lag publications, the period of my declining influence was – presumably – precisely the time I was publishing papers that would prove to be increasingly influential.
2. Any career is likely to involve ups and downs in year-to-year citation rates, which clearly do not reflect any real trend in “influence.”
In short, while I agree that old less-than-active folks shouldn’t continue to take funds and space from more up-and-comers, citation trajectories is NOT a way to make such assessments.
Came across this post by accident - I share concerns about putting too much trust in metrics, but this specific argument doesn't convince me.
ReplyDeleteIt seems you are saying the metric is useless or invalid because it fluctuates - well, so does the annual temperature, but that doesn't mean that we can't infer trends (e.g. that the long-term trend is that it's getting warmer).
If I want to use future citations as a criterion for hiring (I'm not saying that this is a good idea, but it seems your committee did), then I would think that looking at the time series is a logical starting point to get an estimate.