Disclaimer: the following story may or may not have actually happened as I describe it, but it does happen; it has happened to my own students, students in my graduate program, and to me personally.
I recently met with a distressed graduate student, whose identity I will conceal. Let's call this person 'Bob'. Bob met with a visiting seminar speaker, and gave a 5 minute pitch about his research on... let's say the evolution of spontaneous combustion in corals. A few months ago, Bob did what we encourage our graduate students to do: he signed up to meet with a visiting seminar speaker. Someone really well-known, highly respected. Let's call the visitor, "Dr. Doom".
Bob is a first year PhD student, and is really excited about his research plans. He presents his research plan, briefly, to Dr. Doom. You know, the five minute version of the 'elevator pitch'. Dr. Doom listens, asks some polite questions, then pronounces:
"Are you mad? This will never work! It is: [pick one or more of the following]
a) impossible to do
b) too risky
c) too expensive
d) sure to be biased by some uncontrollable variable
e) uninterpretable
f) uninteresting
How can your committee have possibly approved such a hare-brained scheme? Don't waste your time. Instead, you should do [thing Dr. Doom finds interesting]."
Bob, the first year student, is of course shattered. Here's this famous biologist, a fountain of wisdom and knowledge, crapping on Bob's idea. Bob obsesses over this criticism for weeks, considers completely changing his research. Considers dropping out of graduate school and becoming a terrorist, or an energy company executive, or something equivalent. Finally, Bob came to me. This post is about my advice to Bob. And, to Dr. Doom, whoever you may be. (Note, Dr. Doom is quite possibly a very nice and well-meaning professor who wasn't watching their phrasing carefully, for lack of coffee going into their 9th meeting of the day).
Point 1: Bob, you have spent the past 6 months obsessively thinking about your research: the theory, the hypotheses, the study system, and the methods. Maybe you have preliminary data, or perhaps not. No matter what, I can almost guarantee you that even a few months into your studies, you have thought in more depth about your experiment, than Dr. Doom has. Doom got a 5 minute elevator pitch and probably wasn't entirely paying attention the whole time (he has a grant proposal due in 10 minutes, after all), and leapt to some assumptions about your ideas. You have thought this through more than Doom.
Point 2: If Dr. Doom says your hypothesis/prediction is surely false, and thus testing it not worth while, there is a chance that Dr. Doom is wrong. Let's be Bayesians for a second. You and Doom have different priors concerning your hypothesis. Dr. Doom knows a great deal, it is true. But that does not mean that Doom's priors are correct. You have chosen to work on a question whose answer is, I hope, not fully known. That's the point of doing it, right? So, in truth neither you nor Doom know the correct answer. As long as you have some good reason for your prior, then proving Dr. Doom wrong will be all the more worthwhile, right?
Case study: In graduate school, Tom Schoener was on my dissertation committee. I have immense respect for Tom. Such enormous respect, in fact, that I think its okay to name names here, because his skepticism really drove me to some important projects, so I owe him a great deal. You see, my PhD work was going to focus, in part, on how frequency-dependent selection acts on populations where different individuals eat different things from each other. Thus competition is more severe for common-diet individuals, less so for rare-diet individuals, generating disruptive selection that was the foundation for some models of speciation. So, I present the idea and Tom says "but Roughgarden, and Lister, and Taper, and Case, all proved that this among-individual diet variation does not happen much. You are barking up the wrong tree, your basic premise that justifies your research is wrong." [ I am paraphrasing the idea here, it has been 20 years after all]. I disagreed. So, I got together a group of fellow grad students and we reviewed the relevant literature together. The result was my most-cited paper, Bolnick et al 2003 American Naturalist on individual specialization. The very next year, Tom used this paper in the Davis Population Biology core course class reading list. The point is, the more sure your famous critic is that you are wrong, the more impactful it might be if you turn out to be right.
Point 3: If Dr. Doom says that you have a good question, but the methods are flawed, pay attention. Doom may or may not be right, but you certainly need to take a careful look and give your approach a rethink. That's not a reason to abandon the work, but it is a chance to maybe tweak and improve your protocol to avoid a possible mistake. Sometimes Doom has a point. But, a fixable one.
Case study: My PhD student Chad Brock wanted to study stickleback male color differences between lakes. We hosted a certain Dr. Hend- er -Doom as a seminar speaker. Dr. H told Chad that male color was too plastic: if you trap males in minnow traps, their color is changed before you can pull them out of the water, and by the time you photograph them, the data has no meaning. If you euthanize them, they change color. if you co-house them with other males, they change color. Chad was devastated, so he rethought his approach. He snorkled rather than trapped. Hand-caught males and immediately handed them to a boater, who immediately took spec readings. Over time, we learned that (1) MS-222 euthanasia keeps the color pretty well, whereas some other forms of euthanasia do not, (2) housing males singly in opaque dark containers keeps their colors just fine for hours, and (3) color is stable enough to see really strong effect sizes. So, Dr. Hend... I mean, Doom was wrong in that case (happens to the best of us), but his criticism did push us to change Chad's approach in a way that ended up yielding great dividends. By measuring color on hand-caught males from nests, we knew male nest depth (not knowable when trapping). This led to the discovery of depth-dependent variation in male color within lakes, that became Chad's actual thesis.
 |
| Stickleback from various lakes in BC |
Point 4: What seems obvious to Dr. Doom (who is, after all, an expert in the field and has read every single paper ever published), might not be obvious to other people. Doom remembers that back in 1958, somebody-or-other published an observational study with low sample size that resembled your hypothesis, and in 1978 Peter Abrams probably wrote 5 papers each containing a paragraph that could be read as establishing your idea [note, Peter Abrams truly has thought of pretty much everything, the guy is amazing]. But the rest of us were still learning to read Dr. Suess back then and haven't caught up. So, the broader community might be fertile ground for your ideas even if Dr. Doom already is on board.
Case study: In 1999 I was entranced by a couple of theory papers on sympatric speciation that were published in Nature (Dieckmann and Doebeli; Kondrashov and Kondrashov). I designed my thesis around testing whether competition drove disruptive selection, which was a core assumption in both papers' models. Soon after I designed my experiments, Alex Kondrashov himself visited. I showed him my plan, much like Bob showed Dr. Doom. I explained how I wanted to test his model experimentally, with Drosophila. I figured he'd be thrilled. But, Alex asked, "Why would you do this?" I was floored. He explained that if the experimental system exactly matched all of the assumptions of the math, then the outcome was not in question. On the other hand, if the experimental system deviated one iota from the math assumptions then it wasn't a test of the model. In short, he dismissed the very idea of an experimental test of the model's predictions. Either it is inevitable, or irrelevant. I realized that in a fundamental epistemological sense, he's not wrong. In fact, he taught me something crucial about the relationship between theory and data that day. Testing a model is a tricky thing. Often we are better off evaluating the model's assumptions, how common and strong are they, what's the slope and curvature of a function of interest? And yet, I went ahead anyway and did the experiment. The result was my second-ever paper. Bolnick 2001 Nature was a single-authored paper, confirming experimentally that resource competition drives niche diversification. I really truly owe my career to the fact that I ignored Alex's critique, and Tom Schoener's skepticism (which is why I name them, not to call them out but to thank them).
So, Doom's critique should be taken seriously, but not too close to your heart. Think hard about whether you can improve what you are doing. We should always do that, for every idea and plan, but sometimes an outside catalyst helps. But, don't jump ship right away. Doom may be wrong, biased, or just not thinking it through. Don't give up hope, but rather proceed with deliberation and caution and self-confidence. Senior famous people do not have a monopoly on being right. Far from it.
Speaking to Dr. Doom now
There is a fine line between lending constructive advice, and being a Mean Person. When we visit, we want to lend our expertise and experience to students at the host institution, give them advice and also warn them away from potential problems. We think we are doing them a favor in the process. But, when one does this clumsily, one sometimes steps into the realm of just being insensitive to their self-esteem. Be constructive, be helpful, offer advice, but don't be Dr. Doom. Not that you need to be Dr. Flowers instead, and praise everything as flawless and brilliant. The key of course is to dole out helpful constructive advice in a way that is compelling but kind.
Case study: Six months after Alex Kondrashov visited, Doug Emlen visited Davis. I showed Doug my preliminary data. He was really encouraging and enthusiastic, and more than anyone else I remember talking to he made me feel like a colleague and a potential success. Yet, in amidst the encouragement he pointed out some real concerns. I didn't feel the sting of the critique because it was delivered so deftly, yet he was right, and as that realization gradually grew on me I revised my apprroach, ultimately changing how I went about measuring my proxies for fitness.
One of the reasons departments invite seminar speakers to visit, is to encourage meetings between top scientists and the department's graduate students, postdocs, and junior faculty. The graduate students get feedback on their ideas and network with possible future postdoc mentors or colleagues. Same for postdocs. Junior faculty get to know people who might write letters for their tenure case, network with possible future collaborators. And yet, all too often we let senior faculty hog the limited meeting slots. Even when programs put a stop to that, graduate students are often reticent to sign up for meetings, especially individual meetings. I suspect one reason is, fear of the Dr. Doom's of the world. Don't be Dr. Doom. Be like Doug Emlen.
No comments:
Post a Comment