Thursday, November 21, 2019

The parable of Bob and Dr. Doom. Or, "When the seminar speaker craps on your idea"

By Dan Bolnick

Disclaimer: the following story may or may not have actually happened as I describe it, but it does happen; it has happened to my own students, students in my graduate program, and to me personally.


I recently met with a distressed graduate student, whose identity I will conceal. Let's call this person 'Bob'.  Bob met with a visiting seminar speaker, and gave a 5 minute pitch about his research on... let's say the evolution of spontaneous combustion in corals.  A few months ago, Bob did what we encourage our graduate students to do: he signed up to meet with a visiting seminar speaker. Someone really well-known, highly respected. Let's call the visitor, "Dr. Doom".



 Bob is a first year PhD student, and is really excited about his research plans. He presents his research plan, briefly, to Dr. Doom. You know, the five minute version of the 'elevator pitch'.  Dr. Doom listens, asks some polite questions, then pronounces:

"Are you mad? This will never work! It is: [pick one or more of the following]
a) impossible to do
b) too risky
c) too expensive
d) sure to be biased by some uncontrollable variable
e) uninterpretable
f) uninteresting
How can your committee have possibly approved such a hare-brained scheme? Don't waste your time. Instead, you should do [thing Dr. Doom finds interesting]."


Bob, the first year student, is of course shattered. Here's this famous biologist, a fountain of wisdom and knowledge, crapping on Bob's idea. Bob obsesses over this criticism for weeks, considers completely changing his research. Considers dropping out of graduate school and becoming a terrorist, or an energy company executive, or something equivalent. Finally, Bob came to me. This post is about my advice to Bob.  And, to Dr. Doom, whoever you may be. (Note, Dr. Doom is quite possibly a very nice and well-meaning professor who wasn't watching their phrasing carefully, for lack of coffee going into their 9th meeting of the day).

Point 1:  Bob, you have spent the past 6 months obsessively thinking about your research: the theory, the hypotheses, the study system, and the methods. Maybe you have preliminary data, or perhaps not. No matter what, I can almost guarantee you that even a few months into your studies, you have thought in more depth about your experiment, than Dr. Doom has. Doom got a 5 minute elevator pitch and probably wasn't entirely paying attention the whole time (he has a grant proposal due in 10 minutes, after all), and leapt to some assumptions about your ideas. You have thought this through more than Doom.

Point 2:  If Dr. Doom says your hypothesis/prediction is surely false, and thus testing it not worth while, there is a chance that Dr. Doom is wrong. Let's be Bayesians for a second. You and Doom have different priors concerning your hypothesis. Dr. Doom knows a great deal, it is true. But that does not mean that Doom's priors are correct. You have chosen to work on a question whose answer is, I hope, not fully known. That's the point of doing it, right?  So, in truth neither you nor Doom know the correct answer. As long as you have some good reason for your prior, then proving Dr. Doom wrong will be all the more worthwhile, right?
Case study:  In graduate school, Tom Schoener was on my dissertation committee. I have immense respect for Tom.  Such enormous respect, in fact, that I think its okay to name names here, because his skepticism really drove me to some important projects, so I owe him a great deal. You see, my PhD work was going to focus, in part, on how frequency-dependent selection acts on populations where different individuals eat different things from each other. Thus competition is more severe for common-diet individuals, less so for rare-diet individuals, generating disruptive selection that was the foundation for some models of speciation.  So, I present the idea and Tom says "but Roughgarden, and Lister, and Taper, and Case, all proved that this among-individual diet variation does not happen much. You are barking up the wrong tree, your basic premise that justifies your research is wrong." [ I am paraphrasing the idea here, it has been 20 years after all]. I disagreed. So, I got together a group of fellow grad students and we reviewed the relevant literature together. The result was my most-cited paper, Bolnick et al 2003 American Naturalist on individual specialization. The very next year, Tom used this paper in the Davis Population Biology core course class reading list.  The point is, the more sure your famous critic is that you are wrong, the more impactful it might be if you turn out to be right.

Point 3: If Dr. Doom says that you have a good question, but the methods are flawed, pay attention. Doom may or may not be right, but you certainly need to take a careful look and give your approach a rethink. That's not a reason to abandon the work, but it is a chance to maybe tweak and improve your protocol to avoid a possible mistake. Sometimes Doom has a point. But, a fixable one.
Case study:  My PhD student Chad Brock wanted to study stickleback male color differences between lakes.  We hosted a certain Dr. Hend- er -Doom as a seminar speaker. Dr. H told Chad that male color was too plastic: if you trap males in minnow traps, their color is changed before you can pull them out of the water, and by the time you photograph them, the data has no meaning. If you euthanize them, they change color. if you co-house them with other males, they change color. Chad was devastated, so he rethought his approach. He snorkled rather than trapped. Hand-caught males and immediately handed them to a boater, who immediately took spec readings. Over time, we learned that (1) MS-222 euthanasia keeps the color pretty well, whereas some other forms of euthanasia do not, (2) housing males singly in opaque dark containers keeps their colors just fine for hours, and (3) color is stable enough to see really strong effect sizes. So, Dr. Hend... I mean, Doom was wrong in that case (happens to the best of us), but his criticism did push us to change Chad's approach in a way that ended up yielding great dividends. By measuring color on hand-caught males from nests, we knew male nest depth (not knowable when trapping). This led to the discovery of depth-dependent variation in male color within lakes, that became Chad's actual thesis.


Stickleback from various lakes in BC



Point 4: What seems obvious to Dr. Doom (who is, after all, an expert in the field and has read every single paper ever published), might not be obvious to other people. Doom remembers that back in 1958, somebody-or-other published an observational study with low sample size that resembled your hypothesis, and in 1978 Peter Abrams probably wrote 5 papers each containing a paragraph that could be read as establishing your idea [note, Peter Abrams truly has thought of pretty much everything, the guy is amazing]. But the rest of us were still learning to read Dr. Suess back then and haven't caught up. So, the broader community might be fertile ground for your ideas even if Dr. Doom already is on board.
Case study: In 1999 I was entranced by a couple of theory papers on sympatric speciation that were published in Nature (Dieckmann and Doebeli;   Kondrashov and Kondrashov). I designed my thesis around testing whether competition drove disruptive selection, which was a core assumption in both papers' models.  Soon after I designed my experiments, Alex Kondrashov himself visited. I showed him my plan, much like Bob showed Dr. Doom.  I explained how I wanted to test his model experimentally, with Drosophila. I figured he'd be thrilled. But, Alex asked, "Why would you do this?" I was floored. He explained that if the experimental system exactly matched all of the assumptions of the math, then the outcome was not in question.  On the other hand, if the experimental system deviated one iota from the math assumptions then it wasn't a test of the model. In short, he dismissed the very idea of an experimental test of the model's predictions. Either it is inevitable, or irrelevant. I realized that in a fundamental epistemological sense, he's not wrong. In fact, he taught me something crucial about the relationship between theory and data that day. Testing a model is a tricky thing. Often we are better off evaluating the model's assumptions, how common and strong are they, what's the slope and curvature of a function of interest?  And yet, I went ahead anyway and did the experiment. The result was my second-ever paper. Bolnick 2001 Nature was a single-authored paper, confirming experimentally that resource competition drives niche diversification.  I really truly owe my career to the fact that I ignored Alex's critique, and Tom Schoener's skepticism (which is why I name them, not to call them out but to thank them).


So, Doom's critique should be taken seriously, but not too close to your heart. Think hard about whether you can improve what you are doing. We should always do that, for every idea and plan, but sometimes an outside catalyst helps. But, don't jump ship right away. Doom may be wrong, biased, or just not thinking it through. Don't give up hope, but rather proceed with deliberation and caution and self-confidence. Senior famous people do not have a monopoly on being right. Far from it.


Speaking to Dr. Doom now
There is a fine line between lending constructive advice, and being a Mean Person. When we visit, we want to lend our expertise and experience to students at the host institution, give them advice and also warn them away from potential problems. We think we are doing them a favor in the process. But, when one does this clumsily, one sometimes steps into the realm of just being insensitive to their self-esteem. Be constructive, be helpful, offer advice, but don't be Dr. Doom. Not that you need to be Dr. Flowers instead, and praise everything as flawless and brilliant. The key of course is to dole out helpful constructive advice in a way that is compelling but kind.
Case study: Six months after Alex Kondrashov visited, Doug Emlen visited Davis. I showed Doug my preliminary data. He was really encouraging and enthusiastic, and more than anyone else I remember talking to he made me feel like a colleague and a potential success.  Yet, in amidst the encouragement he pointed out some real concerns. I didn't feel the sting of the critique because it was delivered so deftly, yet he was right, and as that realization gradually grew on me I revised my apprroach, ultimately changing how I went about measuring my proxies for fitness.

One of the reasons departments invite seminar speakers to visit, is to encourage meetings between top scientists and the department's graduate students, postdocs, and junior faculty.  The graduate students get feedback on their ideas and network with possible future postdoc mentors or colleagues.  Same for postdocs.  Junior faculty get to know people who might write letters for their tenure case, network with possible future collaborators.  And yet, all too often we let senior faculty hog the limited meeting slots. Even when programs put a stop to that, graduate students are often reticent to sign up for meetings, especially individual meetings. I suspect one reason is, fear of the Dr. Doom's of the world.  Don't be Dr. Doom. Be like Doug Emlen.


Wednesday, November 13, 2019

How to Manage Your Time - some ideas

The DRYBAR (Hendry-Barrett) lab meeting lastweek was about time management. Everyone in the group shared their strategies for best managing their time. The most important overall message was that different things worked for different people - there isn't any one-size-fits all (or even many) strategy. Many excellent and diverse ideas were raised, and so it seemed most profitable to simply share these diverse ideas. Try them out - one could work well for you. 

I (Andrew) have written a few relevant posts on this already:

How to Be Productive

From Work-Life Balance to Like-Dislike Optimization

The rest of the ideas below are from a diversity of students in the DRYBAR labs - each paragraph in each grouping is generally from a different person.

Organization

I started to use Trello for looking at all my tasks. The cool thing about it is that you can share your activities so is great for group projects too. Here's a link: https://trello.com/en Also some tips about writing goals: https://www.mindtools.com/pages/article/smart-goals.htm Personally, I like to have everything in different categories. Academic related stuff, personal stuff, ideas, etc. Something that I started recently is having something to write down random thoughts to look for future ideas, projects or anything really.


I make to do lists for separate subjects/areas of my life and most days work on my main focus (job/school) but every few days I'll take a day and do a bunch of little things from the other areas so I don't get too behind in them. It also helps that I focus all on the same thing one day so I remember things a bit better because I'm working on related tasks.

Having a weekly to-do list versus a daily to-do list helps me achieve a work-life balance. On Sunday evening, I make a list of tasks I wish to complete over the course of the next 7 days, and in my physical planner I break them down into smaller, daily activities. Even though I have a long list in front of me, subdividing tasks into smaller components spread over multiple days, allows me to feel I am advancing towards my goals. Thus, after I have achieved some items on my daily task list, I am content and can come home with small victories. Outside the office and lab, I can relax knowing tomorrow will be just as productive as today.

Avoiding distractions

Personally, I found when I started my PhD that I was easily distracted. When I ran in to little chunks of empty time (while code ran, or when I got a brief bit of writer’s block, or just lost focus while reading a paper), I’d check the news, or Facebook, or my email, and then get sucked in to that. I’d spend a lot of time “working,” but the distractions meant I never was getting as much done as I wanted. For the past couple years I’ve used  https://freedom.to/ to block these distractions. It blocks websites, but when I really need to hunker down I also have it block my email app so that I only check it once or twice a day. I’ve found it really helpful for making me more productive when I’m at work, which means I can relax more in the evenings and weekends.


I mostly try to focus on one thing at a time, but still reserve some time to work on something else. This allows me to get a break from my main activity while working on something that will be my main activity later on.

Reward systems

Be sure to take holidays and don't feel guilty about it. Do what works for you. Try things out. Sometimes it works, sometimes it won't.


Environment

Environment is also very important to me, depending on what I'm doing. If I'm writing or reading I like to go to the library in my neighborhood where there is complete silence so I can focus. Oddly enough, the coffee shop also works for me because there is so much noise that it all counteracts each other.



Exercise


Commute by bike! Almost always saves time. Also improves mental and physical health (which saves time indirectly). Managing work time is perhaps less important than managing time off. Make sure you get enough time off, and that it is is well spent doing things that make you happy and recharged for working again. Binge-watching Netflix might feel good in the moment, but can often make you feel like crap afterwards so you won’t feel rejuvenated and motivated to go back to work.


Recharging

I keep myself productive and happy as a researcher by taking frequent breaks throughout the day. Besides needing to rest my eyes after looking at a screen for prolonged periods of time, that downtime gives me an extra boost of energy to finish my task at hand. This often takes the form of brewing a cup of coffee or tea, or calling my parents to check-in. In the spirit of the Pomodoro technique, I like to take a long break after 1 hour of continuous work. 

One thing I also find important, personally, is taking a "chill" or mental health day where I watch my favorite movies and lounge around etc. I struggle with this sometimes because you feel guilty for "doing nothing" all day however doing nothing once in a while can be a good thing!

Fuck it (from Andrew's How To Be Productive Post): Go for a walk. Binge watch Game of Thrones. Read a book. Go to the climbing gym. Play guitar. Cuddle the cat (or dog). Play with the kids. Do the weekly ironing. These mental breaks will make you more efficient when you get back to work. Here is a compiled list of cool procrastination techniques of ecologists and evolutionary biologists.

Task switching

In general, switching between activities also keeps my brain stimulated and prevents me from feeling bogged down. For instance, I tend to dedicate my mornings to reading papers and answering emails, and in the afternoons I prefer to do lab work. I push forward with my morning tasks as I look forward to the exciting lab work I have planned for later on in the day.

Serial multi-task (from Andrew's How To Be Productive Post): By serial, I don’t mean do many things at the same time – unless that works for you. What I mean by serial is that, if you have multiple projects on the go, try to stay on the maximal effort-to-payoff area of the function. If one project is slowing, send it to coauthors, and work on the other projects. If one project looks like it will have a higher payoff overall (first authored papers), then work on that first.


A 25-year quest for the Holy Grail of evolutionary biology

When I started my postdoc in 1998, I think it is safe to say that the Holy Grail (or maybe Rosetta Stone) for many evolutionary biologists w...