Part 2 of a series on choosing a research topic.
|Opportuity cost, by SMBC Comics' Zach Weinersmith|
Suggestion 6: P-hack and prepare to be surprised. Having read theory, read the literature, and been observant, go back out and do something. Do a little experiment, start a little observational pilot study, just get some data. Now, do something everyone tells you not to: P-hack it. Analyze the data in many possible ways, look for relationships that you might not have identified a priori. Sure, this can lead to false positives. A lot of people argue strongly against unguided post-hoc data analysis for this reason. But we aren't at the stage yet of publishing, this is exploration, an information-finding foray. Here's a concrete example: most stickleback biologists like myself have long treated each lake as a single genetic population and assumed it is well-mixed in terms of genotypes and phenotypes (except in a few lakes with 2 species present). This has practical consequences. This past summer I watched colleagues throw 10 traps into a lake, along a mere 10 meter stretch of shoreline, then take the first trap out and it has >100 fish, so we use them and release the fish from the other 9 traps. BAD IDEA. It turns out, we now know, there is a lot of trap-to-trap variation in morphology and size and diet and genotype that arises from microgeographic variation within lakes. Here's how I got clued into this. A graduate student of mine, Chad Brock, hand-collected ~30 nesting male stickleback from each of 15 lakes in British Columbia, and immediately did spectroscopy to measure color wavelength reflectance on each male. He also happened to note the substrate, depth, and so on, of the male's nest. Six months later, back in Texas, he P-hacked, and noticed that in the first lake he was examining intensively, male color covaried with nest depth: males 0.5 meters deep were redder and males 1.5 meters deep (just meters away horizontally) were bluer. The different-colored males were within maybe 10 seconds' swimming distance of each other. This clued us in to the fact that something interesting might be going on, and we later confirmed this pattern in 10 other lakes, replicated it across years, and ultimately replicated it experimentally as well. I'm not here to tell you about our male color work though. The key point is, theory would have told me to never expect trait variation among individuals at this spatial scale, because gene flow should homogenize mobile animals at this spatial scale. But it doesn't, apparently. Here's a case where theory puts blinders on us, telling us to not bother looking for microgeographic variation. Then, when we P-hacked we were surprised and ultimately cracked open what turns out to be a very general phenomenon that we might otherwise have overlooked.