Not to
the swift, the race:
Not to the strong, the fight:
Not to the righteous, perfect grace:
Not to the wise, the light.
But often faltering feet
Come surest to the goal;
And they who walk in darkness meet
The sunrise of the soul.
Not to the strong, the fight:
Not to the righteous, perfect grace:
Not to the wise, the light.
But often faltering feet
Come surest to the goal;
And they who walk in darkness meet
The sunrise of the soul.
Excerpt from ‘Reliance’
by Henry Van Dyke
They say ‘good things come to those who wait’. I’m writing
today to say that this is at least sometimes true. We waited. And waited. And
waited. And something really good, that we are really proud of, has finally
arrived. To be more precise, Katie Peichel and Andrew Hendry and I wrote a
grant (2009), and rewrote it (2010), and rewrote it (2011), and rewrote it
(2011), and were all set to give up when we got funded. Then we hired [postdoc
Yoel Stuart, most significantly], planned, did the field work, collected the
data, analyzed the data, wrote the paper. We bounced it across several good
journals. And now, the core paper from the study has come out. We are pretty
excited, but also a bit stunned by the start-to-finish time. To put this in
perspective, the idea for this project was conceived when my first child was
this old:
 |
Me, with more hair and a 4-month-old, just before the
Evolution meeting in Minnesota in 2008 where we hatched our plan
|
and I had a lot more hair. The resulting paper was born nine
years later when she was old enough to play piano, build her own spectroscope,
and get 1st place in the Austin Regional Elementary School Science
Fair for figuring out the elemental composition of the sun:
 |
| That 4-month old can now build a homemade spectroscope and interpret Fraunhaufer Lines to infer elemental composition of the sun. This picture was taken a couple months before our paper was accepted. |
Grad students, maybe that helps you feel like your PhD isn’t taking so long now, after all.
My goal for this blog post isn’t to delve into the science
of our paper: you can read the paper, or read Yoel Stuart’s blog post on Nature.com
for that. My goal is to tell the story of the process. A story of perseverance
that maybe will make you feel better when you get that 4th grant
proposal submission rejected yet again (for this, also see the Heard-Hendryblog on serial rejection). And maybe a broader lesson about the glacial pace of science and what we
might be able to do about it.
There’s a recent trend towards Zombie remakes of classic
literature. I’m thinking specifically of Pride and Prejudice and Zombies, a
Zom-Com-Rom take on Jane Austen’s classic book. In several real senses, this is
a Zombie story too. A Sci-Zom. You’ll see why soon.
I. Zombie films
sometimes involve some initial chance encounter
Andrew Hendry and I had been doing field work on stickleback
in the same general area of Vancouver Island, for over 5 years, when we finally
crossed paths in 2006.
 |
|
The field crew gathering in 2006. Left-to-right: Kate Hudson, On Lee Lau, Dan Bolnick, Andrew Hendry (crouching), Ann-Catherine Grandchamp, Jean-Sebastian Moore, Daniel Berner (crouching),
|
His crew showed up at the cabin we rent on Vancouver Island
(Roberts Lake Resort), and crashed on our floor for about a week. We got a
bottle of good scotch in return, which Andrew drank most of (then proceeded to
build a duct-tape spiderweb in the middle of the cabin at 3 AM). Between doing
our respective field tasks, we spent a lot of time playing McGill-vs-UTAustin
ultimate games, and talking science late into the night. We came up with some
grant proposal idea, sketched out on the wobbly dining table, wrote it up that
fall, submitted it in early 2007, and got it soundly rejected later that year. The
wobbly table may have been a metaphor. Oh well. I don’t even quite recall what
we proposed to do. Something about a continuum of ecological speciation, or
lack thereof.
II. The main
characters hatch a plan to go to the woods. Always a bad idea in Zombie
stories.
Before the 2008 Evolution meeting in
Minneapolis, I emailed Katie Peichel and Andrew Hendry to say that it would be
fun to develop a collaborative project between the three of us. Something on
lake-stream evolution that fused Katie’s genetics know-how, with Andrew’s
experience with the lake-stream system. Not sure what I contributed, really.We
met after the BBQ social and sat and talked for many many many hours. What
emerged was a plan to make a plan. The general outline was clear: we wanted to
know whether ‘parallel’ evolution of lake-stream stickleback was a reflection
of parallel environments, driving parallel selection, causing changes at the
same genes, leading to heritable lake-stream differences in the same traits, in
each of many replicate lake-stream pairs (pictured).
 |
| Comida Lake and its outlet stream. Photo by Thor Veen |
We wanted to go big, with many lake-stream pairs. And we
wanted to fuse QTL mapping, genome-wide association mapping, field measures of
selection, quantitative genetics, and ecology. With this, we could ask whether
the loci that diverge most strongly & repeatedly from lake to stream (GWAS)
are also loci linked to traits (QTL) that are highly divergent (survey data)
and under divergent selection in the wild (field selection experiment). Many
different layers of data would be needed: environmental data, diet data,
parasite data, trait data, functional performance measures, genotypes, and
fitness. We left the Evolution meeting with two definite plans. First, we would
meet again in December 2008 to write a grant. Second, Andrew’s postdoc Renaud
Kaeuffer would genotype six lake-stream pairs at several hundred microsatellite
loci, test for lake-stream differences in allele frequencies, and determine
whether there is evidence for any genetic parallel evolution. This could be
preliminary data for our vaguely-defined grant idea.
III. The idyllic gathering in a guest house in the
woods. Or rather, at a winery.
In December 2008 I was in San Francisco, looking after my
1½-year-old daughter while my wife attended the American Anthropological
Association conference. At the end of the conference, Katie Peichel picked me
up in a rental car and we drove up to Napa, to the Hendry Winery.
Andrew was spending a sabbatical there, in the guest house on his uncle’s and
brother’s winery. The three of us (plus Andrew’s postdoc Renaud Kaeuffer) spent
three days there. We brainstormed the first day, expanded details the second,
and wrote outlines and some text the third day. We worked hard all day, pausing
for a long jog around the vineyard each morning. Around dinner time, we would
receive a delivery of spare bottles of wine, left over from tastings earlier in
the day. The best science of course happened in the evening after those
deliveries. You might say, we were on a Mission.
 |
| Renaud Kaeuffer, Andrew Hendry, Dan Bolnick, Katie Peichel (left to right), in Napa in 2008 |
Yesterday, I dug up some old files
from those meetings: a five-page file I wrote out a couple weeks before our
Napa meeting, to start the conversation, and a document I wrote during the
meeting, summarizing our brain-storming effort. These contained the following
seed of our subsequent work:
Is evolution repeatable and deterministic, or
idiosyncratic and stochastic? This question, elegantly framed by Gould’s ‘tape
of life’ quote, has remained a major puzzle for biologists. At its most
expansive, the question is of course unanswerable, because we are faced with a
single replicate of life's history on earth. However, we can begin to address
Gould's challenge by shifting to smaller temporal and spatial scales, for which
replication is feasible.
The key questions we identified were:
1)
To what extent is
phenotypic divergence (between lake and stream stickleback) parallel between
replicated lake-stream pairs (in different watersheds).
2)
To what extent is
the genetic basis of this divergence parallel?
a.
To be addressed
by:
i. QTL mapping
ii. GWAS across replicated clines
iii. Common garden assays of heritability
3)
To what extent is
selection parallel across lake/stream pairs?
4)
How do mate
choice and habitat choice contribute to lake-stream divergence?
After the meeting, I went home with a new outline and
proceeded to write. By January we had a
proposal. I budgeted $560,000 for field experiments and genotyping (SNP array
at the time) of many wild-caught fish. Katie budgeted $940,000 for doing a
bunch of replicate QTL maps. Andrew would share in my budget for field work and
a common-garden heritability study at McGill. We were pretty excited by the
ideas, but felt that the budget was a bit big. We couldn’t see a way to get it
smaller without compromising the science. After some worrying, we justified the
budget as appropriate for the large-scale that we planned to do. We submitted
the proposal in January 2009 (this was in the days before pre-proposals).
IV. In which the hero
of our tale (the grant proposal) dies. Several times.
It was rejected. I
have the reviews on hand (1 Excellent, 2
Excellent/Very Good, 2 Very Good, 1 Very Good/Good). Basically, they all wanted
us to do even more: more sampling, more lake-stream pairs, more environmental
and phenotypic measures, and more genetic data. And it was too expensive. Do
more, for less.
We resubmitted in January 2010. The budget was about the
same. We proposed to do more, for less money. We got 2Excellent, 1 Excellent/Very Good, 4 Very Good, and 1
Good. Concerns this time mostly centered on our plan to do genomics across
replicate clines, whether we were adequately measuring predation, and whether
the cages for our field fitness experiment would alter the environment
appreciably. All fair concerns, so we made some changes. In conversations with
the program officer, it was also clear that two of us PIs were part of the
trouble. I had both a Packard Foundation fellowship and a Howard Hughes Medical
Institute Early Career Scientist position. Although my last NSF grant had
expired three years earlier, they saw me as over-funded already. I ended up
going 5 years without any federal grant funding, relying on private foundations
instead. And Katie had NIH grants which
to NSF meant she was considered ‘rich’ as well. In fact, as a faculty member at
the Fred Hutchinson Cancer Research Institute she was expected to bring in a
lot of grants to pay her own salary, and was barely scraping by.
We resubmitted in January 2011. My budget went up to
$722,000 (some things were more expensive by then, and we added some features
to appease reviewers), Katie’s was flat at $940,000. The reviewers still didn’t like the clinal genomics
plan, and still were worried that field enclosures would modify flow regimes.
For the first time, the panel specifically recommended that we split the
proposal up, carving off some features for another grant. Said one reviewer, “Quite simply, this proposal is far too
expansive and ambitious”. And again, private conversations made it clear
that NSF saw us as not needing more money.
 |
| Our grant |
V. The zombie grant
Grant proposals, like zombies, sometimes emerge from the
cold hard ground when the main characters of our story least expect it. Our
fourth submission, which got such great reviews, was rejected in December of
2011. And we decided to leave it dead. But, on April 18th 2012 I got
a phone call out of the blue from NSF. Two things happened. They found some
money stuffed into someone’s old shoe, or that had fallen behind some couch
cushions. Something like that. Apparently this is not uncommon. Also, they
realized that Katie, despite her NIH grant, was in dire need of funding because
of how FHCRC pays its faculty. These two factors brought the grant-corpse back
to partial life. We were getting funded. Not our whole $1.7 million 4-year
request, mind you, but just over half that. This is a zombie story, after all.
Not a complete resurrection. But we weren’t about to protest at a $900,000
budget for a 3-year project. $450,000 for Katie, and the balance mostly to my
lab but some as a sub-contract to Andrew. And just like zombies are missing
limbs, jaws, etc, our partly revived project was missing some important parts
because we had to slice out nearly half the budget. No more field measurements
of selection (which reviewers were both critical of and especially excited by).
No more GWAS surveys along replicated lake-stream clines (I did that anyway at
a smaller scale with HHMI money; the first of several papers from that is Weber
et al 2017 Evolution). Scaled-back common-garden rearing. Fewer QTL populations.
The funding started July 1 2012 (just after our
usual field season, sadly). That first summer of funding, my soon-to-be
graduate student Brian Lohman spent three weeks scouting out lake-stream pairs
all over Vancouver Island, giving us a set of 16 study sites. In January 2013
we had a small working group meeting at my house in Austin, with Katie and
Andrew, Rowan Barrett, Dieta Hansen, and an incoming postdoc Yoel Stuart, who
was just finishing his PhD with Jonathan Losos at Harvard. Yoel would be the
glue the bound the project together and the engine that made everything happen.
We talked, ate great tacos, drank scotch, read childrens’ books, played Cards
Against Humanity (Yoel won every time), and went for a climbing excursion to
Reimer’s Ranch.
 |
| Andrew lectures Dan on some of the finer points of parallel evolution. |
 |
| Andrew climbing in Dan’s backyard bouldering gym. |
 |
| Andrew and Katie climbing in parallel. Get it? Parallel? |
 |
|
The Austin 2012 meeting ended in a climbing trip at
Reimer’s Ranch
|
Yoel and I assembled a team of field helpers, and planned the field season. As field crew, he’d have PhD student Brian Lohman, K-12 teachers Tania Tasneem and Andrew Doggett, and undergraduates Rebecca Izen and Cole Thompson. Andrew and Katie contributed people as well (PhD student Dieta Hansen, Undergraduates Elena Motivans and Mingsha Zhou), and Rowan Barrett joined in for some field work. From Seattle, we were joined by Katie’s team including Matt Dubin, Susannah Halbrook, and high school teacher Carole Tanner. There was also a team of 12 people from my lab doing a different project (another NSF grant that was also a back-from-the-dead funding situation in 2012). At our peak we had 28 people in the field at once. I was basically just a glorified travel agent that year.
 |
|
The 2013 field crews (missing a few!). Left to right,
approximately: Dan Bolnick, Andrew Hendry, Cole Thompson, Katie Peichel, Dieta Hansen, Yoel Stuart, Matt Dubin, Kelsey Jiang, Connor French, Hollis Woodard, Alicia sp., Brian Lohman, Amy Doan, Rebecca Izen, Chase sp., Racine Rangel, Kim Hendrix (HS teacher), Travis Ingram, Carole Tanner (HS teacher), Susannah Halbrook, Gina Conte, Rowan Barrett, Ruger is the dog. Yes, as in the gun. He handles
security at Roberts Lake Resort. The mountain lions respect him.
|
 |
| Yoel's in a boat, y'all |
 |
| That log isn't totally stable |
 |
| Rebecca Izen processing stickleback in the truck bed to stay dry |
 |
| Andrew tries out Dan's boat. It is fun. |
 |
| Dieta Hansen catching fish |
 |
| First day, training newcomers |
 |
| It rains a lot there |
 |
| Cluexewe Estuary campground is spectacular |
Videos of field work:
- Fancy sound-track compilation from Andrew: https://www.youtube.com/watch?v=cz5dy2Bptas
- Two sound-track-free compilations from the Bolnick lab:
- An aerial view of one of the lakes in this study (turn volume down – the drone is loud):
- A time-lapse of a marine site on Vancouver Island:
- Rose Carlson’s DPIV video of a stream stickleback fast-start escape response:
- A comparison of swimming performance of lake and stream stickleback:
- Stickleback swimming in a stream (by Rose Carlson):
For the
next two years, Yoel and a group of really dedicated undergraduates worked on
collecting detailed data from samples of fish from each of the 36 sites,
including >70,000 SNPs (via ddRAD, with help from Jesse Weber) on 24 fish
per site, 86 morphological characters
(fin shape, body shape, gill rakers, armor and spine traits, eye traits, jaw
biomechanical traits, brain morphology), diet data, parasite data. They
processed mud and water samples for environmental invertebrate abundances.
Data curation
was also a massive undertaking. The resulting dataset spanned site-level data
(eg., elevation, surface area), microhabitat data taken at each trap
(substrate, flow rates), morphology, diet, parasites, and genetics. Thor Veen
joined the team when he contributed an excellent GitHub data structure and
helped with analyses. Analysis itself took a long time. For instance, we
recruited Mark Ravinet to contribute Approximate Bayesian Simulations to
estimate population genetic history parameters (divergence time, migration
rates, colonization order) for all 16 lake-stream pairs. That computationally
intensive process took months.
Finally, we had results, and a
paper. The usual rounds of edits, then we sent it off.
Rejected (Nature). Rejected
(Science). And finally to Nature Ecology
& Evolution, where we went through three rounds of review including a
remarkable >100-day wait for the second reviews after our first revision. It
was finally accepted at Nature E&E, and came out today. Co-authors include two K-12 science
teachers, and multiple undergraduates. Some of the basic science insights are summarized in a nice blog post, by Yoel Stuart, on Nature Ecology and Evolution's website.
A few weeks ago, I made corrections on the proofs to our paper.
I’ve always enjoyed proofs. We set the paper aside for weeks or months for the
journal to process, then get to see our work as if with entirely new eyes. And
I was really proud of this work. It pulls together many ideas, and an
absolutely massive amount of data, to ask some pretty cool questions. The basic
punch-line is, there’s less parallel evolution in stickleback than you thought,
and we can explain why, at least in part. If you want to know more than that,
well… go read the paper (email me [danbolnick <> utexas <.> edu] if you can't access it). Look: it took us 9 years of dealing with zombies to
deliver this to you, don’t tell me you can’t take half an hour to look it over.
To put it in perspective, it is shorter than this blog post.
The project also spun off other papers as well. There’s an
incomplete list at the end of this post (***footnote). Analysis and writing of
the QTL work is ongoing, and there’s a lot more that we are pulling out of the
genomic and phenotypic data. Look for more papers to come.
Looking back
To conclude, I want to reflect on some lessons learned.
First, science is really slow. From idea (early summer 2008)
to paper (late spring 2017), was enough time for my infant daughter to grow big
enough to read The Hobbit and to play Fur Elise on the piano (nicely, at that).
Almost half that time was waiting for money. This bothers me**.
Second, I tend to do a shock-and-awe approach to grant
writing: put in an ambitious (but in my mind genuinely feasible) agenda with
many interlocking parts. That does not always serve me well, and reviewers
wanted us to do something more modest in the end. They told us to cut, and we
cut. This ultimately paid off, not just because we got funded. Remember, our
requests ended up being $1,700,000, but we had to cut $800,000 off that to get
our grant. As a result, we sadly deleted
what we thought was the most exciting component of the study, the field experiments
to measure selection in multiple replicate lake-stream pairs. A missing arm of
the grant-zombie, meant to test whether parallel phenotypic divergence really
reflects parallel selection. Well, zombie arms have a habit of clambering out
of the ground on their own, later on. My great postdoc Yoel Stuart turned that
cut-out-idea into a new grant proposal. As our first grant came to an end, he
got another $1,000,000 funded on his first try (somebody hire the guy!), to do
what we had previously eliminated. Actually, it wasn’t funded exactly on the
first try; this, too, was a back-from-the-dead proposal. That’s ongoing.
Third lesson: hire a great postdoc for your projects. Yoel
Stuart handled a complex web of field, molecular, and morphological data
collection and analysis, and wrote what I think is a genuinely good paper. As a
corollary, hire Yoel for your faculty.
 |
| Dr. Yoel Stuart |
Finally, persistence does pay off. Think about those
reviews: we went from okay-ish reviews with a mix of good to Excellent, to a
really strong set of reviews on our fourth submission. This project ended up
being better, more focused, and probably more feasible, thanks to those
anonymous reviewers (I’m looking at you, Brian Langerhaans) who worked hard to
give us feedback through multiple rounds of submissions. Keep this in mind when
you get that rejection from NSF this spring. Try, try again. “But
often faltering feet Come surest to the goal;”
 |
| Yoel contemplating evolution |
Footnotes:
**
I’ve had several graduate students come up with great
project ideas that we’ve turned into grant proposals. Twice, these ideas have
been funded. But with lag-times like this, there’s almost no way that a grad
student idea can be turned into a full NSF proposal, and from there to funding
and work and papers, within the time-span of a PhD. That has huge policy
implications to how we train students. If we have to support students as RAs
(rather than teaching assistants), then students will typically have to work on
my ideas, conceived years before they entered our graduate program, instead of
their own ideas. That’s a problem, because my experience is that my students
are more innovative than I am. We need rapid response funding for good student
ideas that won’t take 4 years to get funded. NSF DDIG grants are a good attempt
at that, but have such a limited budget that our students’ imaginations must be
kept in check.
***
A selection of other papers from this project:
UT Austin undergrad Newaz Ahmed has a paper just out in
Ecology & Evolution on brain morphology evolution in lake-stream
stickleback. Basically, no parallel
evolution there. http://onlinelibrary.wiley.com/doi/10.1002/ece3.2918/abstract
UT undergrad Rebecca Izen worked on within-stream phenotypic
variation
UT graduate student Kelsey Jiang published several papers on
lake versus stream divergence in swimming behavior in flowing water:
UT graduate student Brian Lohman published papers on
multivariate lake-stream clines:
UT postdoc Jesse Weber published a paper on clinal
lake-stream divergence. This is basically a smaller version of what we had to
cut out for budgeting reasons when our grant was cut in half. We did it with
HHMI funds instead, and at least one more paper is coming from that dataset.
For now, here’s the paper, and a digest about the paper:
Brian also has a paper in review and on BioRXiv about
transcriptomic response to novel environments in lake and stream stickleback:
McGill PhD student Dieta Hansen got out a paper on the
apparent lack of allochrony-driven reproductive isolation between lake and
stream stickleback:
McGill PhD student Krista Oke has a paper on the role of
phenotypic plasticity in lake-stream parallel evolution:
Krista also has a fantastic new paper out about parallel
evolution in fish more generally:
I have a paper on MHC-parasite associations in three
lake-stream pairs:
McGill postdoc Renaud Kaeuffer had a paper that emerged from
our Hendry Winery meeting:
with a follow-up paper by Andrew:
The Winery meeting also led to this review paper:
I have a paper in press in Nature on the lack of
(detectable) divergent selection in lake-stream stickleback
(embargoed)
UT undergrad Cole
Thompson has a revision in re-review at Evolution on evolution of complex
biomechanical traits in the lake-stream pairs.
I’m sure I’ve missed a few papers that have been touched by
this project, and more will be out in the coming year. We have several people
actively working on more analyses and papers. Keep your eyes peeled!
The counterbalance to “this costs too much” ought to be how many good papers are likely to result from the work. If you divide the size of the grant by the number of good papers that came out of it, suddenly it doesn't look so expensive any more! Also important is the number of people that ended up being supported by the grant. Big grants like this create a whole ecosystem around them; if it's a productive ecosystem making important and unique contributions, then it's worth the money.
ReplyDeleteWish I was more confident that your experience is typical, Dan. But I wonder if the experience of another friend of mine is more typical. Getting invited for a full proposal on the first try and getting very good evals but not funded. Revising conscientiously, getting invited for a full proposal again, and not getting funded again, with reviews that suggested undoing some of the changes the first set of reviewers wanted. Trying a third time after further revision, and this time not even getting invited for a full proposal.
ReplyDeleteOk, in truth I'd be reluctant to generalize from any individual's experience with one grant. And your message that "everybody gets rejected, often" is well-taken. I'm just less sure about the "persistence pays" lesson. Presumably, sometimes it pays and sometimes it doesn't, and it seems like it can be hard to judge when it will pay and when it won't.