How to do statistics

This fall, I wrote a series of “How to” blog posts that proved somewhat popular, or at least well-read:

How to write/present science – 2700+ views

How to be a reviewer/editor – 2600+ views

Where to submit your paper – 4000+ views

I hadn't initially planned a series like this, it just kind of emerged. However, I had long planned one particular “How to” post. Ironically, that post was the one I still hadn’t written. Now that it is 2015, the time seems ripe to get back to the original idea. (Thanks to Ben Haller, Gregor Rolshausen, Joost Raeymaekers, and Chuck Fox for critical comments that helped improve this post.)

How to do statistics.

I used to teach statistics. Really! I was a whiz at SPSS and Systat, and I could find my way around JMP. I was almost at the cutting edge, which then was SAS. No one complained seriously about the stats in the papers I submitted. Now, it seems that – with the same statistical skills as before, and maybe even a bit better – I have become a dinosaur. Increasingly, the feeling seems to be that you can’t be considered even moderately competent at statistics unless you can do a GLMM in R. In this sea-change from [insert your previous status package here] to R, I feel that several important points are getting lost – or at least under-emphasized. My goal in the present post is to revisit what statistics are supposed to be for and how you should do them. I do not mean the details of how to choose and run a particular model but rather how to view stats as a way of enhancing your science and refining your inference. I will outline these ideas through a series of assertions.

1. It’s all about the (appropriate) replication

An incredibly important route to improving your science is to maximize replication at the appropriate level of inference. Imagine you are interested in a particular effect, say the difference in an experiment between two treatments or the difference in some trait between populations in two environments. You need to here strive for maximum replication of the two treatments or the two environments. This might seem obvious but – as a reviewer/editor – I have seen many studies where people wish to make inferences about the effects of two environments, yet they have studied only one population in each environment. In such cases, they are entitled to draw conclusions about differences between the two studied populations but not between the two environments because – with only one population per environment – the investigator cannot gauge the difference between environments in relation to variation within environments. That is, it is quite possible that two populations within each environment would differ just as much as two populations sampled from the different environments. While the temptation is to get larger sample sizes for each measured population, what is much more important is to sample many populations. I have seen many papers rejected for lack of replication at the level for which inferences are desired.

2. The data are real – statistics are merely a way of placing a statement of confidence in an inference you draw from the data.

I have frequently seen students paralyzed by their inability to fit an appropriate error distribution in R. They spend weeks and weeks trying various options only to eventually give up and throw out the offending data. The opinion seems to be that, “if I can’t fully satisfy the requirements of a statistical test, then the data must be bad and I shouldn’t report them.” This is folly! The data are the real thing – the stats are just a tool to aid interpretation. What is infinitely better in cases where a perfect model cannot be fit is to present the data, analyze them the best possible way, and then own up to cases where the data do not fully satisfy the assumptions. The truth is that many statistical tests are extremely robust to small-to-modest violations of their assumptions as long as the P value (but see below) is not too close to the critical value.

Of course, I am not here advocating using a bad model when a better one exists. If a better model exists, by all means you should use it. However, this more practical point is already emphasized quite frequently nowadays to the point that it can become detrimental to a student’s progress, and I am here trying to push the pendulum back a bit. That is, finding the ideal model is valuable and helpful, but slavish dedication to this goal can sometimes detract from the quality of scientific education and insight. Of course, the most important thing is to have a good question and experimental design before you conduct the study, which will simultaneously improve the science and help avoid later statistical constraints.

3. It's not about the P value.

Although opinions are changing, many students are still fixated on obtaining a P value smaller than the critical level of 0.05. This goal is misguided – for three reasons. First, 0.05 is totally arbitrary. If you are focused on P values, what is much more useful is the actual P value – is it small or large? (Journals should always require actual P values in all cases.) Second, any particular set of data can be analyzed multiple ways and cycling through those options can lead to the temptation to choose the one that generates the smallest P value. Third, P values themselves (the probably that, if the null hypothesis is true and you reject it, you will be wrong in doing so) are a silly way to do science – sorry RA Fisher. Among the many reasons, the null hypothesis is – in traditional frequentist statistics – treated as a default rather than as an alternative model, and thus one often rejects the alternative hypothesis even when it has more support than the null hypothesis.

Instead of null hypotheses, it is much better to specify alternative hypotheses that are competed against each other with alternative statistical models to thereby judge the relative support for each hypothesis. Such comparisons can take the form of likelihood ratio tests, Bayesian credibility intervals, AIC comparisons, or the like. One might argue that a level of arbitrariness creeps in here (because a standard yes-no threshold is sometimes lacking) but the truth is that such approaches are much less arbitrary because they quantitatively compare the level of support for competing hypotheses. The author can then draw whatever conclusions he/she wants from the levels of support, while still allowing the reader to draw some other conclusion from the same model comparisons should they wish to do so.

4. Effect sizes are what matter.

P values are determined by an interaction between effect size (strength of an effect) and sample size. Thus, P values are NOT the strength of an effect. As a result, one cannot – without other information – say that a P value of 0.0001 represents a stronger effect than a P value of 0.05. It might simply be that the former analysis has a much larger sample size. Take simulation models as a particularly obvious example. In this case, one can have whatever sample size one wants given computing power and time. Thus, the exact same effect size (determined by the parameters of the simulation) can have totally different P values determined by the number of replicate simulations performed. If you have a tiny (but real) effect size, simply run more simulations and it will eventually become significant! The same logic applies to experiments and surveys. What matters are effect sizes based on how much variance in the data is explained, or based on the difference between group means weighted by the variance or the mean. Examples include R², Cohen’s D, and Eta.squared.

Of course, one still wants to place a statement of confidence in assertions about a given effect size, which is where one adds P values or – better yet –model comparisons as discussed above. Note that, when true effect sizes are small, they tend to be overestimated when sample sizes are also small, which as generates the so-called funnel plot of meta-analyses. Thus, one still wants as large a sample size as possible and one would ideally correct the measured effect size for an estimate of the error – either using Bayesian approaches or through brute force. That is, a measured R² can be adjusted by the R² expected if no effect was present – with an example here.

Effect sizes (here estimates of the strength of selection) are higher when sample sizes are smaller. From Kingsolver et al. (2001 - American Naturalist).

5. Graph your data

In many meetings with students where I am to see the outcome of their experiment or sampling for the first time, I am presented with detailed statistical tables where the student emphasizes whether or not particular effects are significant in this or that model. I find myself incapable of interpreting these results without seeing the data in graphical format. In fact, I think a student should first graph the data in a manner that addresses the original question before running ANY formal statistical tests. This aids not only the assessment of assumptions for subsequent statistical tests (hugely influential outlier errors sometimes pop up when I ask a student to do this) but also reveals – at a first glance – the gestalt effect size assessment that rarely ever changes much as time goes on, notwithstanding any ups and downs that occur in the subsequent formal statistics. In this way, the student and supervisor can have a rough picture of what the experiment has revealed before having to worry about the statistical details. I would bet that 90% of the important work (if not the time investment) is done once you graph your data in a way that informs the original hypothesis/question.

All data sets have the same means, variances, correlations, and regression lines. Only graphing shows how different they really are: Anscombe's quartet from Wikipedia.

Some additional notes about statistical packages.

6. R is simply one of many useful platforms for drawing statistic inference.

Nowadays, students feel incompetent if they don’t analyze their data in R – hell, I even feel that way sometimes. However, R is simply a post-experiment tool – a hammer with which you help massage your data into optimal inference. SPSS, Systat, JMP, and SAS are also hammers – they too can massage your data. Perhaps R is a titanium hammer, better and more efficient at massaging the truth from data; but think of all the amazing inferences that were derived before R was popular. Does the failure of these countless previous studies to use R mean that we should not believe everything published before (and much published after) 2002? (Of course, re-analysis does change the conclusions of some previously-published and superficially-analyzed studies.) Does the fact that something else will eventually replace R mean that our current inferences with R then become incorrect? Nonsense. Valid and excellent inference can be obtained with any number of statistical packages.

Given that R is now the most common statistical program it does make sense for new (and old) researchers to start with (or switch to) R. However, the main advantage is not – in my opinion – dramatically improved inference but rather ease of communication with other scientists, such as through the sharing of code. Moreover, R has many other components not present in canned packages, such as data exploration tools, connection to database and file system structures on your computer, if-else statements, while loops and other programming tools, detailed plotting functions, connection to other programming languages such as C++ and Python just to name a few. It also contains user-motivated novel statistical tools for specific applications that are simply not available in other packages.

How to program a Christmas tree in R.
http://simplystatistics.org/2012/12/24/make-a-christmas-tree-in-r-with-random-ornamentspresents/

In reality, however, most scientists seek much simpler assistance from statistical analysis, for which other packages can do the trick. Moreover, efforts to master R can take so much time and dedication that students sometimes neglect what is really important in science: good and novel ideas, good experimental design, diligent execution with high replication and large sample sizes, effective visual presentation of information, and common sense deduction. I would much rather have a student who mastered these skills and analyzed their data in SPSS than I would have a student who was an R whiz but neglected the key skills of scientific investigation. Of course, what I really want is student who can do both, but the former is vastly more important. (Of course, most students who do learn R certainly don’t regret it afterward.)

7. R has its own foibles.

Any statistical program has bugs or flaws, and R is no different. Many issues with existing packages have been pointed out well after those packages were used in published studies. The simple fact is that R is modified by many people and can (like other statistical packages) suffer from the inadvertent introduction of errors that it takes time for others to discover and the originators to correct. Moreover, R has its own set of defaults that can be confusing or misleading. For instance, the standard default in R is Type I sums of squares (SS), whereas the default in many other stats packages in Type III SS. These different SS options have their own sets of positives and negatives and supporters and detractors. However, one must understand the differences between them. Of critical importance, Type I SS fits the first term of the model first before fitting other terms, whereas Type III SS fits all of the terms simultaneously. As a result – and as my students found out – you can get very different results if you run the same analysis in R and some other package, as well as if you change the order of entry of the terms in the model in R. (For my money Type III SS is usually more appropriate and my students now usually specify this option in R.)

It is important to make clear that I am not suggesting that students forsake the use of R for some other package. In most cases, they should probably use R. What I am instead saying is that learning R is not the most important (although it could be the most useful) thing you do in your education. Do not think that R = science and that, if you don’t learn R you are not a good scientist. Instead, think of R as a titanium hammer. If you need that hammer, then use it. If you don’t yet have any hammer, you might as well go titanium if you have the time. However, remember not to equate knowledge of R with intelligence or with a good study or with your own sense of self worth. Learn R for the right reasons and don’t let it become your raison d’etre – unless you wish to specialize in statistical analyses. Indeed, statistics and the development of R packages is certainly a branch of science in its own right - but my focus in the present post is empirical biologists who do not have a special interest in developing statistical methods.

Coda

There are some basic thoughts about statistics that are sometimes lost or forgotten in this brave new world of R-based statistics. The truth is, I am not a statistics expert by any stretch of the imagination, and so I have concentrated my comments on more basic, perhaps even philosophical, points. However, so much training is now provided in the mechanics of statistics, and R, that I think it is these more basic points that you are more in danger of forgetting or foregoing. Having said all this, it is perhaps time for #SPSSHero to also become #RHero, instead of relying on my lab members to do all the heavy lifting while I simply sit around and complain about it.

--------------------------------

Links added later:

https://scientistseessquirrel.wordpress.com/2015/02/09/in-defence-of-the-p-value/

Conference: Speciation 2015! Register soon!

Hi all.  A quick announcement regarding an upcoming conference:

Who: You – or your grad students!
What: Speciation 2015
When: March 15–20, 2015
Where: Ventura, CA
Why: Modes of Diversification, Ecological Mechanisms, and Genomic Signatures
How: Register at http://www.grc.org/programs.aspx?id=16902

  There’s lots of additional information at the link above.  This is the world’s largest conference on speciation research, and they have put together a truly stellar list of invited speakers.  I wish I could go; so you should go, so that I can live vicariously through you.

  The conference is close to full, so get your registration in soon!

Best evolution/nature books

This is the time of year for lists: the best this, the most important that, and so on. Just such a list was circulating today on twitter about the best science books of the year, presumably also books you should be buying for Christmas presents. This discussion got me to thinking – what would be the best evolution books for Christmas? I don’t just mean published this year, I mean published in any year but still available. I also don’t mean technical books but rather popular books that you might give your friends or spouse or yourself. The kind of book you will read in bed at night rather than in your office in the morning. Books that are easy to read and a great advertisement for how cool evolution is and can be – either remotely or on the front lines of research. Meshing nicely with this idea was the realization that I have, over the years, benefited greatly from Christmas books about evolution – mainly from my Mom but also from students, friends, colleagues and other family members. With all of this in mind, here is my subjective and incomplete listing of the best evolution books for Christmas, or any time. (This post and list is updated nearly every year.)

1. Neanderthal Man by Svante Pääbo

I have only ever read two popular science/nature books twice - and this is one of them. Probably the best book ever by a scientist about his/her science and life. (Other good ones are Naturalist by E.O. Wilson and Privileged Hands by Geerat Vermeij). It combines fundamental insights into our evolutionary origins (interbreeding with Neaderthals and other archaic humans), a compelling narrative of a career (development of methods for studying "ancient DNA"), interactions with technological developments (e.g., Sanger, 454, and Illumina sequencing), personal life (Pääbo is the son of the secret second family of a Nobel Prize winner, whose discovery would later save Pääbo's life), and scientific administration (founding of a new Max Plank Institute). Written in an extremely frank and down-to-earth way, it makes the reader feel a part of - or at least an intimate observer - of the discoveries and controversies. I look forward to reading it again.

2. The Beak of the Finch by Jonathan Weiner.

This book, a Christmas present from my Mom in 1995, quite literally changed my life. It is a wonderfully written and engaging account of the importance of rapid evolution told mainly through the research on Darwin’s finches conducted by Peter and Rosemary Grant. Reading this book that fateful Christmas, I had an epiphany “you can actually watch evolution happen in real time” that almost instantly changed my career goals from wanting to study salmon to wanting to study evolution. To this day, my research focuses squarely on “rapid evolution” – I now even work on Darwin’s finches in Galapagos, as well as on other systems – most notably Trinidadian guppies – described in the book. This book is, for me, the best example of how important and dynamic evolution is and how exciting it can be to study. And it isn’t just me – the book won a Pulitzer Prize, as well as many other accolades. If you read only one book about evolution, if you give only one book about evolution, this should be it.

3. Nature’s Nether Regions by Menno Schilthuizen

This book was also a gift from my Mom, although not for Christmas. She had read it and found it so much fun that she had to buy it for me – in hard cover no less (I always wait for paperback and, even then, for books to go on sale). The book describes, in extremely entertaining prose, the evolution of gentalia in animals, including humans. It is jam-packed with fascinating and extremely well researched examples of how evolution is shaped by penis-driven internal courtship, as well as other titillating (but not superficially) topics. It is the sort of book that you can’t help but read out quotes to your spouse or students or at parties. The sort of book where you can’t help but laugh or exclaim out loud, or think “Wow, I had no idea the clitoris was that big” or that “As familiar as the old in and out is, there is nothing about the biology of reproduction that requires rhythmic penis movements.” (This last is an attempt to paraphrase from memory the infinitely more clever original sentence.) The book also proves that the best writers in English don’t have to be native English speakers.

4. The Voyage of the Beagle by Charles Darwin

Perhaps you weren’t expecting this one as it was published 175 years ago – yet it remains a compelling read. It has all of the innocence and excited of a pre-worried Darwin as he voyages around the world discovering new things. His excitement is contagious and much of the book reads as might a contemporary account of discovery, albeit set in an endearing (to the outside reader) context of the mid-1800s. I have given this book to a number of non-scientists and they have found it enjoyable. It is also worth saying that it is one of the best of the accounts of discovery from the golden age of natural history, among other great accounts I have read by Wallace and Bates and others. And, of course, one can’t help be chuffed by catching the early hints of Darwin’s emerging idea of evolution by natural selection.

5. The Song of the Dodo by David Quammen

Here is another book that, like the Beak of the Finch, has been around for quite a while (although not as long as the preceding book) and is critically acclaimed. It is a deft mash-up of island biogeography, conservation, invasion biology, and evolution. It describes in great detail and with personal stories and anecdotes from actual researchers on the ground how the geographic distribution of the world has shaped the distribution and evolution of animals and how this context has then shaped extinction risk through direct or indirect (e.g., invasive species) human influences. My only complaint is that the text is a bit bloated and, thus, sometimes tedious. I can’t help but think that cutting 20% of the book would have made it much more engaging.

6. Improbable Destinies by Jonathan Losos 

It is sometimes hard to read a book about a topic you work on that features many of the people you know. It is difficult to look beyond the sweeping statements you know to be too simplified and the sanitized versions of personalities. Indeed, this difficulty is why I never re-read The Beak of the Finch - I don't want my original naive enjoyment to be tempered by intimate experience with reality. Improbable Destinies is about the extent to which evolution repeats itself when different organisms encounter similar conditions - so-called "convergent" or "parallel" evolution. Set up as an exploration of Gould's famous "replay the tape of life" thought experiment, this book describes a diversity of research where those thought experiments become reality - in both the lab and the wild. In the end, I found this book to be an exceptional (if imperfect) digestible representation of these topics and controversies, as well as well being written and engaging. I have even recommended it to my own students working on this topic. I asked one of them recently: "So, are you finding it useful?" - to which she replied "Yes, but it is also really well written!" I agree - although I would replace "but" with "and".

7. Dark Banquet by Bill Schutt

Another Christmas present from my Mom, this relatively recent book describes the curious lives of blood feeders: ticks, chiggers, vampire bats, leeches, mosquitoes, and so on. This is another book that generates tons of anecdotes and comments that you just can’t help but share with others (George Washington’s death might well have been accelerated by doctors leeching too much blood). I even use it for teaching – my favorite way to discuss the kidney now is to talk about the difficult challenges and ingenious evolutionary solutions that vampire bats use to deal with their massive blood meals. On the negative side, it makes you itchy at night when reading about bed bugs. A great read though.

8. Dr. Tatiana’s Sex Advice to All Creation by Olivia Judson

This book is like Nature’s Nether Regions in that it takes on the titillating subject of sex and sexual selection. However, the main appeal is its clever gimmick. It is written as though the author is a sex advice columnist to various organisms who write in to describe their problems – sort of a Dr. Ruth or Dan Savage to the animal world. A fun read and it has been adapted into a TV show.

9. The Curse of the Labrador Duck by Glen Chilton

This book is a strange and whimsical choice but it is really fun to read. It describes the quest of the author – charged with writing a species account of the extinct Labrador Duck – to see, measure, and consider every Labrador Duck specimen in the world, something around 55 or so. He traipses around the world finding and measuring each specimen (often with considerable difficulty) and injects his own fun observations and perspectives on the places and people he visits. The numerous cute phrases are a bit much sometimes (and it isn’t really a book about evolution) but it is hard to find a book that feels more like spontaneous fun with natural history.  By coincidence, the duck on the cover is right outside my office door – but this has nothing to do with my affection for the book.

10. How to Tame a Fox by Lee Dugatkin and Lyudmila Trut

This is the second popular science book that I have read twice. The reason this time, however, was partly because I was confused about the way in the science was described. On the first reading, it had been so frustrating that I wasn't sure I liked the book. On the second reading, the description of the science was still a bit confusing but I was able to isolate those parts and instead enjoy the tale of intrigue, persecution, mystery, conflict, perseverance, excitement and success surrounding the nearly half-century long (and still running) Soviet/Russian experiment on domestication. Most importantly, this book gives a first hand (Lyudmila Trut ran the experiment nearly the whole time) look at extremely original and forward-thinking science behind the Iron Curtain.

11. Various books about strange critters

I often read species-centric books, such as Moa, Platypus, Tasmanian Tiger, and A Fish Caught in Time. These books have a singular focus on a particular taxon that allows the author to create a much more coherent narrative than more general books about general phenomena, such as sexual selection or blood feeding (although the above books do a great job regardless). Among these books, I recall with particular affection the Platypus book, by Ann Moyal, which is a fun account of how western science was long befuddled with the evolutionary position and significance of the playtypus. It describes the various well-known (Richard Owen) and obscure personalities who weighed in on the topic and how the discoveries changed our view of the evolution of vertebrates. A Fish Caught in Time, by Samantha Weinberg, is also outstanding.

Many other great books are out there but I don’t have time to describe them all – my apologies to many other great authors, with particular recognition of Carl Zimmer (Parasite Rex) and Rebecca Stott (Darwin and the Barnacle). I welcome further book suggestions by way of comments on this post. I would also like to apologize for any mistakes or bad grammar in the preceding. I only had the idea for this post an hour ago and it seemed appropriate to do NOW (enough time to buy those books) and I want to go play with my kids. So, until next time, clever spending and good reading. Happy holidays. 

Retracing the legacy of guppy introductions past: local differentiation maintained despite high and rapid gene flow

[ This post is by Sarah W. Fitzpatrick; I am just putting it up.  –B. ]

When populations adapted to different environments come into contact through range expansions, invasions, or by human-assisted migration, the outcome is often unknown. How will immigrant individuals fare in the new environment and will they hybridize with native populations? If so, what impact does gene flow with non-native individuals have on the local populations? The question of whether gene flow between adaptively divergent populations promotes or constrains local adaptation is a long-standing puzzle in evolutionary biology and is increasingly relevant for designing effective conservation strategies.

In our recent study (Fitzpatrick et al., 2015) published in Ecology Letters, we turned to Trinidadian guppies, a model system for studying evolution in the wild, to ask questions about how gene flow affects fitness and local adaptation. The guppy system has proved powerful for understanding a diverse set of questions in ecology and evolution, but these rapidly evolving freshwater fish are perhaps most famous for the series of transplant experiments conducted in the streams of Trinidad. Caryl Haskins initiated the first introduction experiment in 1957, moving guppies from an environment where they experienced high mortality from predation to a site in a separate drainage where waterfall barriers limited the upstream colonization of most fish, including predators and native guppies. The introduction site thus represents a release from predation. John Endler and David Reznick and his colleagues have since repeated this transplant scenario in multiple independent drainages, and their studies of rapid adaptation and eco-evo feedbacks focusing on the introduced populations are iconic in evolutionary ecology.

In each of these introduction streams, native guppies existed downstream from introduction sites, and although the introduced populations were initially maladapted to their new environment, gene flow is expected because adaptive divergence has not led to reproductive isolation in guppies (Crispo et al., 2006). In fact, we expected much higher levels of downstream gene flow from introduction sites than what is observed in natural streams due to the fast life history of the introduced fish and female preference for novel males (Fig. 1; click to see at larger size).

Figure 1. Conceptual diagram illustrating the expected differences in amount of gene flow between natural streams and streams with introduced populations. In both hypothetical streams, predation level is colour-coded based on the species listed in the bottom key, and increases in the downstream direction. Black rectangles indicate waterfall barriers that limit upstream fish dispersal. The colour of fish indicates traits matched to a certain level of predation (e.g., the blue fish has traits that are adaptive in a low-predation environment). In the hypothetical natural stream, fish are perfectly matched to their level of predation and gene flow among populations is low, based on biological factors listed in the grey box. In the hypothetical introduction stream, guppies from high-predation (HP) environments were translocated upstream of naturally occurring low-predation (LP) populations. Gene flow is expected to increase relative to natural levels for the reasons listed in the grey box, and the effect of elevated gene flow on locally adapted traits remains unknown (indicated by grey fish and question marks).

Our team, from Colorado State University, asked what the effects of elevated levels of gene flow from an initially phenotypically divergent population would be on locally adapted phenotypes in downstream populations. Following the footsteps of Haskins, Endler, and Reznick, we sampled guppies from six historic introduction sites, from each of their source populations, and from multiple sites downstream from each introduction. We genotyped all individuals at ten microsatellite loci and quantified a suite of known fitness-related traits, such as male color, body shape, and some female life-history traits that tend to differ based on the level of predation experienced. We also included individuals from two native populations prior to the onset of gene flow, providing a powerful opportunity to compare pre- versus post-gene flow phenotypes and allele frequencies. At each site we classified the predator community as either low, medium, or high based on the complexity of the fish community observed.

Clockwise from back left: Jed Smith (undergrad researcher, CSU); Chris Funk (asst. prof, CSU); Sarah Fitzpatrick (PhD student, CSU); Lisa Angeloni (asst. prof, CSU).

We found that the genetic signature of introduced guppies swept throughout all downstream distances, indicating high levels of gene flow downstream from all introduction sites on a rapid timescale (Fig. 3a; click to see at larger size). However, despite genetic uniformity caused by introductions, guppies sampled from different predation communities along the streams maintained phenotypic traits that best allowed them to survive and reproduce, given the local predator community. In other words, genetic homogenization did not cause the loss of locally adapted phenotypes.

Figure 3. (a) Comparison of genetic differentiation (pairwise-FST) among all sites in natural streams vs. among all sites in streams after introductions took place. (b) Within-stream STRUCTURE plots and average pairwise-FST values for all six streams that experienced an upstream introduction. Each line in the plots corresponds to an individual with colours representing the proportion of an individual’s genotype assigned to a given genetic cluster. Old introductions show fine-scale genetic structure despite low genetic divergence (low FST). All sites from the three recent introductions conducted in the Guanapo drainage were included in the same analysis because they share the 5000 m and source sites. These recent introductions are more genetically homogeneous, with the exception of pre-introduction 0 m sites in Taylor and Caigual (shaded in blue) that are very distinct and genetically divergent (high FST) from the rest of the sites. Colored circles on the x axes indicate the predation level at each site: blue = low, green = mid, red = high, as defined in Fig. 1. All plots represent the (k) number of genetic clusters with the highest support (see Appendix S1).

We used the exchangeability analysis described in Hendry et al. 2013 to compare neutral genetic and phenotypic exchangeability among the common source site and two native low-predation sites that were sampled before and approximately 12 generations after gene flow. A major take-home message from our study can be gained from the results of this analysis. Namely, we found phenotypic divergence associated with the local predation regime despite neutral genetic homogeneity with the source of the introductions (Fig. 5; click to see at larger size).

Figure 5. Ordination plots and group classification based on discriminant analysis of principal components (DAPC) for neutral genetic loci (a) and phenotypic traits (b). Colours correspond to a priori groups based on population origin: native low-predation in purple, the same sites post-introduction in blue, and introduction source in red. Bar graphs below the dashed line show the mean (and 95% CIs) proportion of individuals from each population classified into each population. Each bar represents the classification of the population on the x axis, as labelled for one set of bars in (b). The bottom-left insets show eigenvalues of the four principal components in relative magnitude.

Phenotypes were measured from wild fish, so we are unable to separate the relative roles that phenotypic plasticity and adaptive evolution play in causing the observed phenotypic divergence, but we argue that both processes are likely involved. Ongoing work that includes common garden assays conducted before and several generations after the onset of gene flow, and wild pedigree reconstruction throughout the initial pulse and longer-term wave of gene flow, will add to our understanding of the mechanisms by which gene flow impacts adaptive evolution and population growth.

Although we expected to find high gene flow from the introduced populations, we were surprised by the near-extinction of native alleles, especially downstream from the set of recent introductions. However, differential rates of introgression across the genome may result in homogenizing effects of gene flow at neutral loci, while locally adapted native loci or genomic regions are maintained by selection.

Whether the loss of native genetic signatures at neutral loci represents a true detriment is an intriguing philosophical debate. The costs of gene flow between populations of the same species may be outweighed by the benefits in cases where selection for a local ecotype is strong, or where recipient populations are inbred. This study reveals how model systems that can be manipulated in the wild have the potential to inform smart management decisions for threatened populations.

Indeed, we have pioneers of model systems like Haskins, Endler, and Reznick to thank for building foundations that continue to generate creative questions and provide a deeper understanding of nature.

References

Crispo, E., Bentzen, P., Reznick, D.N., Kinnison, M.T., and Hendry, A.P. (2006). The relative influence of natural selection and geography on gene flow in guppies. Mol. Ecol. 15, 49–62.

Fitzpatrick, S.W., Gerberich, J.C., Kronenberger, J.A., Angeloni, L.M., and Funk, W.C. (2015). Locally adapted traits maintained in the face of high gene flow. Ecol. Lett.

Hendry, A.P., Kaeuffer, R., Crispo, E., Peichel, C.L., and Bolnick, D.I. (2013). Evolutionary inferences from the analysis of exchangeability. Evolution 67, 3429–3441.

Figures are cited based on the numbering in the manuscript.

Where to submit your paper – response to reviews.

My blog posted in open access Where to submit your paper. Or “If at first you don’t succeed, fail fail again … then try open access” has been viewed 2771 times in 5 days and has now been subject to some post-publication reviews. It seems appropriate to do a quick follow up in which I respond to those comments.

The original post made several points:

1. If you have some great work, submitting to high-impact journals like Science/Nature is fine – even though the chances of acceptance are low. The whole process usually makes for a better paper.
2. Traditional society-based journals are great outlets: they are well respected in the community and are frequently scanned for papers by many scientists. If you publish there, folks know you have received a rigorous and critical evaluation of your science from the perspective of its rigor AND importance.
3. Rejection is an ever-constant companion for ALL scientists trying the above routes and it is good idea to find a coping mechanism that works for you.
4. Open access journals are not good options (except as a last restort) because they don’t have the above properties.

Of the numerous comments on twitter and in personal emails, only item 4 seems to have - perhaps predictably - raised hackles, specifically in relation to citation rates. So let’s look at some of those comments and see what new insights they can bring.  

@JacquelynGill @EcoEvoEvoEco I don't see what this has to do w OA. It is about journal quality, which isn't necessarily correlated with OA.
— ElenaBennett (@ElenaBennett) December 3, 2014

This statement is totally correct. Most of us think of PLoS ONE as the archetypal open access journal, which is why I didn't think to initially draw the distinction with "better" open access journals. I therefore added the follow comment after the post.

To make sure my opinion is clear, I am FOR open access PAPERS (in whatever journal) and even for open access journals as long as they are selective (e.g., PLoS Biology, Evolutionary Applications). What I am not for is for-profit open access journals where you pay your way to publish pretty much whatever you want. Those are merely profit making machines for publishing houses - they only make money when they publish your paper. PLoS ONE is non-profit but the problem there is that it is (rightly) viewed as a dumping ground for papers that people couldn't get published elsewhere. Thus, it is not good for exposure of your paper, for the influence of your paper (citations), or for your career. It should be a last resort when you are in a hurry or you (or your student) are sick of trying other places. Indeed, that is how many people already view it and you should too or people will think that was the case for your paper even if you submitted there first.

By far the best option as far as everyone (except for many publishers) is concerned is to publish in a respected traditional (often society based) journal and pay for the paper to be open access. Everyone wins. 

A number of other comments were along the lines that I had N = 2 for my PLoS ONE papers, which thus doesn't say much beyond my own limited personal experience. This is also entirely true given that the post was from my personal perspective and speaks only to my own experience.

@ucfagls @Protohedgehog @alisonatkin Yes, data speak to only 2 papers but, for me, those 2 are the ones that matter. Doesn't encourage more.
— Andrew Hendry (@EcoEvoEvoEco) November 30, 2014

But, wait, it turns out I have more experience than I thought (it seems I even ignore my own PLoS ONE papers). While reviewing another manuscript today, I remembered that I had another PLoS ONE paper - this one published in 2010. Yeeha: N = 3. I figured I better update my stats accordingly. I was a bit worried this time as I quite like the paper and know it has been cited at least a few times, so I was thinking that my new data point would mess up my story. Nope. Same thing - the worst cited of all my papers in that year.

Updated stats (adding 2010) for citations (Web of Science) to my PLoS ONE papers in comparison to citations to all my other papers in those years.

So, at this point, I can say with confidence that PLoS ONE is not working for me. In fact, it seems that I wasted three awesome papers by publishing them in an outlet where no one sees them or wants to cite them. I did receive a lot of comments from other people about their PLoS ONE papers also being very poorly cited (although at least one person said they had not noticed a difference). Then I was pointed (by a PLoS ONE senior editor) to something more quantitative.

@JacquelynGill @ElenaBennett @EcoEvoEvoEco This study gave different picture of cites in PLOS ONE vs ecology journals http://t.co/O4lJVFUSC4
— Matt Hodgkinson (@mattjhodgkinson) December 3, 2014

The paper referred to here collected data on citations to 30 empirical ecological papers (selected in a "stratified" manner) published in 2009 in PLoS ONE, in some traditional ecology journals (Ecology, Oikos, Functional Ecology), in Ecology Letters, and in the big boys (Nature, Science). The results were that citations to ecology papers in PLoS ONE were roughly equivalent to those in Ecology and Functional Ecology and higher than those in Oikos. However, the citation rates in PLoS ONE were much lower than in Ecology Letters and the big boys.

From Wardle (2012 - Ideas In Ecology and Evolution)

These results run counter to my own experience and seem to suggest that PLoS ONE is a good target journal. The results were so different from my own that I decided to take a quick closer look at the question. For each of the three years in which I published a PLoS ONE paper, I used Web Of Science to search for all papers in PLoS ONE and Ecology (the sort of target journal I suggest shooting for) that had the word "ecology" as a topic (an objective way of comparing ecology papers from the two journals). I then used "citation report" to calculate all citations to those papers and, from that, calculated the mean citation rate (total citations divided by total papers).

Citations to "ecology" papers in PLoS ONE and in Ecology in each of three years.

 Of course, this analysis is still crude: I didn't actually read the papers, I didn't examine other journals, and I didn't examine more years. Nevertheless, these larger sample sizes than in Wardle (2012) seem more in line with my argument that citations are lower in PLoS ONE than in the canonical society-based journal. My papers, which are more evolution than ecology, are cited below these PLoS ONE rates. I wanted to do a similar analysis for PLoS ONE versus my own target journal Evolution but the "topic" "evolution" pops up too many other things in PLoS ONE that are not organismal evolution and I was too lazy to sort through them all. My suspicion (and that is all it is) is that the difference will be more severe in evolution than in ecology, where PLoS ONE is perhaps more accepted as a reasonable outlet. It would be awesome if someone did a proper analysis - but not my students - they need to be working on papers and not blogging.

Of course, none of this stuff is definitive in any way but these post publication reviews of the original blog have lead to revisions that bolster my original findings and thus strengthen my general conclusions. I hope that my blog is now acceptable for publication in your journal, whether open access or otherwise.

-------------------------------------------

More just for fun:

Letter from the frustrated article of a journal paper

Get me off your fucking email list

Carnival of Evolution #77: The Carnival is Dead, Long Live the Carnival

Carnival of Evolution #77 is now up at… the Carnival of Evolution blog.

Apparently, Eco-Evo-Evo-Eco had the unanticipated honor of hosting the very last independently hosted CoE, Carnival of Evolution #76, on 2 October 2014.  I guess Felipe P. J. did such an amazing job hosting that nobody wanted to try to follow his act; Bjørn was unable to find a host for November.  So Bjørn will now be hosting the CoE locally, on the CoE blog itself; he has apparently gotten tired of pleading for hosts.  It’s a bit sad, but not really unexpected; blog carnivals used to be more of a thing, but apparently kids nowadays have moved on to Twitter and such.  Case in point: you can follow Andrew at @EcoEvoEvoEco (but he has not yet announced his Snapchat handle).  CoE actually lasted much longer than most blog carnivals did.

But since Bjørn has apparently decided to keep it limping along on his own, it’s not actually dead yet (just pining for the fjords), and we’ll keep participating in it for as long as Bjørn keeps running it.  In #77, we have three posts; I’m not sure how that happened, since I always nominate just one, but there they are.  :->  First off is Andrew writing about how to choose a study system; second is Erik Svensson responding to the kerfuffle about the proposed Extended Evolutionary Synthesis; and third is Yoel Stuart writing about his fascinating work on character displacement in Anolis. I’m glad all three ended up in the Carnival; they’re all great posts!  Check out Carnival of Evolution #77 for lots more cool stuff.

Bjørn is referring to the new, local-hosted carnival as the “Phoenix Edition”, so in honor of his ongoing commitment to the Carnival of Evolution, here is a phoenix.  Thanks Bjørn.

Art credit: Enigma.

Where to submit your paper. Or “If at first you don’t succeed, fail fail again … then try open access”

The confluence of two experiences motivated this post. First, I was involved in a conversation on Twitter (below) that was reacting to suggestions (in a commentary in Nature) that the high volume of open-access papers was the cause of the reviewer fatigue that so often bedevils journals and editors (such as myself). At one point in this thread, someone pointed to a blog post titled “Why I Published in PLoS ONE. And Why I Probably Won’t Again for Awhile.” The main point of that post was to contrast the desire of young scientists to better the world by publishing in open-access journals with the perception that senior scientists don’t view a paper published in open-access journals as equivalent to a paper published in a more traditional journal. This latter sentiment was similar to my own experience on search committees in which candidates would be considered less impressive if they published too much in open-access journals.

.@BrunaLab @EcoEvoEvoEco @PLOS @NatureNews Ummm...in 2013 PLOS One published over 31K articles. More than Elsevier's entire output?
— Adam P. Summers (@Fishguy_FHL) November 27, 2014

@Fishguy_FHL @BrunaLab @EcoEvoEvoEco@PLOS @NatureNews Elsevier published 250,000 in 2013 http://t.co/78RYgVoB22
— Trevor A. Branch (@TrevorABranch) November 27, 2014

@Fishguy_FHL @BrunaLab @PLOS @NatureNews #OA jour. (note: many more than just PLOS) decrease reviews b/c they don't reject = less re-review
— Andrew Hendry (@EcoEvoEvoEco) November 27, 2014

The second motivation came from our weekly lab meetings. Near the start of each meeting, we go around the room asking “Who had a paper or proposal accepted or published this week?” And then, after a hopefully long discussion, we ask, “Who had a paper or proposal rejected this week?” I kind of like this two-part question because it enables us to get excited about our successes while also making the failures seem more acceptable. (“Oh, it happened to her too, so my own rejection is OK.”) And we can also complain about reviewers and can discuss how we will make our papers better in response. It just so happens that, over the past few months, no one has been able to speak up for the first question and pretty much everyone has spoken up for the second. One lab member even noted that rejection seemed to be a recent trend in the lab.

These two experiences led me to consider the question: “Should you – as a young scientist – take the easy route and publish in open-access journals, or the hard route (likely entailing multiple rejections) of trying more traditional journals, either the big boys or the classic society-based journals?” First, let’s consider the benefits of open access. The basic idea is, of course, that everyone will see the paper and you won’t waste your time cycling through journals that don’t think your paper is “important enough.” Moreover, citation rates are pretty decent for open-access journals, right? At least, that’s what everyone says. I would like to put this presumption to the test based on my own experiences.

I have published three papers in PLoS ONE (and several in other open-access journals). I quite liked all three papers and first tried traditional journals, but the papers were rejected a few times and the students wanted to move on with their lives and research, so we sent them to PLoS ONE, which accepted them quickly. So I decided to ask: How well have these papers been cited relative to papers I published in the same year in other journals? It turned out that I had a decent sample size because my two early PLoS ONE publications (2007 and 2009) happened to occur in years when I published a good number of papers (10 in 2007 and 17 in 2009). So I simply tallied the cumulative number of citations (here always from Web of Science simply for convenience; Google Scholar tells the same story) for each paper I published in those years, ranked them in order of citations, and asked where the PLoS ONE papers fell in relation to the others.

My previous two PLoS ONE papers are cited (Web of Science) least among all my papers published in those two years.

The graph tells the whole story. For my papers published in each of the two years, the PLoS ONE paper ranked DEAD LAST in terms of citations. I did have an a priori expectation that these papers hadn’t been heavily cited, but I had no idea it would be this bad. Moreover, I want to reiterate that I felt these two papers were interesting, well conducted, and potentially important – here and here they are for your citation convenience. Indeed, both were long included in a list of my favorite 15 papers. Of course, the alternative is simply that they really weren’t that good and I just can’t see it. Perhaps so but it remains clear that publishing in PLoS ONE will not enhance your citation rate for a given level of paper quality. So much for one of the supposed benefits (or at least lack of costs) of open access – at least in my case, which I am sure you will agree is what matters here. Of course, this general point is also clear intuitively: How many of us routinely check the papers coming out in Evolution or Ecology versus Evolution and Ecology (or PLoS ONE or PeerJ or Scientific Reports or anything starting with International Journal of …)?

OK, so clearly I am going to argue that you should forego open access and publish in traditional journals – but what of the annoyance, time consumption, and stress of rejection? Well, it is certainly true that rejection is common – for everyone. Many young scientists are stressed out when a paper gets rejected and feel that this somehow reflects on the quality of the work. They also often feel they are getting rejected more often than other scientists. The reality is that even established scientists get rejected all the time. A first proof of this maxim comes from a review paper by Cassey and Blackburn (2004) in Bioscience (cited only 15 times!) that surveyed the most successful ecologists (those with many publications in major society-based journals) and asked them how often their papers were rejected. The answer is often - as shown in the graph below. Indeed, many of these successful ecologists have had the same paper rejected multiple times and some of them still had at least one paper that they had failed to publish despite many tries.

Acceptance rates for papers submitted by the most successful ecologists. From Cassey and Blackburn (2004).

The above data are for the most successful ecologists and are from the good old days when rejection rates were not so high. How about some more recent data from a more mediocre (but established) ecologist (or, more precisely, evolutionary biologist)? 

I am a compulsive record-keeper. In this context, I have recorded every single submission of a paper on which I have been an author, as well as the outcome of those submissions. The sample size is pretty large now and allows me to illustrate the frequency of rejection and some factors that influence it, as summarized in the table below. I have been involved in a total of 275 submissions to journals (including multiple submissions of the same paper to different journals), of which 148 led to acceptance – an acceptance rate of 54%. However, some of those submissions were invited papers or commentaries, or appeared in special issues for which I was an editor. Taking away those near-sureties, my acceptance rate decreases to 43%. On the other hand, I have submitted 45 manuscripts to “big” journals (Science, Nature, PNAS, Current Biology, PLoS Biology) and only 1 was accepted – the first one I submitted. (I realize this looks crazy – 45 such submissions – but more about that later.) Removing these submissions from the tally as well, the acceptance rate jumps back up to 53% for the remaining “real submissions.” Finally, considering only “real submissions” on which I was first author, the rate jumps up again, to 68%. Whew, lots of stats that all simply say: rejection is an ever present companion in science.

Rejection/acceptance stats for my own submissions. 

I am not ashamed of these numbers, nor my relatively low number of open-access publications – because both facts reflect sending papers to the very best journals with low acceptance rates that then subject them to extremely rigorous and critical review, not just for the methods but also for their importance. Of course, it means that one needs to develop a mechanism to cope with rejection. My own mechanism – and the one that I try to tell my students – is that as soon as I submit a paper, I ask myself “OK, where will I submit this paper when it gets rejected.” (Hope for the best, prepare for the worst.) That way, a rejection simply means that I am days away from submitting a paper – turning a bad feeling into a good one. I also tell my students (and myself) that journals reject papers for all sorts of arbitrary reasons and it really doesn’t mean the work isn’t any good. 

While letting this post mature for a few days, I came across a great video of famous failures by people that went on to become wildly successful. This video also reminded me of a story about Tim Mousseau failing (or at least not immediately passing) his qualifying exam at McGill and having to write a remedial paper that has gone on to be cited more than 1000 times. How’s that for turning failure into success? I have heard of many other instances of papers getting rejected from a journal only to be greatly improved, some so much so that they end up getting published at a much “better” journal, such as Science/Nature.

So how high should one shoot? I have been a part of 45 submissions to big journals and all but one failed. Yet I don’t regret them (at least not all of them) – for several reasons. First, analyses (published in Science, of course) have shown that papers submitted to, and rejected from, Science/Nature end up receiving more citations than those that were first submitted elsewhere. Perhaps these were good studies to begin with and were written in a general way, and perhaps the review process improved them. Second, I think a number of my own papers submitted to Nature/Science were very good studies. (Perhaps better than many other papers published there – but then this is the sentiment of everyone that gets rejected from those journals, otherwise they wouldn’t have submitted there in the first place.) Indeed, at least five of my papers rejected from Nature/Science (some from both) have been cited more than 50 times. Three of the rejected papers ended up in Molecular Ecology and each is doing well: one published in 2012 already has 67 citations and two published in 2014 have received considerable attention (one of these was previously rejected from 8 other journals). So, if you have a great study, it is fine – good even – to submit it to Nature/Science. You will write it better and more succinctly, and you might even get some great reviews.

Articles published on their first submission (first intents) are cited less often than articles published in the same journal/year that were first rejected from elsewhere (resubmissions). Data from Calcagno et al. (2012 - Science.)

My favorite route, though, is society-based journals, like Ecology, Evolution, American Naturalist, Proc Roy Soc B, J Evolutionary Biology, J Animal Ecology (I still haven’t cracked the last of these nuts), and so on. These journals are where I want all of my work to end up (and you should too) – it looks good on your CV, and many more people see it and cite it. But, wait, I hear you saying: those papers won’t be accessible to the rest of the world because it requires an expensive subscription. Nonsense. Anyone can get access to any paper from any journal – many papers are posted on someone’s website and, for those that aren’t, all you have to do is email the author to ask for a copy! (I admit getting papers is harder – but certainly not impossible – without institutional access.) Moreover, you can pay for open access in those journals at a cost that isn’t much higher than at PLoS ONE or many other open access journals.

In summary, I suggest you work toward publishing in traditional and well-respected general or society-based journals as your goal, learn to deal with rejection, and only when you are so sick of the paper that you vomit (actually vomit, that is, not just feel nauseous) send it to PLoS ONE or another open access journal. (Or if you need really quickly publications to graduate.) Someone is bound to cite it someday – probably anyway. With this in mind, perhaps you might like to cite the cool new paper we just published this year in PLoS ONE.

-----------------------------------------------------

Note added Dec 3: see my follow-up post (Where to submit your paper - response to reviews.) that responds to post-publication comments on the present post. 

-----------------------------------------------------
Just for fun:

Invited by journal to review your own paper? More common than you think. Here is 1 way to respond. via @danielbolnick pic.twitter.com/Qxq8qTyImz
— Andrew Hendry (@EcoEvoEvoEco) November 24, 2014

@PLOSONE No apology needed, as long as you aren't annoyed by the tongue-in-cheek response.
— Daniel Bolnick (@DanielBolnick) November 25, 2014

Speaking of tongue-in-cheek, see my parody of open access journals here: 

MYScience: A newer faster cheaper easier BETTER open access journal