ALTERNATIVE FACTS: Scientists got 'em too!

“WHAT”, I hear you saying, “That is precisely the point. Science has the facts and the facts are the facts – no alternatives. Anyone who makes statements not supported by the data are presenting fiction, not alternative facts.”

Seen in an Ottawa book store.

Yet maybe we scientists shouldn't be so smug about our FACTS. As the editor and reviewer of countless papers, I am struck by how often two (or more) fully qualified, insightful, and fair scientists can make opposite assertions of fact based on the same data and analysis. Most obviously, data are often interpreted in the light of preconceived expectations (we are all Bayesians) and, so, faced with a series of mixed results, scientists consciously or unconsciously emphasize the subset of results that support their expectations, largely ignoring or discounting or explaining away the rest.  “Ah”, I hear you saying “Those are interpretations, not facts. Facts belong in the results, interpretations are in the discussion.”

I am not so sure. Everyone knows that the same data can be analyzed in multiple ways, each yielding different outcomes. Hence, the statements and statistics in the results are interpretations too, not facts. “Ok, then, the DATA are the facts.” Yet, data are not collected with error, and – sometimes – are biased. Data collected in different ways at different times in different years can yield different outcomes. Different measure of central tendency (mean, median, etc.) yield different numbers. Different data transformations yield different outcomes. Given that data have no meaning without interpretation, even in the narrowest possible sense of a central tendency, data are not facts either.

Alternative facts?

“Ah, but this is what measures of uncertainty are for.” Indeed, standard errors, confidence intervals, credibility intervals and so on can – and should – be calculated but, again, they have no meaning without interpretation. Anytime you look at a result and a credibility interval and say “I therefore conclude the evidence more strongly supports option 1 than option 2.” Sure, but some other scientist can come along and say “I require a higher degree of confidence, so my conclusion is different.” Or. “I would use a different measure of uncertainty.” Or. “You did not include all sources of uncertainty.” Or “You didn’t consider option 3.” And, of course, we ultimately need to accept an outcome one way or the other or we never act on the information we have collected.

So, are there any real facts out there, any “objective truths?” Yes, of course, there are many of them, but many, perhaps most, of those are – ultimately – unknowable. Instead, they are “latent variables” about which we try to make inferences based on our imperfect surrogate measures.

Andy Gonzalez and I with our alternative facts

Alternative facts, which I have just argued are pervasive in science, depend on the level of inference. “Evolution isn’t a theory, it is a fact.” Sure, evolution as a process is a fact. However, evolution in any given case (a particular population facing a particular environmental change) might not be. Don’t believe me? Just look at the knots people studying adaptation to climate change or fisheries have tied themselves into trying to say that evolution has or has not occurred in any given instance. “Look at allele frequency changes,” you might say. But no one cares if those are changing (and they ALWAYS are), we care about the evolution of the traits. “Ok, then, measure the genes influencing those traits.” Yes, that works but what matters is the effect size: how much of the change in phenotype is the result of evolution? That is extremely hard to measure. Climate change is a fact, certainly, but stating that it is a fact is not informative or helpful. We need to know the effect size (rate of change) and its consequences – and then it depends on the place, the time, and the measure taken. Plenty of room for “alternative facts” in this context.

Alternative facts?

So, maybe us scientists shouldn’t be so smug. If we are ever going to state conclusions as facts (as opposed to facts being unknowable), then room exists for alternative “facts” in the same sense. Otherwise we wouldn’t have debates and arguments. Otherwise, we wouldn’t need to replicate our studies, and those of other scientists. Otherwise, we wouldn’t have retractions.

The difference between science and propaganda is that scientists are willing to give up our “facts” when enough evidence suggests they aren’t “facts” after all. 



--------------------------

This post was motivated by reading scientific papers where authors are testing an hypothesis, obtain results that are mixed with respect to that hypothesis, emphasize only those results supporting the hypothesis, and then conclude in favor of the hypothesis. Presumably this sequence of events usually transpires without INTENT to deceive - and, regardless, the consequences are rarely dire. The same cannot be said for the current US government.