I don’t recall where I saw the link to the Harper’s article, The revolution will not be pasteurized: Inside the raw-milk underground, but I’m glad it came to my attention. The article is quite long, but very interesting, and mostly excellent. Oddly enough, it isn’t the subject of raw milk that leads me to write about the article.

The last two paragraphs are what shouted to me; they’re presented in abridged form below.

But when it comes down to it, here’s why I drank the raw milk. The sun had just come up, and we’d already finished three hours of work in the barn. I was filled with a righteous hunger. .... Although dairy isn’t for everyone, I come from the people of the udder: my ancestors relied so heavily on milk that they passed down a mutation allowing me to digest lactose. For many generations my forefathers sat down to meals like this after the morning milking. It felt unambiguously right.

This, of course, is the very definition of bias: the conflation of what feels right with what is scientifically correct. But as it was, I could only hope that my biases were rooted in something more than nostalgia. .... Someday the uncertainties of dietary science will fall to manageable levels, but until then I will rely on my gut. I drained my cup and poured thick clabbered milk and apple syrup on my porridge. If any bacteria disagreed with my body, the conflict was too small to detect.

Hate to break it to you, Mr. Johnson, but that isn’t the most common definition of bias in science. Usually it refers to the researchers skewing methodology and/or reporting to favor a particular result or interpretation. But that’s the least of my gripe with your words.

What, exactly, does “scientifically correct” mean? Contrary to strong popular myth, scientific methods do not definitively prove things—they rely on probabilistically-based tests and careful experimentation to find a result, and give an idea of how likely that finding is due to chance factors. That’s what the p level indicates; and it used to be (I’ve been out of this field for a long time) that the minimum standard for significance in most studies was .05—meaning that 5 times out of 100, the results could have been obtained by chance.

There isn’t a friendly sign that indicates when chance factors rather than the experimental manipulation(s) have led to a statistically significant result, so researchers try to control the research conditions as much as possible; and more important, studies are often repeated to some degree, to help address this issue. (One problem, however, is that absolute replication would be expensive and is often seen as not worth it, so subsequent studies may address the same issue but in a slightly different way. Almost certainly the test subjects are different, for starters.) It’s really difficult to assess whether studies that touch on the same issue(s) are really close enough to count as a replication. This is just one issue I have with meta studies—research that gathers all the data from many studies examining a particular question, and looks at that pooled information to try to definitively address the research question.

Another tool in the researcher’s kit is power—setting up the study so that if meaningful effects are present, they will be found in the statistical analysis. One easy way to increase power is to increase the sample size—the number of subjects tested. Having too many participants can lead to problems, but the cost of research generally precludes that situation occurring.

The important thing to understand is that the statistical results, and the conclusions the researcher draws from them, are based on aggregate information—in most tests, the mean and standard deviation (or similar measures of central tendency and variability) are the core figures underlying the statistical analysis. In fact, a careful researcher will have established, in advance of collecting any data, a set of criteria for eliminating outliers from the analysis. In most cases that’s probably wise, but sometimes, those outliers may be where the most interesting information resides.

So, given this aggregate-based methodology, what can one study—or even a whole raft of studies on the issue—say about any given individual’s likely behavior under the studied conditions? Not much, actually. Speaking probabilistically, the way to bet is that each individual will be close to the mean, but that assumes a so-called normal distribution (that nice bell curve that so much science is based upon), and it assumes that individuals are more alike than they really are. [It is an interesting side note to wonder exactly how common a symmetrical distribution is in nature. Given that hand dominance in humans is highly skewed—decidedly not a normal distribution—one wonders why it is so universally assumed to hold for just about everything else.]

Also, science in a lab can be very far removed from the conditions of everyday life. As a brief example, in my former field of expertise, tachistoscopic presentation of letter combinations for very brief periods (we’re talking milliseconds here) has led to various theories of reading and general perceiving: but does anyone really think that the bulk of our daily reading/perceptual experience is captured by those static, barely-there stimuli?

I’m getting rather far afield from my basic point, which is this: scientific findings rely on aggregate information often gathered under artificial conditions. At best, they offer ideas about general human functioning (again, an interesting side topic for exploration, since many studies of “normal” behavior rely on college students or paid volunteers rather than the population at large). Also, despite all the advances current technology offers, there’s still a lot we don’t know about human functioning and its myriad intricacies.

In Mr. Johnson’s case, he identified a reason for going against what is current scientific wisdom regarding raw milk (albeit one that speaks not to contamination issues, but overall digestion capabilities). But there, too, is the influence of the aggregate: government busybodies and bureaucrats have decided that raw milk is risky to us all, and since most research in this country is funded by some institution or other (gov or corporate interests), the findings are couched in language that appeases the pocketbooks.

Science, it seems to me, has become an institution in its own right—an unassailable purveyor of rock-solid conclusions, of truth. But for many reasons—and for me, its reliance on aggregate information being foremost among them—that perspective is narrow and potentially dangerous. Science is supposed to be a self-correcting way of gaining understanding; it was not intended to prescribe and proscribe things at an individual level. This view, along with the myth that a study or two proves a particular finding, has left me (a former researcher!) very skeptical of the institution.

Still, I read the news articles, and when I’m sufficiently interested I’ll try to find the original research in order to grok more fully. But—perhaps because I am one of the minority of lefties on the planet—I do not order and re-order my life around the findings of the day. I might try something, and if it works for me, great; if not, I don’t worry that I am somehow defective. I’m simply an individual—and the bulk of science is not oriented toward an individualistic understanding.