Speaking of High Quality Research Methods ...

Sunni's picture

As we were a couple of weeks ago—it appears one of the gold–standard methods is most often just gold plated.

Somehow or other yesterday, I wound up on Science 2.0, and discovered an intriguing but infuriating article: If Placebos Are Standards For Trials, What’s In Placebos?. Here’s a sample (leaving out the tasty twist on this research):

[T]he researchers ... found that the placebo ingredients for pills were disclosed in fewer than 10 percent of cases. The nature of the “control” was significantly more likely to be stated for other types of treatments, like injections, acupuncture, or surgery, where people are more likely to question what ‘placebo’ actually means in that context.

“How often study results are affected by what’s in the placebo is hard to say — because, as this study showed, most of the time we have no idea what the placebo is,” Golomb concluded.

For anyone interested in the original research, the abstract is all that’s freely available online; but it’s short and easily understood. (For those with adequate self control, check out this blog post.)

There are all kinds of ways to push research findings via placebo contents, and as another quote from Golomb makes clear, the temptation to do so is very high, since researchers have a vested interest—and the possibility of a lot of money—riding on the study results. The scenario that intrigues me the most, however, is the case in which researchers want to accept the null hypothesis: that of no statistically significant difference between control (placebo) and independent variable (treatment).

Think about it: if you’ve been hard at work for years, trying to formulate something like a low-calorie sweetener, what better way to push the data the way you want than to use a placebo that has a small amount of your substance? That might explain how possible excitotoxins like aspartame and monosodium glutamate have been classified by the FDA as “generally regarded as safe”, despite mounting evidence that they are not.

Golomb’s findings leave me questioning much of medical research—particularly recent research, as the pharmaceutical industry has shown itself to be rapacious, and not always honest, in its quest for new profit-making drugs as well as pushing extant, patented drugs to doctors and patients. It’s very sad, how far these professions have fallen.

Predictability

I think most of us would like to have things work predictably, especially in our medicine and diet, but the longer I live the more illusive that seems to be.

Not only do different people respond with rather wide variability to substances, depending on many things each individual can react quite differently from one dose to another, greatly frustrating the hope of tight control. And what is merely frustration with one medicine or herb, may be very serious indeed for something like fast acting insulin for a brittle diabetic.

Add dishonest research methods and ulterior motives, and it all becomes a nightmare.

I've participated in some pain management research for various drugs, and I never saw an honest study. They all had their conclusions mapped out and would toss any data that did not support it. I quit in disgust when they tried to tell me how they wanted me to document my visits.

Of course, I never knew for sure who was getting a placebo, but it never even occurred to me that the placebo might be anything but a completely inert substance. Diabolical!

Health care

I saw a comment the other day to the effect that it probably wasn't until 1930 or so that going to the doctor was likely to do more good than harm. This set me to wondering the date at which this inversion had reverted back to the norm. 1970?

Incentives

Apropos of this I ran into Why published research findings are often false on Slashdot a few days ago. Add to that Does the Vaccine Matter? from The Atlantic a bit more than a year ago and it is starting to look like science, perhaps especially medical science, is not quite as pure as its supporters make it out to be.

It seems that in addition to the results, data and methodology one must now look at what incentives those conducting the "research" have. As this is likely to be next to impossible......

So instead of a "test" group

So instead of a "test" group given a compound and a "control" group given... another compound _called_ a placebo, how 'bout this:

Primary Test Group: given the drug specifically meant to be tested
Secondary Group: given a different drug/compound, and to be evaluated to the same standards as the primary
Control Group: not given anything (or given empty gel-caps from the same batch as those used to administer the primary and secondary test compounds, or injected with th same distilled water used as a carrier in the other two groups, and so forth).

Or simply tell the control group the truth that they won't get anything. That eliminates even the psychosomatic variable.

I've often wondered if the real reason double-blind placebo testing isn't to ease the consciences of the experimenters who can then honestly say they didn't know they weren't really treating a specific individual.

Which is worse? To tell some poor guy dying of cancer that they'll flip a coin to see if he gets the real medicine but will pretend he did either way and that he'd better hope the trial ends before it's too late, or to tell him "We honestly need to see how a drug performs compared to those who don't get it. If you'll agree to skip the drug and let us monitor you, you'll get the treatment free of charge if it pans out in time, or we'll give each of your kids a 20 year annuity if it doesn't."

Of course, this still assumes the researchers are honest and aren't letting vested interests bias the experiment.

From what I was taught in

From what I was taught in Psych 101 a few years ago placebo's are generally composed of sugar and if not some other inert ingredient. Though if the ingredient is not mentioned in a study that would definitely trigger some suspicion from me. Any legitimate scientific experiment should disclose ALL variables and should be repeatable. A true thesis can be repeated elsewhere with equal results. If a thesis (from a study) fails to disclose a variable than the experiment cannot be repeated elsewhere rendering it unprovable.