Can We Trust Scientific Research?

A comment from a Twitter follower (@AddictionDrW) asserted that studies supporting the effectiveness of opioid maintenance therapy for opioid addiction must have been funded or supported in some way by "Big Pharma" and therefore cannot be trusted. He rejected all the research, saying that we should be listening to recovering addicts instead, or that "evidence is mixed." Is that true?

Methadone was invented in Germany as an analgesic during WWII. The first study of methadone maintenance, by Vincent Dole, Marie Nisswander and then-resident Mary Jeanne Kreek, was funded not by Big Pharma (who wanted nothing to do with "junkies"), but by the Health Research Council of New York City, due to the failure of abstinence-based approaches in the face of a growing heroin problem (Joseph et al., 2000). To my knowledge, it's never been a significant money maker for a large pharmaceutical company. Subsequent studies have for the most part been funded by government organizations such as the National Institutes on Health (NIH). The most recent large study of Suboxone maintenance for prescription opioid addicts, was funded by the NIH.

If NIH funded research, conducting in a rigorous way, is not trustworthy information, then what is? In my mind this is a way of asserting, in essence, that whatever one wants to believe is true. There are no objective methods to determine whether one assertion is true and its opposite is false. Therefore, there can be no accumulated knowledge, and we are all on our own, depending on whatever we wish to believe, or have heard from others. We can simply reject information we don't like or agree with and accept evidence that supports what we believe.


I don't think that is a supportable position, because it leads to the radical conclusion that there is no objective truth, only subjective opinions. Please, folks, how besides through scientific research are we going to improve outcomes for addiction treatment? And if we choose only to believe results that support our pre-existing ideas, how can we progress?

I understand that we all operate that way to a significant degree, because these biases are built into our brain structure and function, and are almost completely unconscious  Why is not clear, but then traits that nature selects through evolution seldom have immediately clear rationales. As my colleague, Steve Gilbert, argued in a previous blog, we make decisions and then rationalize them after the fact. We weave a coherent narrative of our lives because we need to. We need reasons why things happen, we need to feel in control, and we need to feel like a specific person with a specific history and characteristics. We need be able to explain how we arrived at a decision, and to simply say, "My gut told me so," is seldom adequate. Imagine a presidential debate where that was the rationale for political positions: "It just feels right to me."

On the other hand, it is because of these biases that the scientific method was devised: to minimize bias, to constrain interpretation, and to reveal the precise basis for arriving at a specific conclusion, so that others could replicate the study. Although single large trials may be provide evidence so strong that it changes thinking and practice, more often it is an accumulation of evidence from many studies conducted by multiple independent scientists. Evidence from multiple studies is then collected into systematic reviews, and if there are enough high-quality studies, are then subjected to an obscure procedure called meta-analysis. Meta-analysis is a systematic statistical method for combining findings across studies in order to determine whether, overall, a particular treatment is more effective than its alternative. Two famous examples are aspirin and beta-blockers for prevention of heart attacks after a first heart attacks. It wasn't until the meta-analyses were done that it became clear that these are effective approaches. Sometimes, it takes multiple meta-analyses before a conclusion can really be drawn.

Recent history is littered with examples of approaches that were fervently held and promoted prematurely. How many thousands of women underwent a horrendous and often fatal procedure, bone-marrow transplant, to treat end-stage breast cancer, before it was shown to be ineffective? At the time, women were lobbying legislatures around the country to mandate insurance coverage for it. Those of us who said, "Wait for the research before moving ahead," were shouted down, with tragic consequences for the unfortunate women who received the procedure and their families. Hormone replacement after menopause is another example, being strongly recommended and widely prescribed in the past, to prevent osteoporosis and heart disease, only to learn through meta-analyses, as well as very large well-done trials, that the treatment is actually harmful.

Perhaps the best recent example is that of PSA testing for prostate cancer in men. It was never clear that PSA testing was a good idea, but it became extremely widespread, if not universal. Millions of men have had  prostate biopsies (a very unpleasant procedure,) radical prostatectomies, radiation that scarred their rectums, and other treatments that left them incontinent and impotent. However, two large randomized controlled trials, one in the US and one in Europe, determined that almost all the tumors being treated were slow-growing and probably would never require treatment if left alone. In one trial, there was no difference in mortality, while in the other, there was a slight advantage to PSA testing, but 50 men would have to be treated in order to save one life. I know I don't want to be one of the 49 treated who didn't need it.

However, PSA testing is still popular in the US. Why? Because we are terrified of cancer, and we have been told that "catching it early and getting it out" is the best way to avoid dying of it. Many urologists refuse to believe the studies, based I would guess on their experience treating men with advanced, fast-growing tumors. (It's not even clear that treating fast-growing tumors alters survival.) I don't believe this is because of their financial interests in performing procedures; I know physicians too well to believe they don't have their patients' best interests in their minds and hearts. But they suffer from biases as well. We whack out prostates not because it is beneficial to most men undergoing surgery, we do it because it "feels like the right thing to do."

One of the reasons our health care costs so much is because of our refusal to stop funding ineffective or harmful medical procedures and treatments because "people want them." Here's another example: people presenting to the doctor with back pain were randomly (like the toss of a coin) assigned to either 1) receive regular XRays of their spine, or 2) to get an MRI. After one year, what was the result? In terms of function and pain, there were no significant differences. But there was one significant difference: people getting an MRI had more surgery. The fact is, most people over the age of 35 have abnormal MRIs of their spines, but the correlation with clinical symptoms is very low. And yet, people traipse into the doctor's office demanding an MRI of their back "because I want to know what it is." And the US does more back surgery than any other country in the world, probably more than the rest of the world combined, with highly dubious results. We also consume more prescription opioid pain medication than the rest of the world combined. So biases of this type are not limited to the field of addiction treatment. However, there is one difference: urologists and other physicians are obligated to advise patients of all alternative treatments and to do so in an unbiased way. The patient needs this information to make an informed decision.

A footnote about the "the evidence is mixed" strategy. This approach is not new, having been used by tobacco company executives and climate change deniers for many years. It is always possible to find one of 6 scientists in the world who reject climate change, and then present their opinions as equivalent to the 95% of scientists who are convinced of the evidence. This is a problem with current media, an obsession with presenting two sides to the story without providing the context of the overwhelming majority opinion or of the research itself. In the case of opioid maintenance treatment, the evidence is simply overwhelming in showing that it reduces relapse to addictive use, improves health and function, and reduces crime, and that abstinent-based approaches do not. If you choose to believe something else, that's up to you, but a professional in my opinion has en ethical responsiblity to accept scientific findings and act accordingly. In similar vein, anti-relapse medications for smoking and for alcohol use disorder are not home runs, but they improve outcomes consistently. To say otherwise is to filter the evidence according to preference, to see only what one wants to see. Treatment programs and their professional staff members who deny people with a devastating and often fatal illness access to these treatments, or even to information about them, are imposing their own biases and beliefs on their clients, in violation of the basic tenets of informed consent.

References
Joseph H, Stancliff S, Langrod J. (2000) Mt Sinai J Med 67:347.