Do Alcoholics Anonymous participants do better at abstinence than non-participants because they're more motivated or is it because of something inherent in the AA program?  How researchers answer these questions in studies offers insight into the challenges of evidence-based medicine and evidence-informed policy.  That's the topic of this week's Healthcare Triage.

(Intro)

I have to credit my friend and colleague Austin Frakt, whose Upshot column was the basis for this episode.

This study I referenced earlier, published in the journal Alcoholism: Clinical and Experimental Research teased apart a treatment effect, improvement due to AA itself and the selection effect, driven by the type of people who seek help.  The investigators found that there is a genuine AA treatment effect.  Going to an additional two AA meetings per week produced at least three more days of alcohol abstinence per month.

Separating treatment from selection effects is a long-standing problem in social and medical science.  Their entanglement is one of the fundamental ways in which evidence of correlation fails to be a sign of causation.  For many years, researchers and clinicians have debated whether the association of AA with greater abstinence was caused by treatment or a correlation that arises from the type of people who seek it.

Such confounding is often addressed with an experiment in which individuals are randomly assigned to either a treatment or a non-treatment or a control group in order to remove the possibility of self-selection.  The treatment effect is calculated by comparing outcomes obtained by participants in each group.  Several studies of AA has applied this approach.

For instance, researchers randomized alcoholics to receive treatment that strongly encouraged and supported AA participation or a control group.  The former exhibited a greater degree of abstinence.  In an ideal RCT, everyone selected for treatment gets it, and no one in the control group does.  The difference in outcomes is the difference in the treatment effect, which is free of the bias of selection of choosing a group.

That's the ideal.  However, in practice, randomized controlled trials can still suffer selection problems.  It's one thing to assign people to treatment or control.  It's another to compel them to stick to the group to which they're assigned.  In many studies, researchers can't.  For instance, what's to stop someone assigned to the non-AA group or the control group from going to an actual AA meeting or what forces those in the treatment group to actually attend those meetings?  Nothing.

A real-world trial has what's known as crossover, people not sticking to their random assignment.  It can happen, for instance, if less motivated or sicker people stop adhering to the treatment or it can happen if more motivated ones find a way to receive treatment even when they're assigned to a control group.  Because motivation and health can affect switching and also can be related to outcomes, they can obscure genuine treatment effects.  In other words, they inject a selection effect.

For a study with crossover, comparing treatment and control outcomes reflects the combined real-world effects of treatment and the extent to which people comply with it or receive it even when it's not explicitly offered.  If you want to toss around jargon, this type of analysis is known as an "intention to treat".  A limitation is that the selection effect introduced by crossover can really obscure genuine treatment effects.

To know whether we should do more work to help individuals comply with treatment, it's important to know if the treatment itself actually works.  For that, we need an assessment that's free of the effects of crossover.  Keith Humphreys and colleagues provided one for Alcoholics Anonymous.  Though it's based on study data with crossover, it corrects for it by focusing on the subset of participants who do comply with their random assignment.  In a hypothetical example, imagine that 50% of a sample receive treatment regardless of which group they've been assigned to. 

And likewise imagine that 25% are not treated no matter their assignment. In this imaginary experiment, only a quarter would actually be affected by random assignment. These are known as marginal patients. Not marginal because they don't matter, but because they're the margin affected by randomization. Analysis of
marginal patients yields an estimate of the treatment effect that's free from the bias introduced by crossover. However it's not always the case that the resulting treatment effect is the same as you'd get from an ideal RCT where everyone complied with assignment and no crossover occurred.

Marginal patients may be different from other patients. This is a limitation of analyses such as these. It provides an estimate of the true treatment effect, but only for those who change behavior due to treatment availability. These types of analyses, what economists and other social scientists call instrumental variables analysis, have
been covered in other episodes of healthcare triage.

Despite the limitation, analysis of marginal patients reflects real-world behavior to. Not everyone will comply with treatment, but among those who do, are they better off? That's a question worth
answering. This study does so and tells us that Alcoholics Anonymous helps alcoholics, apart from the fact that it may attract a more motivated group of individuals. With that established, the next step is to encourage even more to
take advantage of its benefits.

Healthcare triage is supported in part by viewers like you through patreon.com, a service that allows you to support the show through a monthly donation. We'd especially like to thank our research
associate Joe Sevits and our surgeon Admiral Sam. Thanks Joe! Thanks Sam!  More information can be found at patreon.com/healthcaretriage.