Previous: You Can Do a Lot to Prevent Some Cancers
Next: August 17, 2016 - LIVE



View count:33,285
Last sync:2018-05-01 05:30
Do Alcoholics Anonymous participants do better at abstinence than nonparticipants because they are more motivated? Or is it because of something inherent in the A.A. program?

How researchers answered these questions in a recent study offers insight into challenges of evidence-based medicine and evidence-informed policy. That's the topic of this week's Healthcare Triage.

This episode was adapted from one of Austin Frakt's Upshot columns. Links to references and further reading can be found there:
John Green -- Executive Producer
Stan Muller -- Director, Producer
Aaron Carroll -- Writer
Mark Olsen -- Graphics

And the housekeeping:

1) You can support Healthcare Triage on Patreon: Every little bit helps make the show better!
2) Check out our Facebook page:
3) We still have merchandise available at
Do Alcoholics Anonymous participants do better at abstinence than non-participants because they're more motivated or is it because of something inherent in the AA program?  How researchers answer these questions in studies offers insight into the challenges of evidence-based medicine and evidence-informed policy.  That's the topic of this week's Healthcare Triage.


I have to credit my friend and colleague Austin Frakt, whose Upshot column was the basis for this episode.

This study I referenced earlier, published in the journal Alcoholism: Clinical and Experimental Research teased apart a treatment effect, improvement due to AA itself and the selection effect, driven by the type of people who seek help.  The investigators found that there is a genuine AA treatment effect.  Going to an additional two AA meetings per week produced at least three more days of alcohol abstinence per month.

Separating treatment from selection effects is a long-standing problem in social and medical science.  Their entanglement is one of the fundamental ways in which evidence of correlation fails to be a sign of causation.  For many years, researchers and clinicians have debated whether the association of AA with greater abstinence was caused by treatment or a correlation that arises from the type of people who seek it.

Such confounding is often addressed with an experiment in which individuals are randomly assigned to either a treatment or a non-treatment or a control group in order to remove the possibility of self-selection.  The treatment effect is calculated by comparing outcomes obtained by participants in each group.  Several studies of AA has applied this approach.

For instance, researchers randomized alcoholics to receive treatment that strongly encouraged and supported AA participation or a control group.  The former exhibited a greater degree of abstinence.  In an ideal RCT, everyone selected for treatment gets it, and no one in the control group does.  The difference in outcomes is the difference in the treatment effect, which is free of the bias of selection of choosing a group.

That's the ideal.  However, in practice, randomized controlled trials can still suffer selection problems.  It's one thing to assign people to treatment or control.  It's another to compel them to stick to the group to which they're assigned.  In many studies, researchers can't.  For instance, what's to stop someone assigned to the non-AA group or the control group from going to an actual AA meeting or what forces those in the treatment group to actually attend those meetings?  Nothing.

A real-world trial has what's known as crossover, people not sticking to their random assignment.  It can happen, for instance, if less motivated or sicker people stop adhering to the treatment or it can happen if more motivated ones find a way to receive treatment even when they're assigned to a control group.  Because motivation and health can affect switching and also can be related to outcomes, they can obscure genuine treatment effects.  In other words, they inject a selection effect.

For a study with crossover, comparing treatment and control outcomes reflects the combined real-world effects of treatment and the extent to which people comply with it or receive it even when it's not explicitly offered.  If you want to toss around jargon, this type of analysis is known as an "intention to treat".  A limitation is that the selection effect introduced by crossover can really obscure genuine treatment effects.

To know whether we should do more work to help individuals comply with treatment, it's important to know if the treatment itself actually works.  For that, we need an assessment that's free of the effects of crossover.  Keith (?~3:22) and colleagues provided one for Alcoholics Anonymous.  Though it's based on study data with crossover, it corrects for it by focusing on the subset of participants who do comply with their random assignment.  In a hypothetical example, imagine that 50% of a sample(?~3:36)