Deciding on Outcome Measures in Behavioral Health
We are at a time where payers are becoming increasingly unwilling to simply pay for access or services delivered (e.g., length of treatment in days or hours, or number of treatment sessions). When looking at outcome measures selected for people with substance use disorders (SUDs), the first thing that comes to mind is abstinence, but using it as an outcome measure is fraught with problems.
The first one is that abstinence is only a guarantee of cessation of drinking or drug use behaviors, and indicates nothing about sober functioning. Think of the father who has an alcohol use disorder and is abusing his children. A child welfare agency becomes involved and refers him to addiction treatment where he gets involved, makes good progress, and is discharged after which he remains sober. However, he continues to abuse his children.
When I was working as a therapist providing group therapy, we had a male patient in the group that I would describe as “hinkey,” meaning that something was off about him, although his participation in the group left nothing to be desired. After about two weeks in treatment he told his fellow group members what he did for living: he was a hitman for organized crime in northern New Jersey. This just goes to show that alcoholism is an equal opportunity employer! Other than his admission about his work there was nothing to indicate that he was doing anything other than being involved in treatment and trying to get sober.
Six months after his discharge, we found out that he had been continuously sober since leaving treatment and was even occasionally attending AA. The clinical staff were congratulating themselves on a job well-done when I suddenly had this awful thought: what we had done was improve his aim.
Using abstinence as the outcome measure disregards patients who, after treatment, still use but do so significantly less frequently, for shorter durations of time, and the using episodes result in less severe consequences. As an example, would you consider it a successful outcome if an alcoholic who in the three years prior to admission to treatment was unable to achieve three consecutive days of abstinence, but who after treatment was able to remain sober for an entire year with the exception of one two-day drinking episode? A mental health example would be the individual with schizophrenia who after treatment still hears voices but does not have to do what the voices say and does not have to be rehospitalized. Are these not successful outcomes?
If we are committed to the position that addiction is a chronic, relapsing brain disease, we must accept that chronic diseases are characterized by exacerbations and remissions of symptoms and stop acting as if it is an acute disease in which one return to the use of substances is equivalent to failure. When comparing addiction to another chronic illness like diabetes, having a blood sugar level within a normal range almost all of the time with an occasional spike is still considered a positive outcome.
Under the DSM-IV (APA, 1994), if individuals have been diagnosed as alcohol dependent (meeting at least three of the seven diagnostic criteria), they were considered in remission even if they continued to drink, but met no more than two of the diagnostic criteria. There is the question of how this relates to the outcome measure of abstinence. We must also consider patients with an alcohol use disorder, mild (APA, 2013) which would be akin to the old DSM-IV diagnosis of abuse, who may return to drinking without resultant problems since this is not addiction as most of us understand it. Consider individuals who encounter a situation with which they cannot cope and begin to use alcohol as a coping mechanism, resulting in an alcohol use disorder, mild (APA, 2013). If the situation resolves itself and they no longer use alcohol as a coping mechanism, these individuals may return to drinking without problems.
Even with all of the above, abstinence may be considered a core outcome measure in contrast to other things as employment or enhanced family functioning, which could be considered by-products, secondary benefits or ancillary outcomes. Treatment programs may have limited, if any, influence on them and typically do not get paid to address them. For example, whether individuals are employed is in part dependent on a variety of things unrelated to abstinence including the nature of the economy; patients’ age; sophistication in job acquisition skills like resume writing, dressing for a job interview, and skills in the interview situation; transportation; and childcare. If treatment providers are to be held accountable for these ancillary outcomes, they need to provide and be reimbursed for these readiness services to prepare patients for employability: vocational assessment, job training, job placement, obtaining a GED, and assistance in finding transportation and childcare during the time they work.
Another outcome measure considered is an enhancement in family functioning. The questions are how much of the dysfunction was the result of the substance use and how much is attributable to patients, sober or not. We all have witnessed the family that remains intact all during the active addiction only to break apart when the family member with the SUD gets sober.
Where outcome measures are currently required by payers or regulators, two of the most common in addition to abstinence are retention and treatment completion. While these seem to be appropriate outcomes, the provider can game the system by keeping patients in treatment who are not engaged in working toward recovery, who are just doing time, not doing treatment. Permitting then to continue functioning is this way improves retention and treatment completion numbers but is antithetical to the purposes of treatment.
Good outcome measures are difficult to develop. Currently, if Medicare patients are readmitted to a hospital with thirty days for the same condition, the hospital is penalized financially. The assumption is that if the hospital did a good job, this would not have happened. What may not be considered is the patients’ role in the need for readmission. Did they follow the postdischarge instructions given to them? If the admission was diabetes-related, did patients check their blood sugar? Did they watch their diet? Did they take their diabetes medication as prescribed? Did they stop smoking? Did they lose weight?
There are no easy answers! While the movement toward outcome-based or value-based payment is very necessary and will occur for all of us, my question is whether we have selected and defined appropriate outcome measures. This is complicated by the fact that the SUD treatment field is still struggling to reach agreement on such things as what addiction really is, the use of pharmacotherapy (particularly agonist drugs), and the provider’s response to patients using while in treatment.
American Psychiatric Association (APA). (1994). Diagnostic and statistical manual of mental health disorders (4th ed.). Washington, DC: Author.
American Psychiatric Association (APA). (2013). Diagnostic and statistical manual of mental health disorders (5th ed.). Washington, DC: Author.