This column was challenging — first, because the concepts and the explanations behind this subject are intricate, and trying to get complex material into an easy-to-read format without compromising the quality of the material is tricky. Second, the summarized research findings outlined below are not kind to clinical judgments. Basically, the conclusion, substantiated by an abundant amount of data, is that in many situations actuarial (statistical) prediction methods are more accurate than clinical predictions. That conclusion can be disturbing to folks in our field who pride themselves on their clinical acumen. Third, I could not find corresponding data as it applies to addiction counseling. Therefore, I had to extrapolate the original sources of that information to the addiction field. The problem in this case is that speculation is always a little suspect.

The overall goal of this column is to call attention to a potential problem for addiction counselors — that quite possibly, we make poor predictions about a variety of everyday counseling issues. That possibility has serious implications to our clients.

First — a little background
In 1954 a relatively unknown but important psychology figure named Paul Meehl concluded that statistical prediction outperforms clinical judgment. According to Meehl, psychological theories were unproven and had limited value. He felt that clinicians should stick with observational outcomes verses relying on grand theories. His overall message was that simple models routinely outperform human experts (Trout, 2004).

These hypotheses have been largely ignored by clinicians. Moreover, there may only be a handful of individuals in our field who even recognize this name and his conclusions. It might be safe to assume that counselors, in general, still rely almost exclusively on their clinical judgment to make predictions about their clients. Those findings could easily be adapted to addiction clinicians.

Although some may object to this conclusion (i.e., statistical prediction does a better job than your personal judgment), consider the various judgmental mistakes people make during the course of a day, ranging from misjudging criminal news stories without possessing sufficient facts of the case; or getting a strong negative gut impression of a client that turns out to be totally wrong.

Again, the real point under examination is that our clinical evaluation of clients is an issue to address.

Definitions
The actuarial approach makes decisions based on empirical relations that exist between predictor and outcome variables. Predictor variables are those that predict, such as stress; and boredom factors enhance (predict) relapse potential. Outcome variables are those variables that are predicted. For example, if treatment attendance is steady, treatment outcome (predicted variable) is generally better.

The research reveals that actuarial analysis and algorithms (formulas or computations) have the ability to make highly accurate predictions. On the other hand, the clinical approach, one that makes decisions only with the mental processes of human judgment, is not as accurate (Dawes, Faust & Meehl, 1989). Hence, the difference between the two forms of prediction boils down to one of statistics versus clinical decisions.

Just a minute!
I am assuming that a few readers are thinking, “Hold on, I make good predictions all the time.” While that is certainly an opinion, the arguments and research for the clarity of human thought are not plentiful. Human judgment is by nature flawed (Marchese, 1992; Taleff, 2006). We did not evolve to think particularly well. Our brains evolved more to react (or overreact) and make quick decisions in order to save our skin. Those “over reactions” and other little thinking problems stay with us today.

However, some contend that tests are too simple and miss many variables. They might argue that holistic judgments account for all client information that needs to be processed. Moreover, this is what counselors do well. Appeals such as this imply that those same counselors skillfully sort and combine vast amounts of data. That means clinical judges have to consider each piece of information along with the interactive effects of all those bits of information. The problem is the sheer volume of information to analyze. Holistic judgment often overestimates the extent to which people can combine the mountain of client information, and do all that in their head (Ruscio, 2003).

Specific research data
Dawes, Faust and Meehl (1989) summarized the long history of research on this subject and supported the claim that actuarial methods are superior to clinical ones in terms of diagnosis, prediction of human behavior, and optimal treatment planning options.

The unique human capacity to observe is not the same as the unique capacity to predict. The above authors found that humans are particularly bad at predicting complex outcomes. Dawes et al. note that among other things, humans do not make the same decision every time. People make different predictions from the same set of data. Second, people are bad at distinguishing valid from invalid predictor variables. That is, they do not often have a good handle on what is considered accurate judgment. Third, people lack familiarly with evidence (reality vs. opinion), and often misunderstand it. Lastly, addiction professionals often have their preferred set of biases. The problems are not noticing them, which can interfere with good decision making (Taleff, 2006)

Given these conclusions, why are actuarial methods rarely used in psychology? And, if it is rarely used in psychology, I would be willing to bet it is rarely used in the addiction field. It is likely that professional folks either don’t know this information, or try to ignore it.
Ruscio (2003) and Dawes (2005) proposed that ignoring actuarial methods gets troublingly close to the unethical. This is a powerful point. The same “ignoring” might easily be applied to the addiction field. As such, our leaders, professionals, and government organizations need to suggest or set some guidelines on this issue.

Try it
Since there are no techniques that you can easily apply to a sample of clients and then observe how things turn out, this Try it section is a bit like doing research. In keeping with the theme of this column, compare your clinical judgment to what established test instruments reveal.

I suspect there will be relatively compatible levels of actuarial to clinical judgment in terms of screening clients who might have a potential alcohol or drug problem. This special state of affairs is probably due to the algorithms like screening tests that have existed in the field for a long time.

A more daunting prediction might be what addiction counselors would offer, for example, on a set of Substance Abuse Subtle Screening Inventory (SASSI) sub-scales for a particular client versus what the SASSI itself would indicate for the same client. Those subscales include: attitude toward assessment, defensiveness, emotional pain, ability to acknowledge problems, and risk of legal problems. The trick for counselors is to put forth a prediction on how a particular client would fare in these categories, and compare the results from a client who completed the test.

Perhaps, more interesting would be to predict how good your professional relationship is with a sample of clients. Then ask that sample to take a test on the subject, like the Working Alliance Inventory, and compare results.

Research you can do
Research on this subject might prove daunting. However, you could expand what was outlined above in your own program. Rather than comparing one clinical judgment against one standard test, compare all your staff to a standard test or a variety of standard tests, and note any differences in the predictions.

A more rigorous research design might well become a future publication — something I am considering. Anyone want to collaborate?

Michael Taleff, PhD, CSAS, MAC, is an instructor at the University of Hawai’i, West Oahu, and National University – Hawai’i Branch campuses. He can be reached at This e-mail address is being protected from spam bots, you need JavaScript enabled to view it

References
Dawes, R.M., Faust. D. & Meehl, P.E. (1989). Clinical versus actuarial judgment. Science, 31, 243, 1668-1674.
Dawes, R.M. (2005). The ethical implications of Paul Meehl’s work on comparing clinical versus actuarial prediction methods. Journal of Clinical Psychology, 61,10, 1245-1255.
Marchese, M.C. (1992). Clinical versus actuarial prediction: A review of the literature. Perceptual and Motor Skills, 2, 75, 583-594.
Ruscio, J. Holistic judgment in clinical practice: Utility or futility. The Scientific Review of Mental Health Practice, 2,1, 38-48.
Taleff, M.J. (2006). Critical thinking for addiction professionals. New York: Springer Publications.
Trout, J.D, (2004). The philosophical legacy of Meehl (1978): confirmation theory quality, and scientific epistemology. Applied & Preventative Psychology, 11, 73-76.

This article is published in Counselor,The Magazine for Addiction Professionals, August 2006, v.7, n.4, pp.29-30.

No one has commented on this article.

Please keep your comments brief and on topic, and remember that this is not a discussion thread.
Name :
Comment(s) :

J! Reactions 1.09.00 • Professional Site License
Copyright © 2006 S. A. DeCaro

---