| SUBSCRIBER LOGIN |
|---|
| News Briefs | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
||||||||||
| Polls |
|---|
| Special Offer |
|---|
|
|
| If Something Can Go Wrong, It Will |
| Columns - Research to Practice | ||||||||
| Saturday, 30 September 2000 | ||||||||
|
Editor's note: This article is part 8 in a series written by Dr. Michael J. Taleff to help clinicians, prevention specialists, supervisors and administrators use and understand research. Parts 1-7 of the series, which addressed experimental designs and introduced threats to internal and external validity, were published in NAADAC's The Counselor and are now available online at Counselor Magazine Online
There can be many explanations for a treatment effect. For
example, does the maturity level of the clients under study have a major impact
on substance-use disorder outcome? Or is it the regression effects (the tendency
to return to some level of normality) that have the major impact? Or is it the
passing of time itself that mostly affects outcome? Or does the effect of simply
taking repeated tests, or the test instrument being used, have an impact on
outcome?
The following six threats are less common than those discussed
in Part 7. All are from Heppner, Kivlighan and Wampold
(1999).
You may find that, following an experiment; there are clear
differences in an outcome between one group and another. You feel pleased and
confident you have come across an intervention that seems to produce a
difference. But what if the differences existed before the experiment? That
would be an internal threat to your outcome called selection. Differences in
groups before we divide them into control and experimental groups do exist. An
important way around this possibility is to make sure your subjects were
randomly assigned to a control and an experimental group.
Next, suppose you make this observation: As you share some of
your personal history during a session, there seems to be more progress in
treatment. This seems to be a logical assumption on your part. However, how can
you be sure that a client's progress is not causing more disclosures from you?
This threat to validity is called ambiguity about the direction of a causal
influence. Until the direction is established, all you have is a correlation,
not a causal factor. In our example, one way to check on the direction is to
stop disclosing and see what happens to the progress of treatment.
Suppose you are conducting an experiment to test the
effectiveness of a new form of treatment you developed. You must have two
groups, give one group (the experimental group) the treatment and withhold the
treatment from the other group (the control group). You know that one group is
not going to receive your new treatment, yet the thought of withholding a
treatment that might benefit clients does not sit well with you. So you
indirectly (or directly) provide some additional service to the control group to
make up for not giving them the new treatment.
When you give that extra service to the control group, you
contaminate the outcome of your research. This threat to validity is called
compensatory equalization of treatments. In science, we often must withhold
treatment from a group to find out whether the treatment works better than
nothing at all. That doesn't mean you have to withhold a new treatment forever.
If the experiment demonstrates that the treatment works, you can then apply it
to the control group.
Similar to the deliberate compensation of services to a
control group is the unwitting spread of treatment to that group.
This is most likely to happen if the treatment involves
providing information. The information might be about how to reduce the spread
of HIV, for example, and the objective is to measure whether the information
changes attitudes. If that special set of information unintentionally spreads to
a control group, the outcome will be compromised. This threat to validity is
called diffusion or imitation of treatments. The way to avoid such a situation
is to give the information to the experimental group only.
Staying with the experimental group and the control group
experiment, suppose you remain firm in your conviction to science and do not
apply any parts of your new treatment to the control group.
Now suppose that individuals in the control group find out
that they are not receiving some new form of treatment. They could decide to
show you they can do as well as the experimental group; that is, the control
group acts differently than it usually would. That would certainly confound the
results of your experiment. This threat to validity is called compensating
rivalry by participants receiving less desirable treatments. Again, the simple
way to avoid this situation is to secure what you are doing to only the
experimental group.
In the same situation suppose that the control group, instead
of trying to compensate and acting in a positive way, becomes demoralized and
acts differently, but negatively. Again, this would affect your results. This
threat to validity is called resentful demoralization of participants receiving
less desirable treatments.
Internal threats to validity apply to the various forms of
experimental designs discussed in Parts 1 through 6 of this series. It is in
your best interest to keep these threats to a minimum in any research you
conduct and keep these threats in mind as you attempt to make sense of daily
treatments you institute. Knowledge of internal threats to research validity may
enable you to reach more accurate conclusions about the difference between
effective strategies and guesswork.
Next: External threats to
validity.
Michael J. Taleff, PhD, CAC,
MAC, is assistant
References
Powered by !JoomlaComment 3.25
3.25 Copyright (C) 2007 Alain Georgette / Copyright (C) 2006 Frantisek Hliva. All rights reserved." |
||||||||
| < Prev | Next > |
|---|


















