Counselor Magazine, the Magazine for Addiction Professionals - Research With Reservations: One Group Post-Test And Pre-Test/Post-Test Designs

Daily E-mail Updates
Get news updates in your Inbox! Subscribe to our Counselor Magazine news syndication E-mail service for quick, easy notifications every time we add content to the site. Enter your email address: Delivered by FeedBurner

SUBSCRIBER LOGIN

News Briefs

Innovative Drugs-of-Abuse Testing System

Concateno plc, Europe's largest provider of drug and alcohol testing programmes, and Royal Philips Electronics announced an innovative handheld drugs-of-abuse detection device.

Concateno-Philips pioneering system will make testing for drugs such as cocaine and heroin quicker, simpler and easier, enabling immediate point-of-care drug test results, similar to current alcohol tests. Concateno and Philips will deliver the new solution to selected customers in the second half of 2009.

Polls

Can Recovering Drug Addicts Drink?

	Yes
	No
	I don't know

Special Offer

Research With Reservations: One Group Post-Test And Pre-Test/Post-Test Designs

Columns - On the Web

Monday, 31 January 2000

Fourth in a series of short articles describing the ideas and methods of research in clear, understandable language. Each article will address a segment of research in a way that clinicians, prevention specialists, supervisors, and administrators will find useful and friendly. Collectively, the articles will provide a "refresher course" that can be referred to as needed.

We have reviewed two forms of research that are often used in the Substance Use Disorder (SUD) field-the true and quasi experiments. It was shown that these forms of research are solid, reliable methods to acquire data. This time we will discuss less trustworthy research methods.

Why spend time going over stuff that doesn't work as well? Because it is important to be able to recognize lower-quality methods, and because, despite the bad reviews these methods get, there are a few good reasons to use them.

One metaphor to keep in mind for this whole research series is that of an airplane trip. As the plane takes off, you look out the window and see buildings connected by highway systems.

This sight makes it clear how things are operating on the ground. The plane then begins to encounter cloud cover, and the clarity of the view is diminished. As the plane gains altitude, more clouds obstruct your view of the ground, so that detail is lost and only the largest things on the ground can be seen.

In this metaphor, the clearest part of the trip is the true experiment, the somewhat cloudy part is the quasi experiment, and the part in which you can barely see anything is what follows.

Research 101: Defining Terms

As usual, we need to define our terms.

One group post-test method

This says that we are going to take one group (people, rats, pigeons.), do something to them (such as administer some form of treatment or reinforcement), and assess for any results. More simply and as the name implies, we will collect results of an intervention after (post) providing some kind of intervention or treatment to a group (National Institute on Drug Abuse, 1993; Heppner, Kivlinghan, & Wampold, 1998).

Graphically, this method looks like XO O, where X is the intervention (e.g., a new SUD therapy) and O is the observation or measurement after the intervention was used. That's it. It is the simplest of all the designs this column will address.

However, due to its lack of control, it has been dubbed the "one-shot case study." One-shot even sounds shoddy (Gay, 1992).

If a SUD treatment program wanted to measure its effectiveness using only this design, that program would not a get a very good answer to the question, "Is my program effective?"

Questionable results develop from not using:

any methods of control. (If I can't control for other variables that might affect the outcome, how can I be sure what I did was effective?)
a comparison group. (If there is nothing to compare the results to, how can I be sure what I did was effective?)

Because of these limitations, we can never be sure that some level of change has occurred to the group under investigation. Some experts consider this method to be a pre-experiment rather than an experiment (Gay, 1992). Beware of those who claim success in a SUD program using only this design.

Despite these drawbacks, the post-test does have its place in SUD research. For example, this design is less intrusive than the ones we saw earlier, it is less expensive, and it requires far less effort. It can also assess for the usefulness of a more rigorous evaluation, and it can help search for promising variables or perceived improvements that need explaining (Posavac & Carey, 1997).

Let's move on to something a bit more reliable.

One-group pre-test/post-test method
The difference here is that we can measure change (National Institute on Drug Abuse, 1993). This design is also simple. Essentially, we take a pre-test measurement from a group, institute some form or therapy or do something to the group, and then take a post-test measurement to see if anything changed since the first measurement. Graphically, it looks like O1 O X O O2, where O1 is the first observation or measurement, X is the intervention, and O2 is the second observation or measurement.

The pre-test/post-test method suffers from many of the same limitations as post-test only. Most importantly, you cannot discount the possibility that other explanations account for a desired outcome. However, in this case you can say with certainty that something really has changed.

Therefore, the case for this design has its positives, and its place in SUD research as well.

A Note on Terminology

Please note that the four research designs we have addressed so far-the true experiment, the quasi experiment, one group post-test, and one group pre-test/post-test-can be defined in little more than a sentence. Some people think research must be presented in a convoluted manner, with high-sounding names. That's not accurate, but it is true that researchers often condense their results into a compact set of words (formulas), sometimes using mathematics. This can be intimidating to SUD practitioners, most of whom are accustomed to obtaining their information in sentence form. Later in this series, we will discuss formulas and statistics in an attempt to remove any mystery.

One additional thought: Some researchers will modify the basic designs we have discussed. For example, you might see a pre-test/post-test design that is a "simple time-interrupted series." It is still a pre-test/post-test, which you already know about; there is simply an added element of multiple observations that occur before and after a treatment was used. In graphic form, this example looks like (O1,O2,O3,O4 O X O O5,O6,O7,O8), where each O is a separate observation and the capital X is the treatment. In this case, there are more observations, and that makes the data a little more reliable.

That drive for reliable data is the motivation behind this research series — to find ways that tell us how our clients really change, rather than rely on unconfirmed reports.

Next: case reports, surveys, and correlation research.

Michael J. Taleff, PhD, CAC, MAC, is assistant professor in the Counselor Education Department and project director of Chemical Dependency Programs at Pennsylvania State University. He is also a member of the NAADAC Research Committee and welcomes comments on this series. His e-mail address is This e-mail address is being protected from spam bots, you need JavaScript enabled to view it .

References
Gay, L.R. (1992). Educational research: Competencies for analysis and application (4th ed.). New York: Merrill Publishing.

Heppner, P.P., Kivlinghan, D.M., & Wampold, B.E. (1998). Research design in counseling (2nd ed.). Pacific Grove, CA: Brooks/Cole.

National Institute on Drug Abuse (1993). A guide to evaluation: How good is your drug abuse treatment program? (NIH Publication No. 93-3609). Washington, DC: U.S. Government Printing Office.

Posavac, E.J., & Carey, R.G. (1997). Program evaluation: Methods and case studies (5th ed.). Belmont, CA: Prentice Hall.

Comments

Add New

RSS