Just because you see a study in print doesn't automatically make it a good piece of research. Sadly, there are more than a few studies that have made it into print that are not good, and some are downright ugly. So, how do you tell the difference? For example, if a piece starts off with a sentence like, "The purpose of this study was to prove ..." This should alert you to an upfront bias. This person may be trying to say he/she will prove to you that he/she has found some definitive answer to a problem.

As you recall, science never proves anything, it can only give some level of confidence, and that's about it. Another warning is in the design. Was the design the best one for what was examined? For instance, could the researcher have used a more powerful design to get better data? If researchers say they used a simple one group, pre-test post-test design, could they have not at least used some type of a control group? Or, could they not have used some repeated post-test approach? These are just some of the reasons why you need to stay alert to judge the good stuff from the poor.

What follows is a basic set of guidelines that is a combination of material taken from Gay (1992), Polit and Hungler (1995), Oleson and Arkin (1996), and Ray (2000). The numbers indicate the major headings (in italics) usually seen in a research paper. (Remember, formats may differ.) The guideline questions are directed at each heading. This guideline is far from exhaustive. If you want more depth, please refer to the references provided.

1. The abstract (An abstract is a brief summary of the study, and is always at the very beginning of the article.)
   
   • Are the main purposes outlined?
   • Does it say who was studied? (the sample and size)
   • Does it spell out the design?
   • Does it outline the main findings of the study?
2. The introduction (This section sets the stage for the study. It outlines the literature, framework, need for, and importance of the study, as well as the questions or hypotheses that were addressed.)

   Critique of the literature review

   • Was the review satisfactory?
   • Does it contain all the relevant background for this project?
   • Was the review well organized, and were the references appropriate and recent for the problem under study?
   • Does the review critically appraise and compare key contributions, or is the review merely a review?
   • Does the review conclude with a synopsis of what was stated?

   Critique of theoretical frameworks

   • Does the report describe a theoretical or conceptual framework for the study, and why there is a need for the study?
   • Does the report adequately describe the major features of the theory, so the reader can understand the basis of the problem?
   • Is the theory appropriate to the research problem?
   • Do the research problem and hypothesis flow naturally from the framework?

   Critique of hypotheses/problem statements

   • Does this section clearly present the research problem?
   • Does the problem have significance (in the addictions field)?
   • Has the researcher delimited the scope (defined the boundaries) of the problem?
   • Does the problem statement clearly identify the research variables and the nature of the population under study?
   • Does the report contain a formal stated hypothesis? (If the study requires one.)
   • Is the hypothesis testable?
   • Are the hypotheses properly worded?
   • In what way is the hypotheses stated? As null hypotheses? As research related with direction or non-direction?
3. The method (This section tells the reader exactly how the study was done. It generally consists of several sub-sections: subjects, design, assessments, and procedures.)

   Critique of the subjects/participants

   • Is the target population identified and described?
   • Is the target population appropriate for the design?
   • What inclusion or exclusion criteria were used?
   • Are the size and key characteristics of the sample described?
   • Is the sample sufficiently large?
   • To whom can the sample be generalized?

   Critique of the research design

   • What is the design?
   • Is the design appropriate for the nature of the research question?
   • Did the author spell out the reason for using the particular design?
   • What types of comparisons are specified in the design? Are these comparisons the most appropriate ones?
   • How many times was the data collected? Was this appropriate?
   • What were the controls, if any?
   • How were the groups similar or different in the way they were treated?
   • Why were these groups so critical in relation to the questions under study?
   • Did the design affect the internal or external validity of the study?
   • Does the design allow the researcher to draw conclusions from the variables?

   Critique of the assessment

   • How were the variables of interest measured (e.g., tests, surveys)?
   • Were the instruments used reliable and valid?
   • Were the responses that were gathered able to give relevant information for question or hypothesis under investigation?
   • How was the assessment conducted and who did it?

   Critique of the procedures

   • Where was the study conducted (the setting)?
   • What equipment or apparatuses were used?
   • Was there any other information the reader should know to understand how the subjects were treated?
4. The results (This section presents the data of the study in summary form, along with results of the statistical analysis. It is always in some mathematical form, and usually is avoided by most readers.)

    

   • Was every question/hypothesis tested?
   • Are the results clearly presented?
   • Was the appropriate statistical analysis used to address the question or hypothesis under investigation?
   • Were there significant statistical results?
   • Are the differences large or small?
   • Are there graphs and tables, and do they match the data?
5. The discussion (This section should go into detail about the major findings of the study.)

    

   • What were the major findings of the study?
   • Do the findings add to the current level of research?
   • Do the findings support, refute or update the general theory under investigation?
   • Are there alternative explanations supplied for the findings?
   • Are there limitations to be placed on the study?
   • What are the implications for future study and research?

    

   Of course, there is always a reference section that should hold all the citings used in the study. Make sure that each citing is listed in the reference. These references should be generally primary and have an empirical base of their own.

   By the way, although the writing style of the research article would not generally keep something from getting published, there are a few general guidelines to keep in mind.

   Since we are critiquing, here are a few guidelines to critique the writing of an article:

   • The author needs to be objective at least with the quantitative variety. The qualitative form is not so strict.
   • Most research is given in the passive voice, and rarely if ever uses personal pronouns. The personal language is generally okay with the qualitative research.
   • The report should be clear, simple, and straightforward. Don't believe for a second that if you can't understand what you read that it is your lack of understanding. It may have been badly written. Usually, the better addictions research is rather easy to understand and always well written.

Now you know how to evaluate a piece of research. Don't be afraid to give the next study you read a thorough review using the guidelines we have outlined for you.

It has taken a little over a year, but we have finished the study of research proper. The next time, we move on what many substance use disorder (SUD) counselors really fear — statistics. However, I promise you if you stick with this, you will understand that stuff. And that means you will not have to be afraid of research and especially statistics ever again. This new knowledge will make you more open to reading research, and you might want to incorporate those principles into your clinical work. Someone who combines good clinical skills with research/statistical proficiency makes one hell of a SUD counselor.