Suchergebnisse

The validity of true experiments is threatened by a class of self-report biases that affect all respondents at pretest but which are diminished by treatment, yielding noncomparable treated and control subjects at posttest. These biases include inaccurate self-evaluations due to (a) lack of understanding of dimensions of self-rating, (b) unconscious needs to rationalize sources of severe emotional distress, (c) distressed or altered states (e.g., drug states) that lower the ability to report accurately, and (d) deliberate impression management in service of accessing desired treatment. They are detectable via external criteria, special conditions of measurement, and retrospective pretests, and may be lessened in several ways. Unchecked, they produce treatment by self-report bias interactions that undermine the validity of even true experiments.

Evaluation researchers are often confronted with less than optimal conditions in which to design studies. When this occurs, researchers may be forced to utilize relatively weak designs that do not rule out all threats to internal validity. Using archival data from a sales campaign for a state lottery, this article illustrates a multiplist strategy (Cook, 1985) in which several complementary designs are utilized to help rule out the four threats to internal validity associated with the frequently utilized nonequivalent control group design. Specific methods for addressing each of these threats and strengthening the basic nonequivalent control groups design are also illustrated.