The Messages Replication Factor: Methods Tailored to Messages as Objects of Study

In: Journalism quarterly, Band 71, Heft 4, S. 984-996

Abstract

In research on effects of message variables, it is generally necessary to examine responses to actual messages that represent, embody, or instantiate the values of the variable of interest. Researchers have lately become attentive to problems of confounding in the use of individual concrete messages to represent abstract theoretical contrasts, and replicated treatment comparisons are increasingly common in communication research. How to treat the replications factor in the statistical analysis remains controversial. Whether to treat replication factors as fixed or as random hinges on what is assumed about the relationship between abstract treatment contrasts and their concrete material implementations. We argue that reflection on this relationship justifies a general policy of treating replications as random. Two circumstances in which fixed-effects analyses might seem attractive (the case of matched-message designs and the case of experimental manipulations occurring outside of messages) are considered, but it is concluded that these situations also require random-effects analyses.