Suchergebnisse

AbstractOne of the strongest findings across the sciences is that publication bias occurs. Of particular note is a "file drawer bias" where statistically significant results are privileged over nonsignificant results. Recognition of this bias, along with increased calls for "open science," has led to an emphasis on replication studies. Yet, few have explored publication bias and its consequences in replication studies. We offer a model of the publication process involving an initial study and a replication. We use the model to describe three types of publication biases: (1) file drawer bias, (2) a "repeat study" bias against the publication of replication studies, and (3) a "gotcha bias" where replication results that run contrary to a prior study are more likely to be published. We estimate the model's parameters with a vignette experiment conducted with political science professors teaching at Ph.D. granting institutions in the United States. We find evidence of all three types of bias, although those explicitly involving replication studies are notably smaller. This bodes well for the replication movement. That said, the aggregation of all of the biases increases the number of false positives in a literature. We conclude by discussing a path for future work on publication biases.

We examine the trade-offs associated with usingAmazon.com's Mechanical Turk (MTurk) interface for subject recruitment. We first describe MTurk and its promise as a vehicle for performing low-cost and easy-to-field experiments. We then assess the internal and external validity of experiments performed using MTurk, employing a framework that can be used to evaluate other subject pools. We first investigate the characteristics of samples drawn from the MTurk population. We show that respondents recruited in this manner are often more representative of the U.S. population than in-person convenience samples—the modal sample in published experimental political science—but less representative than subjects in Internet-based panels or national probability samples. Finally, we replicate important published experimental work using MTurk samples.

Election administrators & public officials often consider changes in electoral laws, hoping that these changes will increase voter turnout & make the electorate more reflective of the voting-age population. The most recent of these innovations is voting-by-mail (VBM), a procedure by which ballots are sent to an address for every registered voter. Over the last 2 decades, VBM has spread across the US, unaccompanied by much empirical evaluation of its impact on either voter turnout or the stratification of the electorate. In this study, we fill this gap in our knowledge by assessing the impact of VBM in one state, OR. We carry out this assessment at the individual level, using data over a range of elections. We argue that VBM does increase voter turnout in the long run, primarily by making it easier for current voters to continue to participate, rather than by mobilizing nonvoters into the electorate. These effects, however, are not uniform across all groups in the electorate. Although VBM in OR does not exert any influence on the partisan composition of the electorate, VBM increases, rather than diminishes, the resource stratification of the electorate. Contrary to the expectations of many reformers, VBM advantages the resource- rich by keeping them in the electorate, & VBM does little to change the behavior of the resource-poor. In short, VBM increases turnout, but it does so without making the electorate more descriptively representative of the voting-age population. 2 Tables, 1 Appendix, 35 References. Adapted from the source document.

Good survey and experimental research requires subjects to pay attention to questions and treatments, but many subjects do not. In this article, we discuss 'Screeners' as a potential solution to this problem. We first demonstrate Screeners' power to reveal inattentive respondents and reduce noise. We then examine important but understudied questions about Screeners. We show that using a single Screener is not the most effective way to improve data quality. Instead, we recommend using multiple items to measure attention. We also show that Screener passage correlates with politically relevant characteristics, which limits the generalizability of studies that exclude failers. We conclude that attention is best measured using multiple Screener questions and that studies using Screeners can balance the goals of internal and external validity by presenting results conditional on different levels of attention. Adapted from the source document.

Concerns about video-based political persuasion are prevalent in both popular and academic circles, predicated on the assumption that video is more compelling than text. To date, however, this assumption remains largely untested in the political domain. Here, we provide such a test. We begin by drawing a theoretical distinction between two dimensions for which video might be more efficacious than text: 1) one's belief that a depicted event actually occurred and 2) the extent to which one's attitudes and behavior are changed. We test this model across two high-powered survey experiments varying exposure to politically persuasive messaging (total n = 7,609 Americans; 26,584 observations). Respondents were shown a selection of persuasive messages drawn from a diverse sample of 72 clips. For each message, they were randomly assigned to one of three conditions: a short video, a detailed transcript of the video, or a control condition. Overall, we find that individuals are more likely to believe an event occurred when it is presented in video versus textual form, but the impact on attitudes and behavioral intentions is much smaller. Importantly, for both dimensions, these effects are highly stable across messages and respondent subgroups. Moreover, when it comes to attitudes and engagement, the difference between the video and text conditions is comparable to, if not smaller than, the difference between the text and control conditions. Taken together, these results call into question widely held assumptions about the unique persuasive power of political video over text.

AbstractInattentive respondents introduce noise into data sets, weakening correlations between items and increasing the likelihood of null findings. "Screeners" have been proposed as a way to identify inattentive respondents, but questions remain regarding their implementation. First, what is the optimal number of Screeners for identifying inattentive respondents? Second, what types of Screener questions best capture inattention? In this paper, we address both of these questions. Using item-response theory to aggregate individual Screeners we find that four Screeners are sufficient to identify inattentive respondents. Moreover, two grid and two multiple choice questions work well. Our findings have relevance for applied survey research in political science and other disciplines. Most importantly, our recommendations enable the standardization of Screeners on future surveys.

Does media choice cause polarization, or merely reflect it? We investigate a critical aspect of this puzzle: How partisan media contribute to attitude polarization among different groups of media consumers. We implement a new experimental design, called the Preference-Incorporating Choice and Assignment (PICA) design, that incorporates both free choice and forced exposure. We estimate jointly the degree of polarization caused by selective exposure and the persuasive effect of partisan media. Our design also enables us to conduct sensitivity analyses accounting for discrepancies between stated preferences and actual choice, a potential source of bias ignored in previous studies using similar designs. We find that partisan media can polarize both its regular consumers and inadvertent audiences who would otherwise not consume it, but ideologically opposing media potentially also can ameliorate the existing polarization between consumers. Taken together, these results deepen our understanding of when and how media polarize individuals.

This study investigated the cognitive processing of true and false political information. Specifically, it examined the impact of source credibility on the assessment of veracity when information comes from a polarizing source (Experiment 1), and effectiveness of explanations when they come from one's own political party or an opposition party (Experiment 2). These experiments were conducted prior to the 2016 Presidential election. Participants rated their belief in factual and incorrect statements that President Trump made on the campaign trail; facts were subsequently affirmed and misinformation retracted. Participants then re-rated their belief immediately or after a delay. Experiment 1 found that (i) if information was attributed to Trump, Republican supporters of Trump believed it more than if it was presented without attribution, whereas the opposite was true for Democrats and (ii) although Trump supporters reduced their belief in misinformation items following a correction, they did not change their voting preferences. Experiment 2 revealed that the explanation's source had relatively little impact, and belief updating was more influenced by perceived credibility of the individual initially purporting the information. These findings suggest that people use political figures as a heuristic to guide evaluation of what is true or false, yet do not necessarily insist on veracity as a prerequisite for supporting political candidates.