Search results

Abstract
A prior study found that mailing prepaid incentives with $5 cash visible from outside the envelope increased the response rate to a mail survey by 4 percentage points compared to cash that was not externally visible. This "visible cash effect" suggests opportunities to improve survey response at little or no cost, but many unknowns remain. Among them: Does the visible cash effect generalize to different survey modes, respondent burdens, and cash amounts? Does it differ between fresh samples and reinterview samples? Does it affect data quality or survey costs? This article examines these questions using two linked studies where incentive visibility was randomized in a large probability sample for the American National Election Studies. The first study used $10 incentives with invitations to a long web questionnaire (median 71 minutes, n = 17,849). Visible cash increased response rates in a fresh sample for both screener and extended interview response (by 6.7 and 4.8 percentage points, respectively). Visible cash did not increase the response rate in a reinterview sample where the baseline reinterview response rate was very high (72 percent). The second study used $5 incentives with invitations to a mail-back paper questionnaire (n = 8,000). Visible cash increased the response rate in a sample of prior nonrespondents by 4.0 percentage points (from 31.5 to 35.5), but it did not increase the response rate in a reinterview sample where the baseline reinterview rate was very high (84 percent). In the two studies, several aspects of data quality were investigated, including speeding, non-differentiation, item nonresponse, nonserious responses, noncredible responses, sample composition, and predictive validity; no adverse effects of visible cash were detected, and sample composition improved marginally. Effects on survey costs were either negligible or resulted in net savings. Accumulated evidence now shows that visible cash can increase incentives' effectiveness in several circumstances.

Abstract
Most online survey questions testing political knowledge are susceptible to measurement error when participants look up the answers. This article reports five studies of methods to detect and prevent this common source of error. To detect lookups, "catch questions" are more reliable than self-reports, because many participants lie rather than admit looking up answers. Strongly worded instructions reduced lookups by about two-thirds, while the triple combination of instructions, requesting a promise not to look up answers, and adaptive feedback (asking participants who look up an answer to stop doing so) reduced the percentage of respondents looking up an answer by a further half, to 3%. For office recall knowledge items, photo-based open-ended questions eliminated lookups and had similar validity to traditional text-based versions, making them a good choice when a visual format is viable.

This study presents new measures of opinion about progress toward racial equality and provides a multifaceted rationale for preferring the new measures to the old ones. To reduce several sources of measurement error and improve analytic bite by breaking progress into its constituent elements, surveys should ask about past, present, and ideal conditions, not "progress." These questions reveal racially polarized opinions: Black and White Americans agree on the goal of equality and agree that conditions were worse in the past, but Blacks think conditions were much worse than Whites do. They especially differ in opinions on current conditions and thus in how much change is required to achieve the goal of equality. Blacks see much more current inequality than Whites do. These opinions help explain preferences for affirmative action (AA). Contrary to previously published findings, reactions to AA do not depend on opinions on progress but depend strongly on something related but distinct: how much current racial conditions differ from the ideal. Implications for theories of policy preferences, racial attitudes, progress, and equality are discussed.

Political knowledge research faces a problem, perhaps even a crisis. For two decades, the American National Election Studies asked open-ended questions about political knowledge and coded answers using procedures that are neither reliable nor replicable and that were never shown to be optimally valid. Consequently, conclusions based on these widely used measures of the public's competence are in doubt. This article presents several new and overdue methodological improvements: coding knowledge data using formal and specific coding rules based on a substantive rationale for the validity of the codes, recognizing partially correct answers, using multiple coders working independently, using machine coding, and testing reliability and validity. The new methods are an improvement because they are transparent and replicable and they produce valid and extremely reliable knowledge data. Further, machine coding produces codes nearly identical to those from a team of human coders, at much lower cost.

This study identified affective and cognitive factors predicting American 6th, 8th, and 10th graders' civic orientation, defined here as feelings of effective community service, conceptualizations of citizenship, and participation in student government. Independent variables included measures of interpersonal trust, valuation of religion, and individualistic versus collective action attributions of responsibility for solving social problems. Interpersonal trust predicted four out of the five outcome variables. Individualistic attribution of social responsibility was a predictor of running for student government office, and collective action attribution was a predictor of conceptualizations of citizenship. Religious valuation also predicted conceptualizations of citizenship as well as feelings of effective community service. For the most part, grade level did not play a significant role in predicting youth's civic orientation. Results are discussed in terms of the literature on social capital and developmental theory.

This study identified affective & cognitive factors predicting American 6th, 8th, & 10th graders' civic orientation, defined here as feelings of effective community service, conceptualizations of citizenship, & participation in student government. Independent variables included measures of interpersonal trust, valuation of religion, & individualistic vs collective action attributions of responsibility for solving social problems. Interpersonal trust predicted four out of the five outcome variables. Individualistic attribution of social responsibility was a predictor of running for student government office, & collective action attribution was a predictor of conceptualizations of citizenship. Religious valuation also predicted conceptualizations of citizenship as well as feelings of effective community service. For the most part, grade level did not play a significant role in predicting youth's civic orientation. Results are discussed in terms of the literature on social capital & developmental theory. 2 Tables, 1 Appendix, 57 References. Adapted from the source document.

Data from the 1984 & 1996 American National Election Studies are drawn on to compare public support for the Moral Majority & the Christian Coalition. Separate logistic regression models indicate that Moral Majority supporters (about 9.4% of 1984 respondents) are disproportionately female; Evangelical; & highly religious; & feel negatively toward abortion, gays & lesbians, & the women's liberation movement. Christian Coalition support (about 15.5% of 1996 respondents) is more closely linked to partisanship & to issues such as abortion & gay rights. Support for the Christian Coalition in 1996 was less the province of the highly religious than support for the Moral Majority had been. Overall, support for the Christian Coalition is somewhat more structured than support for the Moral Majority; the model correctly classified 33% of Christian Coalition supporters compared with 20% of Moral Majority supporters. Perhaps because of its more politically astute leaders, the Christian Coalition has a broader & larger base of support than the Moral Majority had, & its supporters constitute a sizable block -- about 17% -- of Americans. 3 Tables, 14 References. Adapted from the source document.

Abstract
This article reports the results of an experiment comparing branch, grid, and single-item question formats in an internet survey with a nationally representative probability sample. We compare the three formats in terms of administration time, item nonresponse, survey breakoff rates, response distribution, and criterion validity. On average, the grid format obtained the fastest answers, the single-item format was intermediate, and the branch format took the longest. Item nonresponse rates were lowest for the single-item format, intermediate for the grid, and highest for branching, but these results were not statistically significant when modeling the full experimental design. Survey breakoff rates among the formats are not statistically distinguishable. Criterion validity was weakest in the branching format, and there was no significant difference between the grid and single-item formats. This evidence indicates that the branching format is not well suited to internet data collection and that both single-item and short, well-constructed grids are better question formats.

AbstractRising costs and challenges of in-person interviewing have prompted major surveys to consider moving online and conducting live web-based video interviews. In this paper, we evaluate video mode effects using a two-wave experimental design in which respondents were randomized to either an interviewer-administered video or interviewer-administered in-person survey wave after completing a self-administered online survey wave. This design permits testing of both within- and between-subject differences across survey modes. Our findings suggest that video interviewing is more comparable to in-person interviewing than online interviewing across multiple measures of satisficing, social desirability, and respondent satisfaction.

Abstract
It is well documented that survey overreporting of voter turnout due to social desirability bias threatens inference about political behavior. This paper reports four studies that contained question wording experiments to test questions designed to minimize that bias using a "pipeline" approach. The "pipeline" informs survey participants that researchers can perform vote validation to verify turnout self-reports. This approach reduced self-reported turnout by 5.7 points in the 2020 American National Election Study, which represents a majority of the estimated overreporting bias. It reduced reported turnout by 4 points in two nonprobability samples. No effect was found in a third nonprobability study with Amazon Mechanical Turk workers. Validated vote data also confirm that the pipeline approach reduced overreporting. We tested heterogeneous effects for sophistication and several other variables, but results were inconclusive. The pipeline approach reduces overreporting of voter turnout and produces more accurate estimates of voters' characteristics.