Suchergebnisse

Abstract
This article evaluates the use of dynamic adaptive design methods to target outbound computer-assisted telephone interviewing (CATI) in the California Health Interview Survey (CHIS). CHIS is a large-scale, annual study that uses an address-based sample (ABS) with push-to-Web mailings, followed by outbound CATI follow-up for addresses with appended phone numbers. CHIS 2022 implemented a dynamic adaptive design in which predictive models were used to end dialing early for some cases. For addresses that received outbound CATI follow-up, dialing was paused after three calls. A response propensity (RP) model was applied to predict the probability that the address would respond to continued dialing, based on the outcomes of the first three calls. Low-RP addresses were permanently retired with no additional dialing, while the rest continued through six or more attempts. We use a difference-in-difference design to evaluate the effect of the adaptive design on calling effort, completion rates, and the demographic composition of respondents. We find that the adaptive design reduced the mean number of calls per sampled unit by about 14 percent (relative to a modeled no-adaptive-design counterfactual) with a minimal reduction in the completion rate and no strong evidence of changes in the prevalence of target demographics. This suggests that RP modeling can meaningfully distinguish between ABS sample units for which additional dialing is and is not productive, helping to control outbound dialing costs without compromising sample representativeness.

Abstract
In theory, offering each sample member the mode sequence that maximizes their response propensity should increase the response rate and/or reduce the amount of nonresponse follow-up relative to a design that offers all sample members the same mode sequence. However, for this sort of tailoring to be feasible in a cross-sectional survey, it must be possible to use data available prior to data collection (e.g., on the sampling frame) to predict sample members' "mode-sensitivity"—the effect of the offered mode sequence on response propensity. Using data from randomized experiments incorporated into the 2016 and 2019 cycles of the National Household Education Survey, we evaluate whether data appended to an address-based sampling (ABS) frame can accurately predict the sensitivity of household-level response behavior to the initial offer of a paper questionnaire instead of a web instrument and whether a modeled-mode design that tailors the offered mode sequence (web-push vs. paper-only) based on the resulting predictions improves household-level data collection outcomes relative to a uniform web-push design. We find that several characteristics available on the ABS frame show statistically significant interactions with the offered mode sequence in determining the probability of response to initial survey mailings. Consequently, relative to a uniform web-push design, the modeled-mode design increased the response rate to early mailings and reduced the number of mailings required per response. However, the modeled-mode design did not meaningfully increase the final response rate, nor did it lead to a substantial reduction in indicators of nonresponse bias.

Abstract
Monetary incentives are frequently used to improve survey response rates. While it is common to use a single incentive amount for an entire sample, allowing the incentive to vary inversely with the expected probability of response may help to mitigate nonresponse and/or nonresponse bias. Using data from the 2016 National Household Education Survey (NHES:2016), an address-based sample (ABS) of US households, this article evaluates an experiment in which the noncontingent incentive amount was determined by a household's predicted response propensity (RP). Households with the lowest RP received $10, those with the highest received $2 or $0, and those in between received the standard NHES incentive of $5. Relative to a uniform $5 protocol, this "tailored" incentive protocol slightly reduced the response rate and had no impact on observable nonresponse bias. These results serve as an important caution to researchers considering the targeting of incentives or other interventions based on predicted RP. While preferable in theory to "one-size-fits-all" approaches, such differential designs may not improve recruitment outcomes without a dramatic increase in the resources devoted to low RP cases. If budget and/or ethical concerns limit the resources that can be devoted to such cases, RP-based targeting could have little practical benefit.