Search results

Inhalt -- 1 Einleitung -- 1.1 Hintergrund und Relevanz des Themas -- 1.2 Fragestellungen -- 1.3 Eingrenzung des Untersuchungsgegenstands -- 1.4 Aufbau und Gliederung -- 2 Satisficing in Befragungen: Theorie und Forschungsstand -- 2.1 Die Satisficing-Theorie -- 2.1.1 Optimizing und Satisficing als Antwortstrategien -- 2.1.2 Die Einflussgrößen in der Wahl der Antwortstrategie -- 2.1.3 Die Antworteffekte von Satisficing -- 2.2 Die Messung von Satisficing -- 2.3 Die Erklärung von Satisficing: Empirische Evidenz und offene Fragen -- 2.4 Die Erklärung von Satisficing: Stabilität und Variabilität in der Wahl der Antwortstrategie -- 3 Die Messung von Satisficing -- 3.1 Die Modellierung der Antwortstrategie als latente Variable -- 3.2 Methodik -- 3.2.1 Analyseverfahren -- 3.2.2 Daten -- 3.3 Operationalisierung der Indikatoren für Satisficing -- 3.3.1 Straightlining -- 3.3.2 Mittelkategorie-Antworten -- 3.3.3 "Weiß nicht"-Antworten -- 3.3.4 Nichtsubstantielle Antworten auf eine kognitiv fordernde offene Frage -- 3.3.5 Speeding -- 3.4 Ergebnisse -- 3.5 Zusammenfassung und Diskussion -- 4 Die Erklärung von Satisficing -- 4.1 Die Einflussgrößen von Satisficing -- 4.1.1 Schwierigkeit der Aufgabe -- 4.1.2 Fähigkeiten -- 4.1.3 Motivation -- 4.1.4 Bedeutung und Zusammenwirken der Einflussgrößen von Satisficing -- 4.2 Methodik -- 4.2.1 Daten -- 4.2.2 Analyseverfahren -- 4.2.3 Operationalisierung der Einflussgrößen von Satisficing -- 4.3 Ergebnisse -- 4.3.1 Effekte der Schwierigkeit der Aufgabe -- 4.3.2 Effekte der Fähigkeiten -- 4.3.3 Effekte der Motivation -- 4.3.4 Die relative Bedeutung der Einflussgrößen von Satisficing -- 4.3.5 Das Zusammenwirken der Einflussgrößen von Satisficing -- 4.4 Zusammenfassung und Diskussion -- 5 Intra-individuelle Stabilität und Variabilität in der Wahl der Antwortstrategie

In dieser empirischen Studie arbeitet Dr. Joss Roßmann die Theorie und den aktuellen Forschungsstand zu Satisficing in Befragungen umfassend auf und präsentiert neben einem innovativen Modellierungsansatz eine Vielzahl von neuen Erkenntnissen, wie das Zusammenwirken der Schwierigkeit der Beantwortung von Fragen, der Motivation und der Fähigkeiten von Umfrageteilnehmern die Wahl ihrer Antwortstrategie erklären. Abschließend wird die Bedeutung der Befunde für die Theorie und Praxis der Umfrageforschung herausgearbeitet und es werden Implikationen für die Vermeidung und Behandlung von Satisficing in Befragungen aufgezeigt und diskutiert. Der Inhalt Theorie und Forschungsstand • Die Messung von Satisficing • Die Erklärung von Satisficing • Intra-individuelle Stabilität und Variabilität in der Wahl der Antwortstrategie • Fazit und Ausblick Die Zielgruppe Wissenschaftler und Praktiker in der Umfrageforschung Der Autor Dr. Joss Roßmann ist Wissenschaftlicher Mitarbeiter bei GESIS - Leibniz-Institut für Sozialwissenschaften

In this paper we discuss a three-step strategy to evaluate data quality in terms of item nonresponse and to identify potentially flawed questions. We provide an example with several data sets of a large-scale social scientific study to illustrate the application of the strategy and to highlight its benefits.
In survey research it is common practice to test questions ex ante, for example by means of cognitive pretesting. Nevertheless, it is necessary to check the respondents' response behavior throughout the questionnaire to evaluate the quality of the collected data. Articles addressing item nonresponse mostly focus on individuals or specific questions – adjusting the focus on the questionnaire as a whole seems to be a fruitful addition for survey methodology. Shifting the perspective enables us to identify problematic questions ex post and adjust the questionnaire or research design before re-applying it to further studies or to assess the data quality of a study. This need may arise from shortcomings or failures during the cognitive pretesting or as a result of unforeseen events during the data collection. Furthermore, result of this ex post analysis may be an integral part of data quality reports.

Grid questions are frequently employed in web surveys due to their assumed response efficiency. In line with this, many previous studies have found shorter response times for grid questions compared to item-by-item formats. Our contribution to this literature is to investigate how altering the question format affects response behavior and the depth of cognitive processing when answering both grid question and item-by-item formats. To answer these questions, we implemented an experiment with three questions in an eye-tracking study. Each question consisted of a set of ten items which respondents answered either on a single page (large grid), on two pages with five items each (small grid), or on ten separate pages (item-by-item). We did not find substantial differences in cognitive processing overall, while the processing of the question stem and the response scale labels was significantly higher for the item-by-item design than for the large grid in all three questions. We, however, found that when answering an item in a grid question, respondents often refer to surrounding items when making a judgement. We discuss the findings and limitations of our study and provide suggestions for practical design decisions.

Attention checks detect inattentiveness by instructing respondents to perform a specific task. However, while respondents may correctly process the task, they may choose to not comply with the instructions. We investigated the issue of noncompliance in attention checks in two web surveys. In Study 1, we measured respondents' attitudes toward attention checks and their self-reported compliance. In Study 2, we experimentally varied the reasons given to respondents for conducting the attention check. Our results showed that while most respondents understand why attention checks are conducted, a nonnegligible proportion of respondents evaluated them as controlling or annoying. Most respondents passed the attention check; however, among those who failed the test, 61% seem to have failed the task deliberately. These findings reinforce that noncompliance is a serious concern with attention check instruments. The results of our experiment showed that more respondents passed the attention check if a comprehensible reason was given.

Identifying inattentive respondents in self-administered surveys is a challenging goal for survey researchers. Instructed response items (IRIs) provide a measure for inattentiveness in grid questions that is easy to implement. The present article adds to the sparse research on the use and implementation of attention checks by addressing three research objectives. In a first study, we provide evidence that IRIs identify respondents who show an elevated use of straightlining, speeding, item nonresponse, inconsistent answers, and implausible statements throughout a survey. Excluding inattentive respondents, however, did not alter the results of substantive analyses. Our second study suggests that respondents' inattentiveness partially changes as the context in which they complete the survey changes. In a third study, we present experimental evidence that a mere exposure to an IRI does not negatively or positively affect response behavior within a survey. A critical discussion on using IRI attention checks concludes this article.

Open-ended questions have a great potential for analyses, but answering them often imposes a great burden on respondents. Relying on satisficing theory as an overarching theoretical framework, we derived several hypotheses about how respondent and survey level characteristics, and their interactions, might affect the quality of the responses to an open-ended attitude question in self-administered surveys. By applying multilevel analyses to data from 29 web surveys, we examined the effects of respondent and survey level characteristics on three indicators of response quality: response length, response latency, and the interpretability of the answers. With respect to all three indicators, we found that more educated and more motivated respondents provided answers of significantly better quality compared to other respondents. However, the present study provides evidence that analyzing response quality exclusively with process-generated measures of quality may produce a misleading picture. Therefore, the addition of content-related indicators, such as the interpretability of responses, provides a more informative result. We found that the further the open-ended question was located towards the end of the questionnaire, the fewer interpretable answers were given. Our results also indicated that if the survey was carried out in close proximity to a federal election, responses were more likely to be interpretable. Overall, our study suggests that the characteristics at the respondent and survey levels influence the response quality of open-ended attitude questions and that these characteristics interact to a small degree.

With the rising popularity of web surveys and the increasing use of paradata by survey methodologists, assessing information stored in user agent strings becomes inevitable. These data contain meaningful information about the browser, operating system, and device that a survey respondent uses. This article provides an overview of user agent strings, their specific structure and history, how they can be obtained when conducting a web survey, as well as what kind of information can be extracted from the strings. Further, the user written command parseuas is introduced as an efficient means to gather detailed information from user agent strings. The application of parseuas is illustrated by an example that draws on a pooled data set consisting of 29 web surveys.

Satisficing often has been assumed to be a hazard to response quality in web surveys because interview supervision is limited in the absence of a human interviewer. Therefore, devising methods that help to mitigate satisficing poses an important challenge to survey methodology. The present article examines whether splitting up cognitively demanding grid questions into single items can be an effective means to reduce measurement error and nonresponse resulting from survey satisficing. Furthermore, we investigate whether modifying the question design decreases the adverse effects of low ability and motivation of the respondents on response quality. The statistical analyses in our study relied on data from two web-based experiments with respondents from an opt-in and a probability-based online panel. Our results showed that using single items increased the response times compared to the use of standard grid questions, which might indicate a greater response burden. However, the use of single items significantly reduced the amount of response nondifferentiation and nonsubstantive responses. Our results further showed that the impact of respondent ability and motivation on the likelihood of satisficing was moderated by the question design.