Search results

Questionnaires are widely used in the social sciences and very often survey data form the basis for governmental and commercial planning or evaluation. Yet the quality of survey data is not attested to, since a large variety of factors in the language-use situation prove to influence the answers unintentionally. The forbid/allow asymmetry is a well-known example of this: when respondents are asked whether something should be forbidden, about 50% may answer 'yes, forbid' - whereas an equivalent question phrased with the verb 'to allow' could well cause up to 75% of the respondents to answer 'no, it should not be allowed'. Which question wording is preferable to measure respondents' true attitudes? Only when we know why the answers differ, can we decide on that. This book is the first to apply a systematic cognitive approach to describe the causes of the forbid/allow asymmetry. The question-answering process is unravelled by a variety of experiments and meta-analytic techniques. Analyses reveal that the difference in question wording does not prompt respondents to retrieve different attitudes. Instead, the asymmetry reflects that the question wording causes the response options to be used differently. Because of the qualifying dimensions in the question text, the meanings of 'yes' and 'no' change, as well as the cognitive distance between them. This study sheds a different light on processes of question-answering and text interpretation. Furthermore, practical advice on questionnaire design and on the interpretation of survey data is given on the basis of these new insights

As the verbs forbid and allow are considered each other's counterparts, one would expect the answers to questions worded with forbid or allow to be each other's opposites. Research shows that this is not the case. Use of these verbs in surveys causes a wording effect known as the forbid/allow asymmetry. Findings do not reveal, however, where the asymmetry originates: during the stage of attitude localization or during the mapping of the attitude onto one of the response options. In this article, a correlational design was used. Two experiments were carried out focusing on the congenericity of forbid and allow, one on attitudes toward environmental issues and a replication on attitudes concerning ethnic groups. Results of both experiments show that forbid and allow questions are congeneric; that is, they measure the same attitude. Answers to forbid/allow questions reflect similar attitudes that are expressed differently on the answering scales due to the use of both verbs. In addition, the explanation of the effect focusing on the answering behavior of indifferent respondents is discussed and explored.

Voting Advice Applications are online tools that provide users with a voting advice based on their answers to a set of political attitude questions. This study investigated to what extent VAA users understand the questions that lead to the voting advice, and what search and response behaviour they expose in case of comprehension difficulties. Two studies were conducted to investigate these issues: a cognitive interviewing study among 60 VAA users during the Dutch municipal elections in the city of Utrecht, and a statistical analysis of all answers provided by 357,858 users who accessed one of the 34 municipal VAAs during these same elections. Results of the two studies show a coherent picture: difficult concepts (e.g., tax names or municipal jargon), geographical locations (e.g., reference to a specific street), and vague quantifying terms (e.g., "more") all complicate the question. In case of comprehension difficulties, Study 1 shows that VAA users make little effort to solve their problems, for example by looking up difficult terms on the Internet. Instead, they draw inferences about what the question might mean and proceed to answer nonetheless. These are often neutral or no opinion answers, which seems to suggest that the meanings of those options are confounded. In Study 2, however, we found that the choice for either a neutral or no opinion response is not accidental: semantic meaning problems often result in no opinion answers, whereas pragmatic problems are related to neutral responses. We discuss the implications of these findings for survey theory and practice.

Voting Advice Applications are online tools that provide users with a voting advice based on their answers to a set of political attitude questions. This study investigated to what extent VAA users understand the questions that lead to the voting advice, and what search and response behaviour they expose in case of comprehension difficulties. Two studies were conducted to investigate these issues: a cognitive interviewing study among 60 VAA users during the Dutch municipal elections in the city of Utrecht, and a statistical analysis of all answers provided by 357,858 users who accessed one of the 34 municipal VAAs during these same elections. Results of the two studies show a coherent picture: difficult concepts (e.g., tax names or municipal jargon), geographical locations (e.g., reference to a specific street), and vague quantifying terms (e.g., "more") all complicate the question. In case of comprehension difficulties, Study 1 shows that VAA users make little effort to solve their problems, for example by looking up difficult terms on the Internet. Instead, they draw inferences about what the question might mean and proceed to answer nonetheless. These are often neutral or no opinion answers, which seems to suggest that the meanings of those options are confounded. In Study 2, however, we found that the choice for either a neutral or no opinion response is not accidental: semantic meaning problems often result in no opinion answers, whereas pragmatic problems are related to neutral responses. We discuss the implications of these findings for survey theory and practice.

"Voting Advice Applications are online tools that provide users with a voting advice based on their answers to a set of political attitude questions. This study investigated to what extent VAA users understand the questions that lead to the voting advice, and what search and response behaviour they expose in case of comprehension difficulties. Two studies were conducted to investigate these issues: a cognitive interviewing study among 60 VAA users during the Dutch municipal elections in the city of Utrecht, and a statistical analysis of all answers provided by 357,858 users who accessed one of the 34 municipal VAAs during these same elections. Results of the two studies show a coherent picture: difficult concepts (e.g., tax names or municipal jargon), geographical locations (e.g., reference to a specific street), and vague quantifying terms (e.g., 'more') all complicate the question. In case of comprehension difficulties, Study 1 shows that VAA users make little effort to solve their problems, for example by looking up difficult terms on the Internet. Instead, they draw inferences about what the question might mean and proceed to answer nonetheless. These are often neutral or no opinion answers, which seems to suggest that the meanings of those options are confounded. In Study 2, however, we found that the choice for either a neutral or no opinion response is not accidental: semantic meaning problems often result in no opinion answers, whereas pragmatic problems are related to neutral responses. We discuss the implications of these findings for survey theory and practice." (author's abstract)

Abstract
Pension participants face complex decisions. Information about the decision alternatives often contains the exact monetary amounts associated with the alternatives. Grounded in Fuzzy-Trace Theory (FTT), we argue that informed decision making requires participants to (accurately) understand meaningful differences between decision alternatives, rather than to rely on purely numerical differences in amounts. The objective of this study was to identify, for three decisions Dutch participants can make regarding their old-age pension, what dimensions participants take into account when giving meaning to the decision alternatives. We conducted 39 semi-structured in-depth interviews with Dutch pension participants in which we discussed the considerations of participants to choose for the decision alternatives. These interviews were analyzed using thematic analysis. Our analysis shows that a number of dimensions are taken into account, partially overlapping for the three decisions: life expectancy, (in)sufficient income, financial maximization, work–life balance, and job satisfaction. According to FTT, participants can be supported in extracting meaningful representations by providing cues about the dimensions that might be affected by the various decision alternatives. The dimensions identified in this paper can serve as a useful starting point for such support.

"For decades, survey researchers have known that respondents give different answers to attitude questions worded positively (X is good. Agree-Disagree), negatively (X is bad. Agree-Disagree) or on a bipolar scale (X is bad-good). This makes survey answers hard to interpret, especially since findings on exactly how the answers are affected are conflicting. In the current paper, we present twelve studies in which the effect of question polarity was measured for a set of thirteen contrastive adjectives. In each study, the same adjectives were used so the generalizability of wording effects across studies could be examined for each word pair. Results show that for five of the word pairs an effect of question wording can be generalized. The direction of these effects are largely consistent: respondents generally give the same answers to positive and bipolar questions, but they are more likely to disagree with negative questions than to agree with positive questions or to choose the positive side of the bipolar scale. In other words, respondents express their opinions more positively when the question is worded negatively. Even though answers to the three wording alternatives sometimes differ, results also show that reliable answers can be obtained with all three wording alternatives. So, for survey practice, these results suggest that all three wording alternatives may be used for attitude measurement." (author's abstract)

Voting Advice Applications (VAAs) provide voting recommendations to millions of people. As these voting recommendations are based on users' answers to attitude questions, the framing of these questions can have far-reaching consequences. The current study reports on a field experiment in which the framing of the header above VAA statements (N = 17) was manipulated (condition 1: no header; condition 2: a right-wing header, e.g., finance; condition 3: a left-wing header, e.g., nature and environment). Visitors of a VAA developed for Utrecht, the fourth largest municipality in the Netherlands, were randomly guided to one of the versions of the tool in which the header type was varied. Results (based on N respondents = 27,404) show that providing a header (left-wing or right-wing) leads to more left-wing answers as compared a condition where there is no header above the attitude statement. This effect, however, is only observed for respondents with lower levels of political sophistication.

Voting Advice Applications (VAAs) fulfill different needs for different citizens. In national elections, the majority of users can be characterized as politically sophisticated citizens who use VAAs for entertainment purposes and confirmation of their party preference, but a significant minority uses VAAs to learn about politics and make an informed vote choice. VAAs might, however, play a different role in second-order elections, since in these elections campaign dynamics and information supply are very different. In the current research, we applied latent class analysis on user data from a widely used Dutch VAA (Kieskompas) for a supranational and several subnational elections in the Netherlands, to test if an extant typology of VAA users for national elections could be replicated. We find that the typology can be replicated for most of these elections, but also that the relative size of the groups of users differs across elections; in all second-order elections except for the provincial elections, more doubters and seekers are found relative to national elections. This suggests that VAAs are likely to have stronger mobilizing potential in these second-order elections.

Online Voting Advice Applications (VAAs) are survey-like instruments that help citizens to shape their political preferences and compare them with those of political parties. Especially in multi-party democracies, their increasing popularity indicates that VAAs play an important role in opinion formation for citizens, as well as in the public debate prior to elections. Hence, the objectivity and transparency of VAAs are crucial. In the design of VAAs, many choices have to be made. Extant research in survey methodology shows that the seemingly arbitrary choice to word questions positively (e.g., 'The city council should allow cars into the city centre') or negatively ('The city council should ban cars from the city centre') systematically affects the answers. This asymmetry in answers is in line with work on negativity bias in other areas of linguistics and psychology. Building on these findings, this study investigated whether question polarity also affects the answers to VAA statements. In a field experiment (N = 31,112) during the Dutch municipal elections we analysed the effects of polarity for 16 out of 30 VAA statements with a large variety of linguistic contrasts. Analyses show a significant effect of question wording for questions containing a wide range of implicit negations (such as 'forbid' vs. 'allow'), as well as for questions with explicit negations (e.g., 'not'). These effects of question polarity are found especially for VAA users with lower levels of political sophistication. As these citizens are an important target group for Voting Advice Applications, this stresses the need for VAA builders to be sensitive to wording choices when designing VAAs. This study is the first to show such consistent wording effects not only for political attitude questions with implicit negations in VAAs, but also for political questions containing explicit negations.

Voting Advice Applications (VAAs) are Web tools that are used to inform increasing numbers of voters during elections. This increasing usage indicates that VAAs fulfill voters' needs, but what these needs are is unknown. Previous research has shown that such tools are primarily used by young males and highly educated citizens. This suggests that VAAs are generally used by citizens who are already well-informed about politics and may not need the assistance of a VAA to make voting decisions. To analyze the functions that VAAs have for their users, this study utilizes unique user data from a popular Dutch VAA to identify different user types according to their cognitive characteristics and motivations. A latent class analysis (LCA) resulted in three distinct user types that vary in efficacy, vote certainty, and interest: doubters, checkers, and seekers. Each group uses the VAA for different reasons at different points in time. Seekers' use of VAAs increases as Election Day approaches; less efficacious and less certain voters are more likely to use the tool to become informed.