ObjectiveCultural Theory (CT) has attracted significant attention across the social sciences and is increasingly being used in survey research. We assess the construct validity of three CT survey operationalizations to help interpret and improve these measures.MethodsA coding protocol for face and content validity of survey items was developed with input from several CT scholars and applied independently by two of authors of this article. Convergent, discriminant, and predictive validity of these items were assessed using survey data.ResultsWe find that these measures generally lack face and content validity but have reasonably good convergent, discriminant, and predictive validity.ConclusionWhile these measures can continue to be used to predict attitudes and behaviors that CT hypothesizes will vary with culture, scholars interested in testing CT's basic claims in survey research should seek to improve their face and content validity, which will also allow better tests of convergent, discriminant, and predictive validity.
AbstractWeather and climate disasters pose an increasing risk to life and property in the United States. Managing this risk requires objective information about the nature of the threat and subjective information about how people perceive it. Meteorologists and climatologists have a relatively firm grasp of the historical objective risk. For example, we know which parts of the United States are most likely to experience drought, heat waves, flooding, snow or ice storms, tornadoes, and hurricanes. We know less about the geographic distribution of the perceived risks of meteorological events and trends. Do subjective perceptions align with exposure to weather risks? This question is difficult to answer because analysts have yet to develop a comprehensive and spatially consistent methodology for measuring risk perceptions across geographic areas in the United States. In this project, we propose a methodology that uses multilevel regression and poststratification to estimate extreme weather and climate risk perceptions by geographic area (i.e., region, state, forecast area, and county). Then we apply the methodology using data from three national surveys (n = 9,542). This enables us to measure, map, and compare perceptions of risk from multiple weather hazards in geographic areas across the country.
Quantitative Research Methods for Political Science, Public Policy and Public Administration for Undergraduates: 1st Edition With Applications in R is an adaption of Quantitative Research Methods for Political Science, Public Policy and Public Administration (With Applications in R). The focus of this book is on using quantitative research methods to test hypotheses and build theory in political science, public policy and public administration. This new version of the text omits large portions of the original text that focused on calculus and linear algebra, expands and reorganizes the content on the software system R and includes guided study questions at the end of each chapter. ; https://dc.etsu.edu/etsu-oer/1004/thumbnail.jpg
AbstractSocial criteria are important to achieving the mission of the National Weather Service. Accordingly, researchers and administrators at the NWS increasingly recognize a need to supplement verification statistics with complementary data about society in performance management and evaluation. This will require significant development of new capacities to both conceptualize relevant criteria and measure them using consistent, transparent, replicable, and reliable measures that permit generalizable inference to populations of interest. In this study, we contribute to this development by suggesting three criteria that require measurement (forecast and warning reception, comprehension, and response) and demonstrating a methodology that allows us to measure these concepts in a single information domain—tornado warnings. The methodology we employ improves upon previous research in multiple ways. It provides a more generalizable approach to measurement using a temporally consistent set of survey questions that are applicable across the United States; it relies on a more robust set of psychometric tests to analytically demonstrate the reliability of the measures; and it is more transparent and replicable than previous research because the data and methods (source code) are publicly available. In addition to describing and assessing the reliability of the measures, we explore the sensitivity of the measures to geographic and demographic variation to identify significant differences that require attention in measurement. We close by discussing the implications of this study and the next steps toward development and use of social criteria in performance management and evaluation.
Methods for identifying relevant policy impacts for valuation in benefit-cost analyses (BCAs) have received relatively little attention in academic research, applied policy analyses, and guidance documents. In this paper, we develop a systematic, transparent, and replicable process that draws upon information contained in records of Congressional hearings to identify relevant policy impacts for valuation in a BCA. Our approach involves classifying – and subsequently analyzing – statements from witnesses testifying in Congressional hearings on the topic of the BCA. By using Congressional hearings as the basis for our approach, we are identifying potential policy impacts from information provided during the very process the BCA is intended to inform. However, because this approach is quite resource-intensive and would be somewhat burdensome for agencies to implement, it may be best applied in the academic realm, with identified impacts resulting from such applications then made available to agency personnel for potential inclusion in BCAs. Using the case of the Glen Canyon Dam, we demonstrate the approach and its resulting improvements in the quality and transparency of the BCA it was intended to inform.
AbstractTornadoes impose enormous costs on society. Relatively simple and inexpensive enhancements to building codes may reduce these costs by 30% or more, but only one city in the United States has adopted these codes. Why is this the case? This analysis addresses this question by examining homeowner support for more stringent building codes in Oklahoma, a conservative state that routinely experiences damaging tornadoes. Survey data show that support for mandatory mitigation policies like building codes is subject to countervailing forces. Push dynamics, including objective risk data, homeowners' risk perceptions, and damage experience, encourage support for mitigation. Pull dynamics, such as individualistic and conservative worldviews, and skepticism about climate change, generate opposition. At the margin, the pull dynamics appear to exert more force than push dynamics, creating only a weak basis of support that is not strong enough to overcome the status quo bias in a state that is cautious about regulatory measures. The concluding section offers suggestions for changing these dynamics.
Theory and conventional wisdom suggest that errors undermine the credibility of tornado warning systems and thus decrease the probability that individuals will comply (i.e., engage in protective action) when future warnings are issued. Unfortunately, empirical research on the influence of warning system accuracy on public responses to tornado warnings is incomplete and inconclusive. This study adds to existing research by analyzing two sets of relationships. First, we assess the relationship between perceptions of accuracy, credibility, and warning response. Using data collected via a large regional survey, we find that trust in the National Weather Service (NWS; the agency responsible for issuing tornado warnings) increases the likelihood that an individual will opt for protective action when responding to a hypothetical warning. More importantly, we find that subjective perceptions of warning system accuracy are, as theory suggests, systematically related to trust in the NWS and (by extension) stated responses to future warnings. The second half of the study matches survey data against NWS warning and event archives to investigate a critical follow‐up question—Why do some people perceive that their warning system is accurate, whereas others perceive that their system is error prone? We find that subjective perceptions are—in part—a function of objective experience, knowledge, and demographic characteristics. When considered in tandem, these findings support the proposition that errors influence perceptions about the accuracy of warning systems, which in turn impact the credibility that people assign to information provided by systems and, ultimately, public decisions about how to respond when warnings are issued.
Quantitative Research Methods for Political Science, Public Policy and Public Administration for Undergraduates: 1st Edition With Applications in Excel is an adaption of Quantitative Research Methods for Political Science, Public Policy and Public Administration (With Applications in R). The focus of this book is on using quantitative research methods to test hypotheses and build theory in political science, public policy and public administration. This new version is designed specifically for undergraduate courses. It omits large portions of the original text that focused on calculus and linear algebra, expands and reorganizes the content on the software system by shifting to Excel and includes guided study questions at the end of each chapter. ; https://dc.etsu.edu/etsu-oer/1003/thumbnail.jpg
The focus of this book is on using quantitative research methods to test hypotheses and build theory in political science, public policy and public administration. It is designed for advanced undergraduate courses, or introductory and intermediate graduate-level courses. The first part of the book introduces the scientific method, then covers research design, measurement, descriptive statistics, probability, inference, and basic measures of association. The second part of the book covers bivariate and multiple linear regression using the ordinary least squares, the calculus and matrix algebra that are necessary for understanding bivariate and multiple linear regression, the assumptions that underlie these methods, and then provides a short introduction to generalized linear models. The book fully embraces the open access and open source philosophies. The book is freely available in the SHAREOK repository; it is written in R Markdown files that are available in a public GitHub repository; it uses and teaches R and RStudio for data analysis, visualization and data management; and it uses publically available survey data (from the Meso-Scale Integrated Socio-geographic Network) to illustrate important concepts and methods. We encourage students to download the data, replicate the examples, and explore further! We also encourage instructors to download the R Markdown files and modify the text for use in different courses.
AbstractAs the United States is at historic lows of trust in government, various agencies are seeking to convince members of the public to take key protective actions and to support novel policy actions intended to reduce the spread of COVID‐19. This article assesses the status of trust in key organizations relevant to pandemic information based on a national survey of residents of the United States. First, the article illustrates the variations in trust placed in various agencies—local to global agencies and governmental and private sector organizations. Second, the analysis reveals considerable variances in trust in specific organizations based on party identification. Finally, the results indicate that trust is important as it is related to the intention to adopt personal protective actions and support for key public policies.
Survey-based contingent valuation (CV) techniques are commonly used to value the potential effects of a policy change when market-based valuation of those effects is not possible. The results of these analyses are often intended to inform policy decisions, which are made within the context of formal policymaking institutions. These institutions are typically designed to reduce the large number of potential options for addressing any given policy problem to a binary choice between the continuation of current policy and a single, specified alternative. In this research we develop an approach for conducting CV exercises in a manner consistent with the decision structure typically faced by policymakers. The data generated from this approach allow for an estimate of willingness to pay (WTP) for a defined policy alternative, relative to leaving policy unchanged, which we argue is of direct interest to policymakers. We illustrate our approach within the context of policy governing the storage of used nuclear fuel in the United States. We value the policy option of constructing an interim storage facility relative to continuation of current policy, wherein used nuclear fuel is stored on-site at or near commercial nuclear generating plants. We close the paper with a discussion of the implications for future research and the role of CV in the policymaking process.
Abstract National Weather Service (NWS) forecasters have many roles and responsibilities, including communication with core partners throughout the forecast and warning process to ensure that the information they are providing is relevant, understandable, and actionable. Although the NWS communicates to many groups, members of the emergency management community are among the most critical partners. However, little is known about the diverse population of emergency managers (EMs) and how they receive, process, and use forecast information. The Extreme Weather and Emergency Management Survey (WxEM) aims to fill this knowledge gap by 1) building a nationwide panel of EMs and 2) fielding routine surveys that include questions of relevance to NWS operations. The panel was built by creating a database with contact information from more than 4000 EMs across the country. An enrollment survey was sent to the list, and over 700 EMs agreed to participate in the project. Following enrollment, WxEM panelists receive surveys three–four times per year that address how EMs use NWS forecast information. These surveys cover a variety of subjects, with the goal of working with other researchers to develop surveys that address their research needs. By collaborating with other research groups to design short, focused surveys, the WxEM project will reduce the research burden on EMs and, at the same time, increase the quality and comparability of research data in the weather enterprise. The results will be shared with the NWS and the research community, and all data gathered from these surveys will be publicly available.
Significance Statement The Extreme Weather and Emergency Management Survey aims to better understand how emergency managers use National Weather Service (NWS) forecast information via a series of surveys regularly distributed to a panel of emergency managers across the country. By collaborating with other researchers, these surveys will cover broad topics and should limit the number of participation requests sent to emergency managers. Results will be distributed to participants, researchers, and NWS forecasters. All data will be publicly available.