Evaluating social policies: Do methodological approaches determine the policy impact?

In: Social policy and administration, Band 55, Heft 5, S. 954-967

Abstract

AbstractOver the last decades, evidence‐based public policy has emerged as a concept and field of research, presenting methodological approaches from the natural sciences as the ideal. However, empirical and interdisciplinary research on the relationship between methodological approaches and policy impact remains scarce. The aim is to investigate whether the impact of evaluations on development of new ideas and policies depends on methodological approaches. Data are drawn from a survey among policy‐makers (n = 222) in six Norwegian agencies. Applying linear regression models, we analyse whether 'evaluations provide new ideas and perspectives' and if 'evaluations play a major role as a foundation for policy‐making', and if these aspects depend on the reason for conducting the evaluation and the type of data collection. The civil servants report strong belief in evaluations' impact on policy and development of new ideas; however, this is more or less independent of type of methodological approaches. Exceptions are evaluation using observational data, interviews, or case studies and evaluations conducted with the intention of obtaining new knowledge. The article brings new empirical knowledge regarding the way different methodological approaches contribute to the usefulness of evaluations. Supporting the criticism of a rigid understanding of the meaning of evidence.