Suchergebnisse

This article presents an argument for how to generalize from a single case study evaluation of a social programme or policy, focusing on two essential elements of case study – context and particularity. Taking an interpretivist perspective and drawing on artistic and humanistic ways of understanding, it examines both different ways of generalizing from the case that retain a connection with the context in which they first arose and how we generalize by direct encounter with the particular. This argument is not new. It has its origins in earlier centuries and, in contemporary evaluation, dating back to the 1970s. However, in a political climate that privileges evaluation approaches stemming from large sample studies and experimental designs, it seems timely to restate the value of generalizing from the single case. Grounded in the reality of programme experience and retaining that connection with context and particulars in the case facilitates the use of evaluation knowledge.

This article considers how evaluation practice can improve so that the support offered to decision makers is a better reflection of the reality being evaluated. The article examines a number of evaluations of the same intervention, Regional Selective Assistance administered by Scottish Enterprise, one of Scotland's two public sector economic development agencies. The assumptions underpinning the evaluations have changed over time. The earlier ones assumed that it was possible to evaluate the intervention in isolation: ignoring the factors that Realistic Evaluation highlights as affecting impacts: Context, Mechanisms and the Outcome Configuration. Later evaluations implicitly acknowledged these factors. However, the evaluations produced different results so that, rather than triangulation resulting in a degree of consensus, policy makers were presented with different views as to the effectiveness of the same intervention. The conclusion is that both evaluators and policy makers need to explicitly accept the Realistic Evaluation agenda and realize that causality is far more complicated than is often assumed. This would be aided if there was more transparency in evaluation methodologies and the use of different approaches to evaluate the same intervention so that any findings were triangulated.

The evaluation discipline has long been put at the service of liberal democratic values. But contemporary evaluation practice is threatened by vested interests, western democracy is under stress and internationalization has propelled evaluation towards illiberal and patrimonial states. What is to be done in contexts where democracy is absent and/or evaluation has been captured by powerful interests whether globally, within countries or within organizations? Are existing democratic evaluation approaches still relevant? Is it time to try something new? This article reviews the evidence and recommends adoption of a progressive evaluation model designed to complement, update and renew existing democratic and social justice evaluation approaches.

The contribution discusses the risks and benefits of the push for evidence-based decision making. More particularly, it focuses on the risks of a results-focused approach; the special risks of the preferential treatment of RCTs; and the benefits of an evidence-focused approach.

As government administrative data sets are increasingly made available for new (non-administrative) purposes, there is a need to improve access to such resources for voluntary and community organizations, social enterprises and private businesses for statistical analysis and evaluation purposes. The Justice Data Lab set up by the Ministry of Justice in the UK presents an innovative case of how administrative data can be linked to other data held by organizations delivering public services. The establishment of a unit within a secure setting holding evaluation and statistical expertise has enabled providers of programmes aimed at reducing re-offending to obtain evidence on how the impact of their interventions differs from that of a matched comparison group. This article explores the development of the Justice Data Lab, the methodological and other challenges faced, and the experiences of user organizations. The article draws out implications for future development of Data Labs and the use of administrative data for the evaluation of public services.

A strategy of 'radical incrementalism' is herein advocated as a useful approach to policy making which makes proper use of evaluation. It is argued that small, incremental changes, supported by small-scale and tightly focused evaluations, are under-utilised as an approach to cost effective policy improvement.