Search results

Organizations involved in the development of innovative social programs are coming under increasing pressure to establish structures through which the users, targets, or beneficiaries of program activities can be involved in program planning and implementation. The dynamic that this sets up between the program and its wider environment, particularly in areas in which "new social movements" are operating, can be quite challenging. This article seeks to examine these dynamics, particularly in the context of program evaluation. It is argued that an engaged evaluation approach (drawing on an action research orientation) has particular strengths in such a situation, although the limitations and challenges of such an approach are also outlined. It may, for example, require careful management in terms of the dynamics of the evaluation team. The discussion is illustrated by examples drawn from the evaluation of a European program for disabled people in which the author as member of a research team from The Tavistock Institute was involved.

Central government guidance seeks to ensure and enhance the quality of practice and decision-making across – and sometimes beyond – government. The Magenta Book, published by HM Treasury, is the key UK Government resource on policy evaluation, setting out central government guidance on how to evaluate policies, projects and programmes. The UK Centre for the Evaluation of Complexity Across the Nexus was invited to contribute its expertise to the UK Government's 2020 update of the Magenta Book by developing an accompanying guide on policy evaluation and 'complexity'. A small multidisciplinary team worked together to produce a set of guidance, going through multiple stages of work and drawing on a variety of sources including academic and practitioner literature and experts and stakeholders in the fields of evaluation, policy and complexity. It also drew on Centre for the Evaluation of Complexity Across the Nexus' own work developing and testing evaluation methods for dealing with complexity in evaluation. The resulting Magenta Book 2020 Supplementary Guide: Handling Complexity in Policy Evaluation explores the implications of complexity for policy and evaluation and how evaluation can help to navigate complexity. This article, designed primarily for practitioners who might be interested in this guidance and how it was developed, describes the processes involved, particularly related to the interdisciplinary dialogue and consultation with other key stakeholders that this involved. It also briefly outlines the content and key messages in the guidance, with reflections on the experiences of the authors in developing the guide – including the challenges and insights that arose during the process, particularly around the challenges of communicating complexity to a broad audience of readers.

Theory of Change diagrams are commonly used within evaluation. Due to their popularity and flexibility, Theories of Change can vary greatly, from the nuanced and nested, through to simplified and linear. We present a methodology for building genuinely holistic, complexity-appropriate, system-based Theory of Change diagrams, using Participatory Systems Mapping as a starting point. Participatory System Maps provide a general-purpose resource that can be used in many ways; however, knowing how to turn their complex view of a system into something actionable for evaluation purposes is difficult. The methodology outlined in this article gives this starting point and plots a path through from systems mapping to a Theory of Change evaluators can use. It allows evaluators to develop practical Theories of Change that take into account feedbacks, wider context and potential negative or unexpected outcomes. We use the example of the energy trilemma map presented elsewhere in this special issue to demonstrate.

The value of complexity science and related approaches in policy evaluation have been widely discussed over the last 20 years, not least in this journal. We are now at a crossroads; this Special Issue argues that the use of complexity science in evaluation could deepen and broaden rendering evaluations more practical and rigorous. The risk is that the drive to better evaluate policies from a complexity perspective could falter. This special issue is the culmination of 4 years' work at this crossroads in the UK Centre for the Evaluation of Complexity Across the Nexus. It includes two papers which consider the cultural and organisational operating context for the use of complexity in evaluation and four methodological papers on developments and applications. Together, with a strong input from practitioners, these papers aim to make complexity actionable and expand the use of complexity ideas in evaluation and policy practice.