Suchergebnisse

Little information is available about how formal, social science theory has been used in outcome evaluations. This gap exists in debates about what role theory has in evaluation. To help address this, we need to understand variation in formal theory use in real-world practice. This research applied systematic qualitative coding to identify and classify patterns of theory use, followed by qualitative comparative analysis. The terms describing theory are defined and formal theory is differentiated from other kinds of theory, such as program theory. The study found instances of borrowing and repurposing theoretical material in a cohesive sample of 17 outcome evaluation reports covering programs addressing complex social problems, drawn from cross-cultural contexts. Theory was mostly used for post hoc explanation and less often used upfront in framing, design and conduct of evaluation. The concrete approach in the literature of applying formal theory to measure and explain causal pathways to behavioural outcomes was found, as was layered use of a range of theoretical material. This article offers insights that may assist evaluators to undertaken more sophisticated and reflective approaches to using theory. Thinking about ways we use theory could form part of the tool-kit of techniques available to evaluation practice.

This paper offers reflections on our experience and learning arising from implementing a study design that used evaluation research to pursue multiple benefits. The Community Safety and Wellbeing Study adopted a mixed methods approach, referred to as a 'both-ways' (or two-ways) research model, that addressed decision maker's needs and heard the people's voice. The study design was inspired by a both-ways learning model and attempted to address both needs together. The aim of the study was to involve local people in communities and encourage them to share their views about changes in community safety. Through systematic research it provided a voice for Indigenous Australians affected by the Northern Territory Emergency Response (NTER), often called 'the intervention'. Regardless of views about the intervention, this paper aims to share the lessons learned from conducting this study. The mixed method approach involved a community based standardised survey and qualitative data collection techniques. The study was undertaken in a representative sample of 17 NTER communities with over 1300 local residents, around five percent of the relevant population. Over 60 Indigenous people were employed in conducting the study; the majority lived in or had strong ties with remote communities in the study and around 10 had significant experience with social research projects. The research had many objectives in addition to providing evidence on outcomes on a multi-faceted and sometimes controversial government policy. Ethically the research had to have benefit for the people and communities involved. In addition, the community had to be able to see there were benefits from the research for them, not just for government. This paper documents how these objectives were achieved in relation to the methodology, content, data collection and reporting aspects of the research, and discusses what worked and what could be done differently in the future.

This paper demonstrates how the voices of people affected by a policy can contribute to evaluation quality and utilisation. In this participatory mixed-methods approach, Indigenous Australians in remote parts of the Northern Territory of Australia were involved in evaluation of an intervention that significantly affected their daily lives. The evaluation was inspired by the ideas and values demonstrated in a both-ways learning model. This mixed-methods approach combines a people-centered participatory methodology with a more standardised policy-focused application of survey methods to contribute to policy development and local planning. The aim of the study was to hear from local people in communities and encourage them to share their views about changes in community safety. It provided a voice for Indigenous Australians affected by the Northern Territory Emergency Response (NTER), which sought to protect children in 73 Indigenous communities across the Northern Territory. The aim of the NTER was to create safer communities and to improve health, nutrition, educational outcomes, service delivery and engagement with Indigenous people. Regardless of views about the NTER, a clear gap in the evidence base for evaluation was to understand if people felt safer after the intervention. Increased police, safe houses, night patrols, health services, family support services and management of income support payments were introduced, but what did Indigenous people living in remote communities think? What was their experience and what will happen to the information they may choose to share? Adopting a mixed-methods approach that included a large-scale survey and qualitative research conducted with local Indigenous residents of remote communities strengthened evaluative findings, particularly for policy makers and community residents. A participatory research cycle was used to engage local people and return results in an ethical feedback process that aimed to build capacity on the ground and link into future planning. We sought to promote the role of evaluation in remote Northern Territory communities as a worthwhile way for local people to share their voice, stories and experiences with the policy makers whose decisions can so significantly affect their lives.

AbstractThe strong presence of Indigenous Australians at the 2014 Australasian Evaluation Society (AES) conference highlights expansion in the ways Indigenous people are working as evaluation practitioners and collaborating in evaluation. This may reflect changing practice, moving from the involvement of Indigenous Australians solely as the users of programs to their more diverse and active engagement in the conduct of evaluations. However, it is not clear how far this shift extends to Indigenous‐led shaping of evaluation questions or framing the interpretation of findings. In this chapter, Indigenous Australian coauthors who are practicing evaluators explore dimensions of Indigenous participation. Themes covering knowledge and engagement processes and principles emerged from the conference, demonstrating how "ways of knowing and doing" make Indigenous evaluation much more than about method. The concept of an "adaptive shared space" is presented as a venue for growing respectful collaboration. Developing deliberative learning among Indigenous and non‐indigenous people, commissioners, and practitioners, offers a way to bring deeper meaning to evaluation practice.