Suchergebnisse

AbstractThe application of mixed methods under the framework of theory‐driven evaluations can minimize the potential tension and conflict of mixing qualitative and quantitative methods, as well as compensate for each method's weaknesses. Mixed methods should not be applied indiscriminantly, however, but rather contingently under particular conditions as described in this chapter.

AbstractIn designing theory‐driven evaluations, it is important to understand the normative and causative aspects of program theory, different theory‐construction approaches, the typology of theory‐driven evaluations, and analytical techniques for carrying out the evaluations.

This article reviews current major methods of ensuring validity in evaluation research. In reviewing rationales advanced for the primacy of the experimental paradigm, I argue that the use of randomized experiments in program evaluation may not be as powerful as its advocates claim. Furthermore, the rigidity of the randomized experimental design may emphasize internal validity at the expense of other types of validity. Especially important, I argue, is that evaluations based on experimental paradigms tend not to yield information useful to policy makers.
Various quasi-experiments have been considered inferior to randomized experiments in internal validity by Campbell and Stanley. However, due to later developments of sophisticated methods as ARIMA models and modelling selection bias, the gap between randomized experiments and quasi-experiments in dealing with internal validity has narrowed. Furthermore, quasi-experiments also have other advantages such as unobtrusiveness and greater applicability to the evaluation of human services programs.
Current approaches to validity tend to be a single-validity oriented approach; that is, they tend to achieve one type of validity at the expense of others. I propose a theory-driven approach that attempts to provide a more balanced view. The emphasis of this approach is to stress the incorporation of formal models or theory into program evaluation, thereby articulating clearly the identified potential threats to validity in a research. The theory-driven approach can and should deal with internal, external, construct, and statistical conclusion validity simultaneously.

This collection of essays edited by Huey-tsyh Chen and Peter H. Rossi underscores the importance of theory in program evaluation and analyzes the nature and implications of theory-driven evaluations. Scholars from different disciplines cover a wide range of issues, such as functions of program theory, the interface between theory and methods, strategies for formulating theory, cost and resource constraints, different types of theory-driven evaluations, and the future implications of such evaluations. This broad-based analysis is designed for interdisciplinary audiences in all the social sciences, including political scientists, sociologists, and public administrators. The policy study analysis points to four types of theory that guide evaluations and describe theory-driven metaevaluation. Five approaches are used to analyze research strategies and methods. Three applications offer insights into programs and policies and their implementation - multigoal theory-driven approaches, programs for the homeless, and children's mental health services. Five chapters examine applications in impact assessment and cost-benefit analysis - the importance of theory in selection modeling, the impact of plant closings, research-based theory for educational planning and evaluation, using meta-analysis in evaluation, and theory-driven approaches to cost-benefit analysis.