Suchergebnisse

Validity and generalization continue to be challenging aspects in designing and conducting case study evaluations, especially when the number of cases being studied is highly limited (even limited to a single case). To address the challenge, this article highlights current knowledge regarding the use of: (1) rival explanations, triangulation, and logic models in strengthening validity, and (2) analytic generalization and the role of theory in seeking to generalize from case studies. To ground the discussion, the article cites specific practices and examples from the existing literature as well as from the six preceding articles assembled in this special issue. Throughout, the article emphasizes that current knowledge may still be regarded as being at its early stage of development, still leaving room for more learning. The article concludes by pointing to three topics worthy of future methodological inquiry, including: (1) examining the connection between the way that initial evaluation questions are posed and the selection of the appropriate evaluation method in an ensuing evaluation, (2) the importance of operationally defining the 'complexity' of an intervention, and (3) raising awareness about case study evaluation methods more generally.

Rising standards for accurately inferring the impact of development projects has not been matched by equivalently rigorous procedures for guiding decisions about whether and how similar results might be expected elsewhere. These 'external validity' concerns are especially pressing for 'complex' development interventions, in which the explicit purpose is often to adapt projects to local contextual realities and where high quality implementation is paramount to success. A basic analytical framework is provided for assessing the external validity of complex development interventions. It argues for deploying case studies to better identify the conditions under which diverse outcomes are observed, focusing in particular on the salience of contextual idiosyncrasies, implementation capabilities and trajectories of change. Upholding the canonical methodological principle that questions should guide methods, not vice versa, is required if a truly rigorous basis for generalizing claims about likely impact across time, groups, contexts and scales of operation is to be discerned for different kinds of development interventions.

This article describes an attempt to synthesize results from four major multi-partner evaluations applying a realist approach. The article argues that synthesizing the results of these four major evaluations proved possible applying (part of) a realistic approach, and also reveals some of the limitations of the realist model as experienced by a very complex synthesis exercise. The researchers identify the concepts and approaches of the realist model that provided a supportive framework for synthesizing a very broad and diverse range of evidence, and which helped convert the findings into accessible outputs to support policy decisions. Lessons learned are extracted in the hope of supporting others facing similar tasks.

The social world is complex and emergent. Inquiry, directed towards establishing universal empirical regularities (i.e. nomothetic inquiry), cannot establish causality in such a world. We can never assign a causal effect to any intervention without assessing the whole context of that intervention. However, we can develop generalizable knowledge if we adopt research approaches that recognize both the implications of assigning causal powers to context (the essence of the realist take on evaluation) and the significance of human agency in relation to 'the social type of causal nexus'. There are literatures that can contribute to developing such knowledge. These include macro-political science's concern with the importance of temporal ordering in relation to outcomes; Ragin's set theoretic understanding of causal relations and his development of systematic comparison as a basis for explicating those relations through Qualitative Comparative Analysis (QCA); and the presentation of causal narratives as foundation for process tracing. Every complex social intervention has to be considered as a 'case'. Systematic comparison across cases allows us to generalize within limits – but this still means we can transfer knowledge beyond the unique ideographically described instance. We can never establish universal/nomothetic accounts of causality in complex systems by using variable-based methods such as Randomized Controlled Trials (RCTs). However, through careful comparison and exploration of complex contingent causation, we can begin to get a handle on what works where (in what context), when (in what temporal context), and in what order.

Partnerships between companies and non-governmental organizations that aim to incorporate smallholder farmers into value chains are increasingly being promoted as a way of pursuing development goals. This article investigates two case studies of such partnerships and the outcomes they achieved in order to refine the rationale underlying such interventions. In two case studies in Uganda and Rwanda, we documented the sequences of events within such partnership interventions, their context, and the intermediate outcomes, identified as the new rules and practices that generate institutional change. By portraying both the configuration of events within a partnership intervention and the contextual factors, these case studies reveal how the interventions produced outcomes that were situated in changing contexts, such as changes in market demand, government policy or business strategy. The research approach made it possible to disentangle partnership interventions and contextual processes, and to give participants a firmer idea of the potential and limitations of value chain partnerships to achieve developmental targets.

This article argues that Qualitative Comparative Analysis can be a useful method in case-based evaluations for two reasons: a) it is aimed at causal inference and explanation, leading to theory development; b) it is strong on external validity and generalization, allowing for theory testing and refinement. After a brief introduction to QCA, the specific type of causality handled by QCA is discussed. QCA is shown to offer improvements over Mill's methods by handling asymmetric and multiple-conjunctural causality in addition to counterfactual reasoning. It thereby allows the explicitly separate analysis of necessity and sufficiency, recognizing the relevance of causal packages as well as single causes and of multiple causal paths leading to the same outcome (equifinality). It is argued that QCA can generalize findings to a small, medium and large number of cases.