Contents -- Foreword -- Annotated Bibliography of Meta-Analytic Books and Journal Issues -- 1. The Meta-Analytic Perspective -- 2. Explanation in Meta-Analysis -- 3. Effects of Psychoeducational Care with Adult Surgical Patients: A Theory-Probing Meta Analysis of Intervention Studies / Elizabeth C. Devine -- 4. Juvenile Delinquency Treatment: A Meta-Analystic Inquiry into the Variability of Effects / Mark W. Lipsey -- 5. Do Family and Marital Psychotherapies Change What People Do? A Meta-Analysis of Behavioral Outcomes / William R. Shadish, Jr
Zugriffsoptionen:
Die folgenden Links führen aus den jeweiligen lokalen Bibliotheken zum Volltext:
These remarks deal with vexing issues of method choice that are currently bedeviling the educational research community as it seeks to ground educational policy decisions in better evidence. Of particular importance are debates in American educational circles about the need for experiments versus multi-method studies, and it is on this issue that we focus here. The remarks are organized around the exposition of a limited number of basic points. The emphasis is on clarity of presentation rather than mellifluous prose.
In experimentally designed research, many good reasons exist for assigning groups or clusters to treatments rather than individuals. This article discusses them. But cluster-level designs face some unique or exacerbated challenges. The article identifies them and offers some principles about them. One emphasizes how statistical power and sample size estimation depend on intraclass correlations, particularly after conditioning on the use of cluster-level covariates. Another stresses assigning experimental units at the lowest level of aggregation possible, provided this does not subtly change the research question. A third emphasizes the utility of minimizing and measuring interunit communication, though neither is easy to achieve. A fourth advises against experiments that are totally black box and so leave program implementation and process unstudied, though such study often makes the research process more salient. The last principle involves the utility of describing treatment heterogeneity and estimating its consequences, though causal conclusions about the heterogeneity will be less well warranted compared to conclusions about the intended treatment, every experiment's major focus.
In experimentally designed research, many good reasons exist for assigning groups or clusters to treatments rather than individuals. This article identifies them. But cluster-level designs face some unique or exacerbated challenges. The article identifies them & offers some principles about them. One principle emphasizes how statistical power & sample size estimation depend on interclass correlations, particularly after conditioning on the use of cluster-level covariates. Another stresses assigning experimental units at the lowest level of aggregation possible, provided this does not subtly change the research question. A third emphasizes the utility of minimizing & measuring interunit communication, though neither is easy to achieve. A fourth advises against experiments that are totally black-box & so leave program implementation & process unstudied, though such study often makes the research process more salient. The last principle involves the utility of describing treatment heterogeneity & estimating its consequences, though causal conclusions about the heterogeneity will he less well warranted compared to conclusions about the intended treatment, every experiment's major focus. 23 References. [Reprinted by permission of Sage Publications Inc., copyright 2005 The American Academy of Political and Social Science.]
This article analyzes the reasons that have been adduced within the community of educational evaluators for not doing randomized experiments. The objections vary in cogency. Those that have most substance are not insurmountable, however, and strategies are mentioned for dealing with them. However, the objections are serious enough, and the remedies partial enough, that it seems hardly warranted to call experiments the "gold standard" of causal inference. Yet even if they are not perfect in research practice, this article shows how they are logically and empirically superior to all currently known alternatives. The article particularly addresses the objection that school personnel will not accept experiments. It shows that hundreds of them have been done there by researchers with backgrounds in psychology and public health who study the prevention of unhealthy behaviors. But experiments are much rarer among researchers trained in education who study changing academic performance. Reasons are adduced for this difference in academic culture within school-based research.
This article analyzes the reasons that have been adduced within the community of educational evaluators for not doing randomized experiments. The objections vary in cogency. Those that have most substance are not insurmountable, however, & strategies are mentioned for dealing with them. However, the objections are serious enough, & the remedies partial enough, that it seems hardly warranted to call experiments the "gold standard" of causal inference. Yet even if they are not perfect in research practice, this article shows how they are logically & empirically superior to all currently known alternatives. The article particularly addresses the objection that school personnel will not accept experiments. It shows that hundreds of them have been done there by researchers with backgrounds in psychology & public health who study the prevention of unhealthy behaviors. But experiments are much rarer among researchers trained in education who study changing academic performance. Reasons are adduced for this difference in academic culture within school-based research. [Copyright 2003 Sage Publications, Inc.]
AbstractThis chapter offers a critical commentary on theory‐based evaluation, stressing its utility as a method of program planning and as an adjunct to experiments but rejecting it as an alternative to experiments.
The basic regression discontinuity design (RDD) has less statistical power than a randomized control trial (RCT) with the same sample size. Adding a no-treatment comparison function to the basic RDD creates a comparative RDD (CRD); and when this function comes from the pretest value of the study outcome, a CRD-Pre design results. We use a within-study comparison (WSC) to examine the power of CRD-Pre relative to both basic RDD and RCT. We first build the theoretical foundation for power in CRD-Pre, then derive the relevant variance formulae, and finally compare them to the theoretical RCT variance. We conclude from this theoretical part of this article that (1) CRD-Pre's power gain depends on the partial correlation between the pretest and posttest measures after conditioning on the assignment variable, (2) CRD-Pre is less responsive than basic RDD to how the assignment variable is distributed and where the cutoff is located, and (3) under a variety of conditions, the efficiency of CRD-Pre is very close to that of the RCT. Data from the National Head Start Impact Study are then used to construct RCT, RDD, and CRD-Pre designs and to compare their power. The empirical results indicate (1) a high level of correspondence between the predicted and obtained power results for RDD and CRD-Pre relative to the RCT, and (2) power levels in CRD-Pre and RCT that are very close. The study is unique among WSCs for its focus on the correspondence between RCT and observational study standard errors rather than means.
