Suchergebnisse

AbstractEvolutionary Evaluation, Relational Developmental Systems Theory, and Systems Thinking are three broad foundational theoretical perspectives that characterize the shifting paradigm in contemporary evaluation. They are central to the development of Relational Systems Evaluation (RSE). From Evolutionary Evaluation, we focus on the variation and selection role of evaluation, as well as the notions of ontogeny, phylogeny, symbiosis, and co‐evolution. From Relational Developmental Systems Theory, we highlight how mutually beneficial bidirectional relations between a program and its environmental context can promote adaptive regulations. From Systems Thinking, we emphasize part–whole relationships, static and dynamic processes, the related concepts of scale and boundaries, multiple perspectives, and causal pathways. We propose that these theoretical elements—all of which are operationalized through RSE—can be transformative in our thinking about the next evolutionary stage for evaluation.

Using a mixed methods approach, the connection between Intentional Self Regulation (ISR) and feelings about the chances of achieving future aspirations among 94 Scottish youth (56% female) was examined. Regression analyses demonstrated ISR, as measured by the SOC 9-item scale, was predictive of youths' feelings about their chances of achieving future aspirations. Qualitative data collected from 26 interviews suggested the use of ISR skills among adolescents, even when those skills were not evident quantitatively. Results indicated the need to employ mixed methodologies when conducting research on ISR with young adolescents, and suggest the need for further testing and development of measures for both ISR and beliefs about the chances of achieving future aspirations. Potential programmatic implications for youth development and character education programs are also discussed.This is a correction to the original article. For information about the changes made, please see the erratum https://doi.org/10.5195/jyd.2017.500.

Character education programs are mission-aligned with the positive youth development (PYD) perspective, which, in research with American youth suggests that intentional self-regulation (ISR) develops through mutually beneficial interactions between youth and their environment. Cross-cultural studies of Western youth suggest an adolescence-specific ISR process may exist. We begin to extend this work to assess the relationship between ISR and positive development in young Scottish adolescents (approximately 7th grade, N = 82; 50% male), a previously unexamined group. ISR was correlated with the Five Cs of PYD and OLS regression analyses demonstrated that ISR predicted two of the Five Cs: Character and Connection. We discuss implications for youth development programs of the findings that ISR and some indices of PYD were linked in a sample of Scottish youth.

A Positive Youth Development (PYD) perspective emphasizes adolescents' potential for healthy, successful development. Researchers have introduced a variety of constructs that may promote PYD. For example, Intentional Self‐Regulation (ISR) enables youth to seek out and use resources in the environment; purpose can help youth navigate and overcome obstacles. We examine connections between ISR, purpose and two PYD outcomes: confidence and connection. Data come from 763 Scottish youth (ages 12–14). We estimated fixed‐effects regression models predicting confidence and connection from both ISR and purpose, confirming outcomes were independently associated with one or both predictors. Our findings are especially salient to the field of youth purpose, linking purpose with specific PYD outcomes.

AbstractIn this paper, we present a comprehensive integrative approach to evaluation planning based on Relational Systems Evaluation (RSE). We describe the major implementation approaches and the tools and resources that support evaluation planning in RSE embodied in the Systems Evaluation Protocol (SEP), a step‐by‐step guide to the tasks that every evaluation should address. The SEP provides both a conceptual framework for thinking about evaluation and a set of specific methods and tools that expand awareness of the influences of multiple parts of complex systems. The SEP is implemented through a partnership approach, joining evaluators and program professionals in an iterative, phased approach to evaluation planning, which also often emphasizes evaluation capacity building (ECB) as an intentional goal. This paper provides a practice‐oriented overview of the specific steps comprising the three stages outlined in the SEP: preparation, model development, and evaluation plan development.

AbstractIn this paper, we present six case studies that illustrate the application of the Systems Evaluation Protocol (SEP) in different real‐world implementation conditions. The SEP is a step‐by‐step guide for how to implement Relational Systems Evaluation (RSE), accounting for the complex factors inherent in the larger systems within which a given program is embedded. We discuss the specific SEP steps used and products developed in each case study project, including an emphasis on how decisions were made to include particular SEP elements in different contexts. The six cases differ in terms of the scale of the programs involved, the nature of the SEP delivery or facilitation process, and the balance among multiple desired outcomes reflecting differential emphases on evaluation capacity building. As such, the cases reported here demonstrate that RSE, as implemented using the SEP, is a widely applicable approach to evaluation and ECB.

AbstractIn this paper, we introduce Relational Systems Evaluation (RSE), the focal topic of this volume. RSE is a framework for program planning and evaluation that is theoretically grounded, empirically tested and focused on building evaluation capacity. Theoretically, RSE is rooted in an evolutionary approach to program development and evaluation informed by systems thinking. The application of RSE involves a collaborative partnership approach pairing evaluators or researchers with program practitioners, the use of collaborative modeling and planning tools, and an emphasis on nurturing an Evaluative Thinking mindset. The goal of RSE is to produce more thoughtful and useful evaluations and, ultimately, better programs.

AbstractIn this paper, we present empirical findings from a mixed‐methods study of an Evaluation Capacity Building (ECB) initiative called the Partnerships for Advancing Character program Evaluation (PACE) Project that applied the Relational Systems Evaluation (RSE) approach. We focused especially on how participation in this partnership‐based ECB program is associated with changes in attitudes and behaviors related to evaluation capacity and Evaluative Thinking. We used a repeated measures design with semi‐structured interviews and surveys to examine two research questions: (1) Do Program Professionals participating in PACE adopt Evaluative Thinking behaviors? And (2) Do Program Professionals participating in PACE demonstrate increased evaluation capacity, including improved attitudes about evaluation? We found that PACE successfully increased Program Professional's behaviors, attitudes, and capacity; in particular, participants overwhelmingly noted that the emphasis on Evaluative Thinking changed the way they think about and approach evaluation, such that evaluation is now more well‐integrated into their practice.

AbstractThis paper presents the results of a study of program participants' responses to an innovative project called the Partnerships for Advancing Character program Evaluation (PACE), which operationalized a Relational Systems Evaluation (RSE) approach to Evaluation Capacity Building (ECB). We examine which tools and concepts from PACE resonated with participants and were sustained, and why. Analysis of participant interviews yielded sixteen distinct value propositions that they associated with the PACE training. Importantly, a number of them extend beyond evaluation to include contributions to other aspects of program professionals' work responsibilities and work life. We explore the relationships between the most widely valued PACE tools and concepts and the value propositions they were associated with. These discoveries about the potential value propositions for the RSE approach to ECB hold promise for efforts to increase investment in evaluation capacity, and to strengthen the transfer of learning that is needed for evaluation to be embedded and sustained in organizations.