Understanding policy workers' policy innovation capacity: An exploratory and qualitative mixed methods evaluation study of a policy hackathon program in Prince Edward Island, Canada

Abstract

Executive Summary: Background: In 2018, the Government of PEI, Veterans Affairs Canada, Atlantic Canada Opportunities Agency and the Start-Up Zone brought together 49 individuals from the public and private sector to participate in a Policy Hackathon Program. A series of learning sessions were delivered while participants moved through a public policy case competition. This paper evaluates and studies this program and makes design recommendations for future policy hackathon programs. In the process, the paper draws attention to not only the relevance, performance and impact of the Program, but also larger discussions related to the unique attributes of the islandness of public policy, policy innovation, and austerity on an island. Methods: The evaluation study adopted a social-constructivist worldview, whereby the perceptions of participants and the interpretation of the researcher were used to understand the Program. A qualitative mixed methods design was employed which involved generating qualitative and quantitative data through a pre-program survey (N=48), post-program survey (N=38), interviews with a random sample of participants (N=6), and interviews with a purposive sample of key informants (N=2). Bason's (2014) design for policy theory and the OECD's (2017) core skills for public sector innovation framework were operationalized to understand the results in relation to theory and best practice. Quantitative and qualitative results were interpreted by the researcher to understand the Program and also to connect the results to public policy theory and constructs. Results: Relevance The Program responded to a need in PEI's policy environment. There was clear indication that participants believed that PEI needs new micro-and meso-level policy tools to develop public policy. Participants indicated that having opportunities to learn about policy innovation was important to them. The Program's emphasis on mentorship was relevant, given that participants believed that such multidisciplinary connections were important for policy development. Performance The Program performed well in terms of increasing participants' individual policy capacity as well as that of the entire group, meeting participants' expectations to receive valuable learning, and allowing participants to meaningfully connect with a broad range of individuals. The Program performed less optimally in the areas of providing participants with new policy tools, mentorship, and connecting with citizens. Impact Participants perceived the Program to have had a positive impact on their skill development in a wide range of areas and in increasing their comfort level with on-the-spot decision-making. Participants indicated that they would seek to integrate similar learning opportunities into their professional development plans in the future. Participants also reported that they believed the Program had a positive impact on the group's policy capacity and capacity to undertake innovative policy work. Policy Innovation The policy workers involved in the Program (i.e., participants) have cognitively established the positive connection between mentorship and innovation. Participants reported an increase in their confidence to apply human-centered design concepts. In terms of Bason's (2014) theory and the OECD's (2017) framework, the Program exposed participants to important policy innovation concepts. Given that participants indicated they thought that individuals who participated in the Program were better prepared to conduct innovative policy work in the future, it is assumed that the Program had a positive impact, to some degree, on increasing the policy innovation capacity of policy workers. Conclusion The study concludes by reiterating that the value of a policy hackathon program is as much related to process as new policies. In other words, in order for policy hackathon programs to be successful, they do PEI Policy Hackathon Program not necessarily need to result in the development of a new policy. Rather, as shown in this study, there can be positive impacts to participants' policy innovation capacity which can occur during the program. Policy hackathon programs therefore should not be judged entirely on the intervention's outputs. The study also concludes with a discussion in relation to the islandness of public policy, policy innovation, policy hackathons, and evaluation heuristics. Finally, the paper offers some thoughts on findings which pointed to the existence of austerity and the need for greater citizen-focus in public policy. ; Cameron, Bobby Thomas. (2018). Understanding policy workers' policy innovation capacity: An exploratory and qualitative mixed methods evaluation study of a policy hackathon program in Prince Edward Island, Canada. Research Paper, Centre for Policy Innovation and Public Engagement, Ryerson University, Toronto, ON, Canada.