Collaborating with the Machines: A Hybrid Method for Classifying Policy Documents

In: Policy studies journal: the journal of the Policy Studies Organization, Band 48, Heft 1, S. 184-206

Abstract

Governments produce vast and growing quantities of freely available text: laws, rules, budgets, press releases, and so forth. This information flood is facilitating important, growing research programs in policy and public administration. However, tightening research budgets and the information's vast scale forces political science and public policy to aspire to do more with less. Meeting this challenge means applied researchers must innovate. This article makes two contributions for practical text coding—the process of sorting government text into researcher‐defined coding schemes. First, we propose a method of combining human coding with automated computer classification for large data sets. Second, we present a well‐known algorithm for automated text classification, the Naïve Bayes classifier, and provide software for working with it. We argue and provide evidence that this method can help applied researchers using human coders to get more from their research budgets, and we demonstrate the method using classical examples from the study of policy agendas.