We examine the effect of clamping variables for approximate inference in undirected graphical models with pairwise relationships and discrete variables. For any number of variable labels, we demonstrate that clamping and summing approximate sub-partition functions can lead only to a decrease in the partition function estimate for TRW, and an increase for the naive mean field method, in each case guaranteeing an improvement in the approximation and bound. We next focus on binary variables, add the Bethe approximation to consideration and examine ways to choose good variables to clamp, introducing new methods. We show the importance of identifying highly frustrated cycles, and of checking the singleton entropy of a variable. We explore the value of our methods by empirical analysis and draw lessons to guide practitioners. ; NICTA is funded by the Australian Government through the Department of Communications and the Australian Research Council through the ICT Centre of Excellence Program. ; This is the author accepted manuscript. It is currently under an indefinite embargo pending publication by MIT Press.
In: Brundage , M , Avin , S , Wang , J , Belfield , H , Krueger , G , Hadfield , G , Khlaaf , H , Yang , J , Toner , H , Fong , R , Maharaj , T , Koh , P W , Hooker , S , Leung , J , Trask , A , Bluemke , E , Lebensold , J , O'Keefe , C , Koren , M , Ryffel , T , Rubinovitz , JB , Besiroglu , T , Carugati , F , Clark , J , Eckersley , P , Haas , S D , Johnson , M , Laurie , B , Ingerman , A , Krawczuk , I , Askell , A , Cammarota , R , Lohn , A , Krueger , D , Stix , C , Henderson , P , Graham , L , Prunkl , C , Martin , B , Seger , E , Zilberman , N , hÉigeartaigh , S Ó , Kroeger , F , Sastry , G , Kagan , R , Weller , A , Tse , B , Barnes , E , Dafoe , A , Scharre , P , Herbert-Voss , A , Rasser , M , Sodhani , S , Flynn , C , Gilbert , T K , Dyer , L , Khan , S , Bengio , Y & Anderljung , M 2020 , ' Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims ' , arXiv.org, e-Print Archive, Mathematics .
With the recent wave of progress in artificial intelligence (AI) has come a growing awareness of the large-scale impacts of AI systems, and recognition that existing regulations and norms in industry and academia are insufficient to ensure responsible AI development. In order for AI developers to earn trust from system users, customers, civil society, governments, and other stakeholders that they are building AI responsibly, they will need to make verifiable claims to which they can be held accountable. Those outside of a given organization also need effective means of scrutinizing such claims. This report suggests various steps that different stakeholders can take to improve the verifiability of claims made about AI systems and their associated development processes, with a focus on providing evidence about the safety, security, fairness, and privacy protection of AI systems. We analyze ten mechanisms for this purpose--spanning institutions, software, and hardware--and make recommendations aimed at implementing, exploring, or improving those mechanisms.