A practical guide to multi-objective reinforcement learning and planning

Abstract

Real-world sequential decision-making tasks are generally complex, requiring trade-offs between multiple, often conflicting, objectives. Despite this, the majority of research in reinforcement learning and decision-theoretic planning either assumes only a single objective, or that multiple objectives can be adequately handled via a simple linear combination. Such approaches may oversimplify the underlying problem and hence produce suboptimal results. This paper serves as a guide to the application of multi-objective methods to difficult problems, and is aimed at researchers who are already familiar with single-objective reinforcement learning and planning methods who wish to adopt a multi-objective perspective on their research, as well as practitioners who encounter multi-objective decision problems in practice. It identifies the factors that may influence the nature of the desired solution, and illustrates by example how these influence the design of multi-objective decision-making systems for complex problems. ; Funding: Fonds voor Wetenschappelijk Onderzoek (FWO)FWO [1SA2820N]; Flemish GovernmentEuropean Commission; FWOFWO [iBOF/21/027]; National University of Ireland Galway Hardiman Scholarship; FAPERGSFundacao de Amparo a Ciencia e Tecnologia do Estado do Rio Grande do Sul (FAPERGS) [19/2551-0001277-2]; FAPESPFundacao de Amparo a Pesquisa do Estado de Sao Paulo (FAPESP) [2020/05165-1]; Swedish Governmental Agency for Innovation SystemsVinnova [NFFP7/2017-04885]; Wallenberg Artificial Intelligence, Autonomous Systems and Software Program (WASP) - Knut and Alice Wallenberg Foundation; LIFT - Dutch Research Council (NWO) [019.011]; 2017 Microsoft Research PhD Scholarship Program; 2020 Microsoft Research EMEA PhD Award