[Submitted on 8 May 2020]

Download PDF

Abstract: We study active preference learning as a framework for intuitively specifying
the behaviour of autonomous robots. In active preference learning, a user
chooses the preferred behaviour from a set of alternatives, from which the
robot learns the user’s preferences, modeled as a parameterized cost function.
Previous approaches present users with alternatives that minimize the
uncertainty over the parameters of the cost function. However, different
parameters might lead to the same optimal behaviour; as a consequence the
solution space is more structured than the parameter space. We exploit this by
proposing a query selection that greedily reduces the maximum error ratio over
the solution space. In simulations we demonstrate that the proposed approach
outperforms other state of the art techniques in both learning efficiency and
ease of queries for the user. Finally, we show that evaluating the learning
based on the similarities of solutions instead of the similarities of weights
allows for better predictions for different scenarios.