Active Comparison Based Learning Incorporating User Uncertainty and Noise

Abstract

Our goal is to facilitate better human-robot collaboration by enabling robots to learn our preferences. To learn preferences, robots need to interact with users. We propose using comparison based learning, which learns preferences by asking a user to compare several alternatives. To minimize user burden, we use active learning. A challenge of comparison based learning is that it can be difficult for a user to say which item they prefer. Forcing the user to provide a preference in these cases leads to noisy responses, which increases the number of needed queries. Our key insight is that users can identify difficult comparisons and that we can use this information to learning their uncertainty. We present CLAUS (Comparison Learning Algorithm for Uncertain Situations), which model uncertainty and uses it to select and process comparison queries. Our user study suggests that CLAUS uses fewer queries than algorithms which force users to choose, while maintaining nearly the same accuracy.