Abstract:
We study how to improve coordination efficiency for multi-agent teams with heterogeneously experienced agents. In such a setting, experienced agents can transfer their knowledge to less experienced agents to accelerate their learning, while leveraging the students’ initial expertise to inform what knowledge to transfer. Inspired by this idea, this work specifically assumes one teacher agent in the team, and explores how it can efficiently utilize these knowledge priors to effectively improve the students’ training by performing experience-based action advising tailored to each student agent. We propose a novel teaching approach that leverages the teacher’s policy to identify a student’s pre-existing skill and subsequently assigns appropriate sub-tasks to each student based on a bandit formulation. As a result, student teammates are assigned to and advised through sub-tasks that enable them to leverage their skills and thus improve overall task convergence. We empirically show that our method outperforms existing teacher-student approaches that do not consider prior knowledge, and achieves faster convergence than teaming without knowledge transfer, demonstrating that tailored action advising accelerates team learning and improves overall performance, particularly as student agents may have accrued prior experience in particular sub-tasks.

Committee:
Katia Sycara (advisor)
Jiaoyang Li
Renos Zabounidis