Maximum Causal Entropy Correlated Equilibria for Markov Games - Robotics Institute Carnegie Mellon University

Maximum Causal Entropy Correlated Equilibria for Markov Games

Brian D. Ziebart, J. Andrew (Drew) Bagnell, and Anind Dey
Conference Paper, Proceedings of 10th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS '11), pp. 207 - 214, May, 2011

Abstract

Motivated by a machine learning perspective---that game-theoretic equilibria constraints should serve as guidelines for predicting agents' strategies, we introduce maximum causal entropy correlated equilibria (MCECE), a novel solution concept for general-sum Markov games. In line with this perspective, a MCECE strategy profile is a uniquely-defined joint probability distribution over actions for each game state that minimizes the worst-case prediction of agents' actions under log-loss. Equivalently, it maximizes the worst-case growth rate for gambling on the sequences of agents' joint actions under uniform odds. We present a convex optimization technique for obtaining MCECE strategy profiles that resembles value iteration in finite-horizon games. We assess the predictive benefits of our approach by predicting the strategies generated by previously proposed correlated equilibria solution concepts, and compare against those previous approaches on that same prediction task.

BibTeX

@conference{Ziebart-2011-7254,
author = {Brian D. Ziebart and J. Andrew (Drew) Bagnell and Anind Dey},
title = {Maximum Causal Entropy Correlated Equilibria for Markov Games},
booktitle = {Proceedings of 10th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS '11)},
year = {2011},
month = {May},
pages = {207 - 214},
}