Where?:https://cmu.zoom.us/j/96355036481?pwd=OCtBeWZpMnZsZzFlRkJWc2dkZW5qUT09

Title: Discrete Communication Learning via Backpropagation for Distributed Computing on Bandwidth-Limited Communication Networks

Abstract:

Efficient inter-agent communication is an important requirement for both cooperative multi-agent robotics tasks, as well as distributed computing.  In both of these domains, the rate at which information can be transferred between robots or computing nodes is often much slower than internal information transfer rates.  Techniques exist for learning communication protocols in such bandwidth-limited channels, however these approaches tend to converge very slowly.  To address this problem, we introduce a discrete communication learning approach, which utilizes a stochastic message encoding/decoding procedure to make discrete communication channels mathematically equivalent to an analog channel with additive noise.  Gradients can thus be backpropagated through the communication channel, enabling rapid, efficient and robust communication learning.  We additionally introduce a variable-length message code that provides agents with a means to modulate the number of bits they send to their neighbors, and introduce a novel message-length penalty objective that encourages agents to send short messages when possible.  When combined, these techniques enable agents to minimize their communication requirements, while still effectively solving their task.  We evaluate these contributions on both a reinforcement learning task involving partially observable robot navigation, as well as a supervised learning task with graph neural networks to model distributed computing.  We find that in both tasks, our discrete differentiable communication approach enables communication learning with convergence rates comparable to approaches that allow the transmission of real-valued messages.  Additionally, we find that in the supervised learning task, our approach for encouraging limited communication enables comparable levels of validation accuracy to be achieved, with up to a of factor of 37.5 fewer bits exchanged, compared to an approach in which nodes communicate 32-bit precision vectors, indicating that our approach provides an effective means to rapidly learn efficient communications in a distributed computing setting.

Committee:

Howie Choset (Advisor)

Jeff Schneider

Jaskaran Grover