Event Location: NSH 1305
Bio: Victor Lesser received the Ph.D. degree in computer science from Stanford University, Stanford, CA, 1973. He is a Distinguished Professor of Computer Science and Director of the Multi-Agent Systems Laboratory at the University of Massachusetts. His major research focus is on the control and organization of complex AI systems. He has pioneered work in the development of the blackboard architecture and its control structure, approximate processing for use in control and real-time AI, and a wide variety of techniques for the coordination of and negotiation among multiple agents. He was the system architect for the first fully developed blackboard architecture (HEARSAY-II), when he was a research computer scientist at CMU from 1972 through 1976. He has also made contributions in the areas of machine learning, signal understanding, diagnostics, plan recognition, and computer-supported cooperative work. He has worked in application areas such as sensor networks for vehicle tracking and weather monitoring, speech and sound understanding, information gathering on the internet, peer-to-peer information retrieval, intelligent user interfaces, distributed task allocation and scheduling, and virtual agent enterprises.

Professor Lesser’s research accomplishments have been recognized by many major awards over the years. He recently received the prestigious IJCAI-09 Award for Research Excellence. He is also a Founding Fellow of AAAI and an IEEE Fellow. He was General Chair of the first international conference on Multi-Agent Systems (ICMAS) in 1995, and Founding President of the International Foundation of Autonomous Agents and Multi-Agent Systems (IFAAMAS). In 2007, to honor his contributions to the field of multi-agent systems, IFAAMAS established the “Victor Lesser Distinguished Dissertation Award.” He also received a Special Recognition Award for his foundational research in generalized coordination technologies from the Information Processing Technology Office at DARPA.

Abstract: Multi-agent reinforcement learning (MARL) provides an attractive, scalable, and approximate approach for agents to learn coordination policies and adapt their behavior to the dynamics of the uncertain and evolving environment. However, for most large-scale applications involving hundreds of agents, current MARL techniques are inadequate. MARL may converge slowly, converge to inferior equilibria, or even diverge in realistic settings. There are no known approaches that guarantee convergence without very constraining assumptions about the learning environment and the knowledge at each agent. These assumptions do not hold in most realistic applications. In this lecture, I will introduce a new paradigm that builds upon conventional MARL techniques for scaling MARL to large agent networks. This paradigm exploits low-overhead, periodic, non-local multi-level supervisory control to coordinate and guide the agents’ learning process. It introduces a more global view into the learning process of individual agents without incurring significant overhead and exploding their policy spaces; it coordinates the learning behavior of tightly coupled agents by constraining their learning processes while still leaving agents to react autonomously to local reward signals. This coordination results in both speeding up and increasing the likelihood of network convergence by reducing the occurrence of oscillatory behavior among agents learning in a non-stationary environment and focusing agents’ exploration. I will also discuss how this supervisory structure can be learned as part of the overall agent learning process. This is joint work with Chongjie Zhang and Dr. Shereif Abdallah.