Using Memory Models to Improve Adaptive Efficiency in Dynamic Problems

Abstract

Many real-world problems involve the coordination of multiple agents in dynamic environments, where characteristics of the problem being solved change over time. In such problems, adaptive, self-organizing agent approaches have been shown to provide very robust solutions. However, these approaches often require non-trivial amounts of time to respond to large environmental shifts. Considering this limitation, we observe that environmental changes in a given dynamic problem are generally not completely random; similar states in the environment tend to reappear over time. Memory is one way to leverage this past information and improve the adaptive efficiency of the agent system. In this paper, we explore the use of memory as a means of boosting the performance of self-organizing agents in solving dynamic coordination problems. We consider the specific problem of coordinating product flows in a factory that is subject to changing job mixes over time, which has been previously solved using a computational model of the task allocation behavior of wasps. We augment this base procedure with a number of memory systems, the most sophisticated of which exploit memory models inspired by estimation of distribution algorithms (EDAs) to manage computational cost. An experimental analysis is presented which demonstrates the advantage of using memory. Configurations using the EDA-inspired memory models are shown to substantially outperform configurations with more standard and infinite-sized memory models, and all are shown to improve the performance of the baseline task allocation procedure.