A Probabilistic Approach for the Adaptive Integration of Multiple Visual Cues Using an Agent Framework

Abstract

Most of current machine vision systems for dynamic state estimation suffer from a lack of flexibility to account for the high variability of unstructured environments. As the state of the world evolves, the potential knowledge provided by different visual attributes can change, breaking the initial assumptions of a non-adaptive vision system. This thesis develops a new comprehensive computational framework for the adaptive integration of information from different visual algorithms. This framework takes advantage of the richness of visual information by adaptively considering a variety of visual properties such as color, depth, motion, and shape. Using a probabilistic approach and uncertainty metrics, the resulting framework makes appropriate decisions about the most relevant visual attributes to consider. The framework is based on an agent paradigm. Each visual algorithm is implemented as an agent that adapts its behavior according to uncertainty considerations. These agents act as a group of experts, where each agent has a specific knowledge area. Cooperation among the agents is given by a probabilistic scheme that uses Bayesian inference to integrate the evidential information provided by them. To deal with the inherent no linearity of visual information, the relevant probability distributions are represented using a stochastic sampling approach. The estimation of the state of relevant visual structures is performed using an enhanced version of the particle filter algorithm. This enhanced version includes novel methods to adaptively select the number of samples used by the filter, and to adaptively find a suitable function to propagate the samples. The implementation of the computational framework is performed using a distributed multi-agent software architecture. This is tested for the case of visual target tracking using a mobile platform. The evaluation of the implementation using computer simulations and real situations compares positively with current state of the art visual target tracking techniques.