As vision systems become more and more complex there is an increasing need to understand the interaction between the various modules that these systems are composed of. In this project we are studying the question of how a high-level module can feed back its knowledge to a low-level module to improve the performance of the overall system. In particular we are working with a system model consisting of a single low-level module that takes a set of low-level parameters as input and a single high-level module that estimates a set of high-level model parameters. We pose the problem as setting the low-level parameters to maximize the performance of the overall system. Previous approaches to this problem include setting the parameters by hand, empirical evaluation, learning, and updating the parameters using the previous image in a video. Instead, we propose an approach based on simultaneous optimization of the high-level and low-level parameters. We have tested our paradigm on a variety of problems.

The first task is color-based head tracking. The results below demonstrate the improvement possible with our approach. The movie compares the standard approach of fixed low-level parameters (top left) against 3 variants of our approach (the other three panels).

The second task is segmentation-based lane tracking. The movie below compares the standard approach of fixed low-level parameters (right) against our approach (middle).