Toward Versatile Structural Modification for Bayesian Nonparametric Time Series Models

Abstract

Unsupervised learning techniques discover organizational structure in data, but to do so they must approach the problem with a priori assumptions. A fundamental trend in the development of these techniques has been the relaxation or elimination of the unwanted or arbitrary structural assumptions they impose. For systems that derive hidden Markov models (HMMs) from time series data, state-of-the-art techniques now assume only that the number of hidden states will be relatively small, a useful, flexible, and usually correct hypothesis. With unwanted structural constraints mitigated, we investigate a flexible means of intro- ducing new, useful structural assumptions into an advanced HMM learning technique, assumptions that reflect details of our prior understanding of the problem. Our investi- gation, motivated by the unsupervised learning of view-based object models from video data, adapts a Bayesian nonparametric approach to inferring HMMs from data [1] to ex- hibit biases for nearly block diagonal transition dynamics, as well as for transitions be- tween hidden states with similar emission models. We introduce aggressive Markov chain Monte Carlo sampling techniques for posterior inference in our generalized models, and demonstrate the technique in a collection of artificial and natural data settings, including the motivating object model learning problem.