Spatiotemporal Modeling using Recurrent Neural Processes

Abstract

Spatiotemporal processes, such as temperature in an area, motion of a vehicle, etc., depend on the spatial features of the underlying phenomena as well as time. Developing models that can estimate both mean and uncertainty associated with the prediction is important for building robust systems capable of performing such tasks. Although, Gaussian Process models can estimate predictive distributions over the target data points, they are not scalable to large data regimes due to their non-parametric nature. Moreover, the explicit functional form of kernels limits the modeling capabilities to only smoothing and interpolation of data.

In this work, we propose a deep neural network model, called Recurrent Neural Processes, adept for modeling spatiotemporal data. Being parametric, our model easily scales to large data regimes. Additionally, the use of neural networks enables the model to overcome functional design restrictions and learn implicit kernels from the data directly. Our proposed model is a latent variable model, i.e., the predicted output for a given input depends on sample drawn from a learned latent distribution. The model's predictive distribution is empirically estimated from the multiple output values obtained as a result of drawing different samples from the latent distribution. We compare our proposed model with the existent ones in the literature and show that our model can make accurate future time step predictions and can also provide meaningful structured uncertainty estimates for spatiotemporal data.