Nonparametric Divergence Estimators for Independent Subspace Analysis

Abstract

In this paper we propose new nonparametric Rényi, Tsallis, and L 2 divergence estimators and demonstrate their applicability to mutual information estimation and independent subspace analysis. Given two independent and identically distributed samples, a “naïve” divergence estimation approach would simply estimate the underlying densities, and plug these densities into the corresponding integral formulae. In contrast, our estimators avoid the need to consistently estimate these densities, and still they can lead to consistent estimations. Numerical experiments illustrate the efficiency of the algorithms.