Pareto models for discriminative multiclass linear dimensionality reduction

Abstract

We address the class masking problem in multiclass linear discriminant analysis (LDA). In the multiclass setting, LDA does not maximize each pairwise distance between classes, but rather maximizes the sum of all pairwise distances. This results in serious overlaps between classes that are close to each other in the input space, and degrades classification performance. Our research proposes Pareto Discriminant Analysis (PARDA); an approach for multiclass discriminative analysis that builds over multiobjective optimizing models. PARDA decomposes the multiclass problem to a set of objective functions, each representing the distance between every pair of classes. Unlike existing LDA extensions that maximize the sum of all distances, PARDA maximizes each pairwise distance to maximally separate all class means, while minimizing the class overlap in the lower dimensional space. Experimental results on various data sets show consistent and promising performance of PARDA when compared with well-known multiclass LDA extensions.

Highlights
We address the class-masking problem (CMP) in Multiclass LDA (MLDA).
We model MLDA as a set of objective functions for pairwise distances between classes. We optimize the compound objective function using Multiobjective Optimization.

Our models overcome the CMP and are extended to multimodal non-Gaussian data. Empirical analysis shows consistent and promising performance in favor of our models.