Mixture of factor models for joint dimensionality reduction and classification
Date
2016
Authors
Tabaghi, Puoya, author
Azimi-Sadjadi, Mahmood R., advisor
Scharf, Louis L., advisor
Pezeshki, Ali, committee member
Kirby, Michael, committee member
Journal Title
Journal ISSN
Volume Title
Abstract
In many areas such as machine learning, pattern recognition, information retrieval, and data mining one is interested in extracting a low-dimensional data that is truly representative of the properties of the original high dimensional data. For example, one application could be extracting representative low-dimensional features of underwater objects from sonar imagery suitable for detection and classification. This is a difficult problem due to various factors such as variations in the operating and environmental conditions, presence of spatially varying clutter, and variations in object shapes, compositions, and orientation. The goal of this work is to develop a novel probabilistic method using a mixture of factor models for simultaneous nonlinear dimensionality reduction and classification. The framework used here is inspired by the work in [1] which uses a mixture of local PCA projections leading to an unsupervised nonlinear dimensionality reduction algorithm. In contrast, the proposed method provides a supervised probabilistic approach suitable for analyzing labeled high-dimensional data with complex structures by exploiting a set of low-dimensional latent variables which are both discriminative and generative. With the aid of these low-dimensional latent variables, a mixture of linear models is introduced to represent the high-dimensional data. An optimum linear classifier is then built in the latent variable-domain to separate the support of the latent variable associated with each class. Introducing these hidden variables allow us to derive the joint probability density function of the data and class label, reduce data dimension and perform clustering, classification and parameter estimation. This probabilistic approach provides a mechanism to traverse between the input space and latent (feature) space and vice versa as well as cluster and classify data. A supervised training based on the Expectation-Maximization (EM) and steepest descent algorithms is then introduced to derive the ML estimates of the unknown parameters. It is shown that parameters associated with dimensionality reduction can be estimated using the EM algorithm whereas those of the classifier are estimated using the steepest descent method. The introduction of latent variables not only helps to represent the pdf of data and reduce the dimension of them but also in parameter estimation using EM algorithm which is used to find ML estimates of the parameters when the available data is incomplete. A comprehensive study is carried out to assess the performance of the proposed method using two different data sets. The first data set consists of Synthetic Aperture Sonar (SAS) images of model-generated underwater objects superimposed on background clutter. These images correspond to two different object types namely Cylinder (mine-like) and Block (non-mine-like). The signatures of each object are synthetically generated and are placed at various aspect angles from 1 to 180 degrees for each object type. The goal of our classifier is to assign non-target versus target labels to these image snippets. The other data set consists of two sets of facial images of different individuals. Each image set contains 2 series of 93 images of the same person at different poses. The goal of the classifier for this case is to identify each indi vidual correctly. The dimensionality reduction performance of the proposed method is compared to two relevant dimensionality reduction methods, namely Probabilistic PCA [2] and Mixture of Probabilistic PCA (MPPCA) [1] while its classification performance is benchmarked against a Support Vector Machine (SVM). The results on both data sets indicate promising dimensionality reduction and reconstruction capabilities compared to PPCA/MPPCA methods. On the other hand, classification performance is competitive with SVM when the data is linearly separable.