Two-layer Mixture of Factor Analyzers with Joint Factor Loading

Xi Yang, Kaizhu Huang, Rui Zhang, John Yannis Goulermas

Research output: Chapter in Book or Report/Conference proceedingConference Proceedingpeer-review

1 Citation (Scopus)


Dimensionality Reduction (DR) is a fundamental yet active research topic in pattern recognition and machine learning. When used in classification, previous research usually performs DR separately, and then inputs the reduced features to other available models, e.g., Gaussian Mixture Model (GMM). Such independent learning could however significantly limit the classification performance, since the optimal subspace given by a particular DR approach may not be appropriate for the following classification model. More seriously, for high-dimensional data classification in the face of a limited number of samples (called small sample size or S3 problem), independent learning of DR and classification model may even deteriorate the classification accuracy. To solve this problem, we propose a joint learning model, called Two-layer Mixture of Factor Analyzers with Joint Factor Loading (2L-MJFA) for classification. More specifically, our proposed model enjoys a two-layer mixture structure, or a mixture of mixtures structure, with each component (representing each specific class) as another mixture model of Factor Analyzer (MFA). Importantly, all the involved factor analyzers are intentionally designed to share the same loading matrix. On one hand, such joint loading matrix can be considered as the dimensionality reduction matrix; on the other hand, a joint common matrix would largely reduce the parameters, making the proposed algorithm very suitable for S3 problems. We describe our model definition and propose a modified EM algorithm to optimize the model. A series of experiments demonstrates that our proposed model significantly outperforms the other three competitive algorithms on five data sets.

Original languageEnglish
Title of host publication2015 International Joint Conference on Neural Networks, IJCNN 2015
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781479919604, 9781479919604, 9781479919604, 9781479919604
Publication statusPublished - 28 Sept 2015
EventInternational Joint Conference on Neural Networks, IJCNN 2015 - Killarney, Ireland
Duration: 12 Jul 201517 Jul 2015

Publication series

NameProceedings of the International Joint Conference on Neural Networks


ConferenceInternational Joint Conference on Neural Networks, IJCNN 2015


  • Analytical models
  • Joints
  • Optimization


Dive into the research topics of 'Two-layer Mixture of Factor Analyzers with Joint Factor Loading'. Together they form a unique fingerprint.

Cite this