Convex ensemble learning with sparsity and diversity

Xu Cheng Yin*, Kaizhu Huang, Chun Yang, Hong Wei Hao

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

45 Citations (Scopus)

Abstract

Classifier ensemble has been broadly studied in two prevalent directions, i.e., to diversely generate classifier components, and to sparsely combine multiple classifiers. While most current approaches are emphasized on either sparsity or diversity only, we investigate classifier ensemble focused on both in this paper. We formulate the classifier ensemble problem with the sparsity and diversity learning in a general mathematical framework, which proves beneficial for grouping classifiers. In particular, derived from the error-ambiguity decomposition, we design a convex ensemble diversity measure. Consequently, accuracy loss, sparseness regularization, and diversity measure can be balanced and combined in a convex quadratic programming problem. We prove that the final convex optimization leads to a closed-form solution, making it very appealing for real ensemble learning problems. We compare our proposed novel method with other conventional ensemble methods such as Bagging, least squares combination, sparsity learning, and AdaBoost, extensively on a variety of UCI benchmark data sets and the Pascal Large Scale Learning Challenge 2008 webspam data. Experimental results confirm that our approach has very promising performance.

Original languageEnglish
Pages (from-to)49-59
Number of pages11
JournalInformation Fusion
Volume20
Issue number1
DOIs
Publication statusPublished - Nov 2014

Keywords

  • Classifier ensemble
  • Convex quadratic programming
  • Diversity
  • Sparsity

Fingerprint

Dive into the research topics of 'Convex ensemble learning with sparsity and diversity'. Together they form a unique fingerprint.

Cite this