Multiple-instance ensemble for construction of deep heterogeneous committees for high-dimensional low-sample-size data

Qinghua Zhou, Shuihua Wang, Hengde Zhu, Xin Zhang, Yudong Zhang*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)

Abstract

Deep ensemble learning, where we combine knowledge learned from multiple individual neural networks, has been widely adopted to improve the performance of neural networks in deep learning. This field can be encompassed by committee learning, which includes the construction of neural network cascades. This study focuses on the high-dimensional low-sample-size (HDLS) domain and introduces multiple instance ensemble (MIE) as a novel stacking method for ensembles and cascades. In this study, our proposed approach reformulates the ensemble learning process as a multiple-instance learning problem. We utilise the multiple-instance learning solution of pooling operations to associate feature representations of base neural networks into joint representations as a method of stacking. This study explores various attention mechanisms and proposes two novel committee learning strategies with MIE. In addition, we utilise the capability of MIE to generate pseudo-base neural networks to provide a proof-of-concept for a “growing” neural network cascade that is unbounded by the number of base neural networks. We have shown that our approach provides (1) a class of alternative ensemble methods that performs comparably with various stacking ensemble methods and (2) a novel method for the generation of high-performing “growing” cascades. The approach has also been verified across multiple HDLS datasets, achieving high performance for binary classification tasks in the low-sample size regime.

Original languageEnglish
Pages (from-to)380-399
Number of pages20
JournalNeural Networks
Volume167
DOIs
Publication statusPublished - Oct 2023
Externally publishedYes

Keywords

  • Attention
  • Committee learning
  • Deep learning
  • HDLS

Fingerprint

Dive into the research topics of 'Multiple-instance ensemble for construction of deep heterogeneous committees for high-dimensional low-sample-size data'. Together they form a unique fingerprint.

Cite this