PointNu-Net: Keypoint-Assisted Convolutional Neural Network for Simultaneous Multi-Tissue Histology Nuclei Segmentation and Classification

Kai Yao, Kaizhu Huang, Jie Sun, Amir Hussain

Research output: Contribution to journalArticlepeer-review

Abstract

Automatic nuclei segmentation and classification play a vital role in digital pathology. However, previous works are mostly built on data with limited diversity and small sizes, making the results questionable or misleading in actual downstream tasks. In this article, we aim to build a reliable and robust method capable of dealing with data from the ‘the clinical wild’. Specifically, we study and design a new method to simultaneously detect, segment, and classify nuclei from Haematoxylin and Eosin (H&E) stained histopathology data, and evaluate our approach using the recent largest dataset: PanNuke. We address the detection and classification of each nuclei as a novel semantic keypoint estimation problem to determine the center point of each nuclei. Next, the corresponding class-agnostic masks for nuclei center points are obtained using dynamic instance segmentation. Meanwhile, we proposed a novel Joint Pyramid Fusion Module (JPFM) to model the cross-scale dependencies, thus enhancing the local feature for better nuclei detection and classification. By decoupling two simultaneous challenging tasks and taking advantage of JPFM, our method can benefit from class-aware detection and class-agnostic segmentation, thus leading to a significant performance boost. We demonstrate the superior performance of our proposed approach for nuclei segmentation and classification across 19 different tissue types, delivering new benchmark results.

Original languageEnglish
Pages (from-to)1-12
Number of pages12
JournalIEEE Transactions on Emerging Topics in Computational Intelligence
Volume8
Issue number1
DOIs
Publication statusAccepted/In press - 2023

Keywords

  • Convolution
  • Convolutional neural networks
  • deep learning
  • digital pathology
  • Feature extraction
  • Heating systems
  • Image segmentation
  • Kernel
  • Nuclei segmentation and classification
  • Proposals

Fingerprint

Dive into the research topics of 'PointNu-Net: Keypoint-Assisted Convolutional Neural Network for Simultaneous Multi-Tissue Histology Nuclei Segmentation and Classification'. Together they form a unique fingerprint.

Cite this