MiTPose: Multi-Granularity Guided Vision Transformer for Human Pose Estimation

Two-dimensional human pose estimation (HPE) has been extensively applied across various domains, including behavioral analysis, identity verification, and automated industrial manufacturing. Compared to convolutional neural networks (CNNs), the Vision Transformer (ViT) has demonstrated impressive results in human pose estimation. However, two main challenges arise: (1) the complexity of image size and parameters grows quadratically, making traditional ViT unsuitable for deployment on edge devices, and (2) the attention mechanism in transformers lacks the ability to capture local fine-grained perception. To address these issues, we propose a novel method Multi-Granularity guided Vision Transformer for human pose estimation (MiTPOSE), which integrates both CNN and transformer for feature encoding. Specifically, we introduce an improved SCConv encoder with Global Response Normalization, which consists of a spatial reconstruction unit and channel reconstruction unit to reduce redundant computations and enhance representative feature learning. Furthermore, we incorporate a novel Multi-Granularity block to address the shortcomings of traditional self-attention mechanisms in capturing local fine-grained details, and a lightweight decoder for keypoint detection. Comprehensive evaluations and tests on the COCO benchmark datasets demonstrate that MiTPose achieves competitive performance in pose estimation compared to state-of-the-art methods.