Scoring and Classification Multi-Functional Pose Matching Network Combining Alignment and Attention Mechanism

In recent years, posture matching has found extensive applications in rehabilitation training. However, the recent known study has primarily concentrated on posture scoring performance while neglecting the incorporation of classification capabilities. To address this limitation, this paper introduces a novel concept and presents a functional module that integrates both classification and scoring, with the objective of generating similarity and classification outcomes simultaneously. This study chose three yoga poses from Kaggle as the original dataset, and then used OpenPose to extract the skeletal features of the joints and other parts of the images, and used the Scale Invariant Feature Transform (SIFT) algorithm to align the images as the input dataset. This study designed an attention-based Siamese network combined with a classification module. Additionally, this study combined the SENet attention module with the VGG-16 backbone. Two images are input and pass through 13 convolutional layers, 5 max pooling layers, 1 dropout layer and two fully connected layers, and then combine with CosineEmbeddingLoss and CrossEntropyLoss functions to obtain their posture classification and similarity. Through the training and testing of the model, excellent experimental results can be obtained. The outcomes of posture extraction and alignment exhibit remarkable clarity. The output results indicate a similarity of 0.941 for identical postures, -0.390 for different postures, an accuracy rate of 0.997, and a loss value of 0.112. These findings offer precise posture classification and scoring results for patients.