TY - GEN
T1 - HGR-ResNet: Hand Gesture Recognition with Enhanced Residual Neural Network
AU - Tan, Chun Keat
AU - Ming Lim, Kian
AU - Lee, Chin Poo
AU - Kwang Yang Chang, Roy
AU - Lim, Jit Yan
N1 - Publisher Copyright:
© 2023 IEEE.
PY - 2023
Y1 - 2023
N2 - Hand Gesture Recognition (HGR) has garnered increasing attention in recent years due to its potential to enhance human-computer interaction (HCI) and facilitate communication between individuals who are mute or deaf and the wider public. HGR can facilitate non-contact interaction between humans and machines, offering an effective interface for recognizing sign language used in everyday communication. This paper proposes a novel approach for static HGR using transfer learning of ResNet152 with early stopping, adaptive learning rate, and class weightage techniques, referred to as HGR-ResNet. Transfer learning enables the model to utilize previously acquired knowledge from pre-training on a large dataset, allowing it to learn from pre-extracted image features. Early stopping serves as a regularization technique, halting the training process before overfitting occurs. Adaptive learning rate adjusts the learning rate dynamically based on the model's error rate during training, promoting faster convergence and improved accuracy. Additionally, the class weightage technique is employed to address the issue of class imbalance in the data, ensuring fair representation and mitigating biases during the training process. To assess the effectiveness of the proposed model, we conduct a comparative analysis with multiple existing methods using three distinct datasets: the American Sign Language (ASL) dataset, ASL with digits dataset, and the National University of Singapore (NUS) hand gesture dataset. HGR-ResNet achieves remarkable results, with an average accuracy of 99.20% across all three datasets, and individual accuracies of 99.88% for the ASL dataset, 98.93% for the ASL with digits dataset, and 98.80% for the NUS hand gesture dataset.
AB - Hand Gesture Recognition (HGR) has garnered increasing attention in recent years due to its potential to enhance human-computer interaction (HCI) and facilitate communication between individuals who are mute or deaf and the wider public. HGR can facilitate non-contact interaction between humans and machines, offering an effective interface for recognizing sign language used in everyday communication. This paper proposes a novel approach for static HGR using transfer learning of ResNet152 with early stopping, adaptive learning rate, and class weightage techniques, referred to as HGR-ResNet. Transfer learning enables the model to utilize previously acquired knowledge from pre-training on a large dataset, allowing it to learn from pre-extracted image features. Early stopping serves as a regularization technique, halting the training process before overfitting occurs. Adaptive learning rate adjusts the learning rate dynamically based on the model's error rate during training, promoting faster convergence and improved accuracy. Additionally, the class weightage technique is employed to address the issue of class imbalance in the data, ensuring fair representation and mitigating biases during the training process. To assess the effectiveness of the proposed model, we conduct a comparative analysis with multiple existing methods using three distinct datasets: the American Sign Language (ASL) dataset, ASL with digits dataset, and the National University of Singapore (NUS) hand gesture dataset. HGR-ResNet achieves remarkable results, with an average accuracy of 99.20% across all three datasets, and individual accuracies of 99.88% for the ASL dataset, 98.93% for the ASL with digits dataset, and 98.80% for the NUS hand gesture dataset.
KW - Hand gesture recognition
KW - Human-computer interaction
KW - ResNet
KW - Sign language recognition
UR - http://www.scopus.com/inward/record.url?scp=85174393338&partnerID=8YFLogxK
U2 - 10.1109/ICoICT58202.2023.10262710
DO - 10.1109/ICoICT58202.2023.10262710
M3 - Conference Proceeding
AN - SCOPUS:85174393338
T3 - 2023 11th International Conference on Information and Communication Technology, ICoICT 2023
SP - 131
EP - 136
BT - 2023 11th International Conference on Information and Communication Technology, ICoICT 2023
T2 - 11th International Conference on Information and Communication Technology, ICoICT 2023
Y2 - 23 August 2023 through 24 August 2023
ER -