Vision-Based Human Presence Detection by Means of Transfer Learning Approach

Human–robot interaction (HRI) and human–robot collaboration (HRC) has become more important as the industries are stepping towards the phase of digitalization and Industry 4.0. Indeed, the emphasis is often placed on the safety of the physical well-being of the human workers. To safeguard the human operators from being hurt by the robots or collaborative robots (cobots), a traditional method is to isolate the robots from the human workers by means of fences and sensors. However, the deployment of deep learning models is unknown and shown to be non-trivial in downstream tasks such as image classification and object detection. The present study aimed to exploit the effectiveness of object detection models, particularly EfficientDet models via a transfer learning approach—fine-tuning. A total of 1463 images were obtained from the surveillance cameras from TT Vision Holdings Berhad and split into training, validation, and test by a ratio of 70:20:10. The training images were further augmented using horizontal flip and scale jittering techniques to increase the total training images up to 3072 images. As an outcome, the result revealed that the EfficientDet-D2 fine-tuned model achieved a test AP of 81.70% with an inference speed of 97.06 ms on Tesla T4 while the EfficientDet-D0 fine-tuned model attained a test AP of 69.30% with an inference speed of 30.24 ms on Tesla T4. In comparison between the EfficientDet-D0 fine-tuned model and EfficientDet-D2 fine-tuned model, the performance improved in terms of AP with the inference speed as the trade-off. The research has shown that it is feasible to detect the presence of human workers and can possibly serve as the visual perception of the robot with regards to human presence detection. Last but not least, the present work proved the applicability of transfer learning methods in human presence detection, specifically fine-tuned object detection models.