Multifactor user authentication with in-air-handwriting and hand geometry

On wearable and Virtual Reality (VR) platforms, user authentication is a basic function, but usually a keyboard or touchscreen cannot be provided to type a password. Hand gesture and especially in-air-handwriting can be potentially used for user authentication because a gesture input interface is readily available on these platforms. However, determining whether a login request is from the legitimate user based on a piece of hand movement is challenging in both signal processing and matching, which leads to limited performance in existing systems. In this paper, we propose a multifactor user authentication framework using both the motion signal of a piece of in-air-handwriting and the geometry of hand skeleton captured by a depth camera. To demonstrate this framework, we invented a signal matching algorithm, implemented a prototype, and conducted experiments on a dataset of 100 users collected by us. Our system achieves 0.6% Equal Error Rate (EER) without spoofing attack and 3.4% EER with spoofing only data, which is a significant improvement compared to existing systems using the Dynamic Time Warping (DTW) algorithm. In addition, we presented an in-depth analysis of the utilized features to explain the reason for the performance boost.