Predicting Gaze-based Target Selection in Augmented Reality Headsets based on Eye and Head Endpoint Distributions

Yushi Wei; Rongkai Shi; Difeng Yu; Yihong Wang; Yue Li; Lingyun Yu; Hai Ning Liang

doi:10.1145/3544548.3581042

Predicting Gaze-based Target Selection in Augmented Reality Headsets based on Eye and Head Endpoint Distributions

Yushi Wei
, Rongkai Shi
, Difeng Yu
, Yihong Wang
, Yue Li
, Lingyun Yu
, Hai Ning Liang^*

^*Corresponding author for this work

University of Melbourne

Research output: Chapter in Book or Report/Conference proceeding › Conference Proceeding › peer-review

56 Citations (Scopus)

Abstract

Target selection is a fundamental task in interactive Augmented Reality (AR) systems. Predicting the intended target of selection in such systems can provide users with a smooth, low-friction interaction experience. Our work aims to predict gaze-based target selection in AR headsets with eye and head endpoint distributions, which describe the probability distribution of eye and head 3D orientation when a user triggers a selection input. We first conducted a user study to collect users' eye and head behavior in a gaze-based pointing selection task with two confirmation mechanisms (air tap and blinking). Based on the study results, we then built two models: a unimodal model using only eye endpoints and a multimodal model using both eye and head endpoints. Results from a second user study showed that the pointing accuracy is improved by approximately 32% after integrating our models into gaze-based selection techniques.

Original language	English
Title of host publication	CHI 2023 - Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
Publisher	Association for Computing Machinery
ISBN (Electronic)	9781450394215
DOIs	https://doi.org/10.1145/3544548.3581042
Publication status	Published - 19 Apr 2023
Event	2023 CHI Conference on Human Factors in Computing Systems, CHI 2023 - Hamburg, Germany Duration: 23 Apr 2023 → 28 Apr 2023

Publication series

Name	Conference on Human Factors in Computing Systems - Proceedings

Conference

Conference	2023 CHI Conference on Human Factors in Computing Systems, CHI 2023
Country/Territory	Germany
City	Hamburg
Period	23/04/23 → 28/04/23

Keywords

Augmented reality
error prediction
eye input
selection modeling
target selection

Access to Document

10.1145/3544548.3581042

Cite this

Wei, Y., Shi, R., Yu, D., Wang, Y., Li, Y., Yu, L., & Liang, H. N. (2023). Predicting Gaze-based Target Selection in Augmented Reality Headsets based on Eye and Head Endpoint Distributions. In CHI 2023 - Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems Article 283 (Conference on Human Factors in Computing Systems - Proceedings). Association for Computing Machinery. https://doi.org/10.1145/3544548.3581042

@inproceedings{d19ea4edfa954795aa6c2c9fa2860f90,

title = "Predicting Gaze-based Target Selection in Augmented Reality Headsets based on Eye and Head Endpoint Distributions",

abstract = "Target selection is a fundamental task in interactive Augmented Reality (AR) systems. Predicting the intended target of selection in such systems can provide users with a smooth, low-friction interaction experience. Our work aims to predict gaze-based target selection in AR headsets with eye and head endpoint distributions, which describe the probability distribution of eye and head 3D orientation when a user triggers a selection input. We first conducted a user study to collect users' eye and head behavior in a gaze-based pointing selection task with two confirmation mechanisms (air tap and blinking). Based on the study results, we then built two models: a unimodal model using only eye endpoints and a multimodal model using both eye and head endpoints. Results from a second user study showed that the pointing accuracy is improved by approximately 32\% after integrating our models into gaze-based selection techniques.",

keywords = "Augmented reality, error prediction, eye input, selection modeling, target selection",

author = "Yushi Wei and Rongkai Shi and Difeng Yu and Yihong Wang and Yue Li and Lingyun Yu and Liang, \{Hai Ning\}",

note = "Funding Information: We thank the volunteers who participated in our studies. We also thank the reviewers for their valuable time and insightful comments that helped improve this paper. This research was partly funded by Xi{\textquoteright}an Jiaotong-Liverpool University Special Key Fund (\#KSF-A-03), the National Science Foundation of China (\#62272396; \#62207022), and the Natural Science Foundation of the Jiangsu Higher Education Institutions of China (\#22KJB520038). Publisher Copyright: {\textcopyright} 2023 ACM.; 2023 CHI Conference on Human Factors in Computing Systems, CHI 2023 ; Conference date: 23-04-2023 Through 28-04-2023",

year = "2023",

month = apr,

day = "19",

doi = "10.1145/3544548.3581042",

language = "English",

series = "Conference on Human Factors in Computing Systems - Proceedings",

publisher = "Association for Computing Machinery",

booktitle = "CHI 2023 - Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems",

}

Wei, Y, Shi, R, Yu, D, Wang, Y , Li, Y , Yu, L & Liang, HN 2023, Predicting Gaze-based Target Selection in Augmented Reality Headsets based on Eye and Head Endpoint Distributions. in CHI 2023 - Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems., 283, Conference on Human Factors in Computing Systems - Proceedings, Association for Computing Machinery, 2023 CHI Conference on Human Factors in Computing Systems, CHI 2023, Hamburg, Germany, 23/04/23. https://doi.org/10.1145/3544548.3581042

Predicting Gaze-based Target Selection in Augmented Reality Headsets based on Eye and Head Endpoint Distributions. / Wei, Yushi; Shi, Rongkai; Yu, Difeng et al.
CHI 2023 - Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, 2023. 283 (Conference on Human Factors in Computing Systems - Proceedings).

Research output: Chapter in Book or Report/Conference proceeding › Conference Proceeding › peer-review

TY - GEN

T1 - Predicting Gaze-based Target Selection in Augmented Reality Headsets based on Eye and Head Endpoint Distributions

AU - Wei, Yushi

AU - Shi, Rongkai

AU - Yu, Difeng

AU - Wang, Yihong

AU - Li, Yue

AU - Yu, Lingyun

AU - Liang, Hai Ning

N1 - Funding Information: We thank the volunteers who participated in our studies. We also thank the reviewers for their valuable time and insightful comments that helped improve this paper. This research was partly funded by Xi’an Jiaotong-Liverpool University Special Key Fund (#KSF-A-03), the National Science Foundation of China (#62272396; #62207022), and the Natural Science Foundation of the Jiangsu Higher Education Institutions of China (#22KJB520038). Publisher Copyright: © 2023 ACM.

PY - 2023/4/19

Y1 - 2023/4/19

N2 - Target selection is a fundamental task in interactive Augmented Reality (AR) systems. Predicting the intended target of selection in such systems can provide users with a smooth, low-friction interaction experience. Our work aims to predict gaze-based target selection in AR headsets with eye and head endpoint distributions, which describe the probability distribution of eye and head 3D orientation when a user triggers a selection input. We first conducted a user study to collect users' eye and head behavior in a gaze-based pointing selection task with two confirmation mechanisms (air tap and blinking). Based on the study results, we then built two models: a unimodal model using only eye endpoints and a multimodal model using both eye and head endpoints. Results from a second user study showed that the pointing accuracy is improved by approximately 32% after integrating our models into gaze-based selection techniques.

AB - Target selection is a fundamental task in interactive Augmented Reality (AR) systems. Predicting the intended target of selection in such systems can provide users with a smooth, low-friction interaction experience. Our work aims to predict gaze-based target selection in AR headsets with eye and head endpoint distributions, which describe the probability distribution of eye and head 3D orientation when a user triggers a selection input. We first conducted a user study to collect users' eye and head behavior in a gaze-based pointing selection task with two confirmation mechanisms (air tap and blinking). Based on the study results, we then built two models: a unimodal model using only eye endpoints and a multimodal model using both eye and head endpoints. Results from a second user study showed that the pointing accuracy is improved by approximately 32% after integrating our models into gaze-based selection techniques.

KW - Augmented reality

KW - error prediction

KW - eye input

KW - selection modeling

KW - target selection

UR - https://www.scopus.com/pages/publications/85159641837

U2 - 10.1145/3544548.3581042

DO - 10.1145/3544548.3581042

M3 - Conference Proceeding

AN - SCOPUS:85159641837

T3 - Conference on Human Factors in Computing Systems - Proceedings

BT - CHI 2023 - Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

PB - Association for Computing Machinery

T2 - 2023 CHI Conference on Human Factors in Computing Systems, CHI 2023

Y2 - 23 April 2023 through 28 April 2023

ER -

Wei Y, Shi R, Yu D, Wang Y , Li Y , Yu L et al. Predicting Gaze-based Target Selection in Augmented Reality Headsets based on Eye and Head Endpoint Distributions. In CHI 2023 - Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery. 2023. 283. (Conference on Human Factors in Computing Systems - Proceedings). doi: 10.1145/3544548.3581042

Predicting Gaze-based Target Selection in Augmented Reality Headsets based on Eye and Head Endpoint Distributions

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Natural Interaction Techniques for Immersive Data Visualization

Virtual Reality Education Technology for Smart Museums

Study on Extended Reality Collaboration Technology for Metaverse

Cite this

Predicting Gaze-based Target Selection in Augmented Reality Headsets based on Eye and Head Endpoint Distributions

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Projects

Natural Interaction Techniques for Immersive Data Visualization

Virtual Reality Education Technology for Smart Museums

Study on Extended Reality Collaboration Technology for Metaverse

Cite this