Exploring Gaze-Assisted and Hand-based Region Selection in Augmented Reality

Rongkai Shi, Yushi Wei, Xueying Qin, Pan Hui, Hai Ning Liang*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

7 Citations (Scopus)


Region selection is a fundamental task in interactive systems. In 2D user interfaces, users typically use a rectangle selection tool to formulate a region using a mouse or touchpad. Region selection in 3D spaces, especially in Augmented Reality (AR) Head-Mounted Displays (HMDs) is different and challenging because users need to select an intended region via freehand mid-Air gestures or eye-based actions that are touchless interactions. In this work, we aim to fill in the gap in the design of region selection techniques in AR HMDs. We first analyzed and discretized the interaction procedure of region selection and explored design possibilities for each step. We then developed four techniques for region selection in AR HMDs, which leveraged users' hand and gaze for unimodal or multimodal interaction. The techniques were evaluated via a user study with a controlled region selection task. The findings led to three design recommendations and two proof-of-concept application examples.

Original languageEnglish
Article number160
JournalProceedings of the ACM on Human-Computer Interaction
Issue numberETRA
Publication statusPublished - 18 May 2023


  • augmented reality
  • eye-Tracking
  • gaze interaction
  • head-mounted display
  • multimodal interaction
  • region selection


Dive into the research topics of 'Exploring Gaze-Assisted and Hand-based Region Selection in Augmented Reality'. Together they form a unique fingerprint.

Cite this