IText: Hands-free Text Entry on an Imaginary Keyboard for Augmented Reality Systems

Xueshi Lu, DIfeng Yu, Hai Ning Liang, Jorge Goncalves

Research output: Chapter in Book or Report/Conference proceedingConference Proceedingpeer-review

25 Citations (Scopus)

Abstract

Text entry is an important and frequent task in interactive devices including augmented reality head-mounted displays (AR HMDs). In current AR HMDs, there are still two main open challenges to overcome for efficient and usable text entry: arm fatigue due to mid-air input and visual occlusion because of their small see-through displays. To address these challenges, we present iText, a technique for AR HMDs that is hands-free and is based on an imaginary (invisible) keyboard. We first show that it is feasible and practical to use an imaginary keyboard on AR HMDs. Then, we evaluated its performance and usability with three hands-free selection mechanisms: eye blinks (E-Type), dwell (D-Type), and swipe gestures (G-Type). Our results show that users could achieve an average text entry speed of 11.95, 9.03 and 9.84 words per minutes (WPM) with E-Type, D-Type, and G-Type, respectively. Given that iText with E-Type outperformed the other two selection mechanisms in text entry rate and subjective feedback, we ran a third, 5-day study. Our results show that iText with E-Type can achieve an average text entry rate of 13.76 WPM with a mean word error rate of 1.5%. In short, iText can enable efficient eyes-free text entry and can be useful for various application scenarios in AR HMDs.

Original languageEnglish
Title of host publicationUIST 2021 - Proceedings of the 34th Annual ACM Symposium on User Interface Software and Technology
PublisherAssociation for Computing Machinery, Inc
Pages815-825
Number of pages11
ISBN (Electronic)9781450386357
DOIs
Publication statusPublished - 10 Oct 2021
Event34th Annual ACM Symposium on User Interface Software and Technology, UIST 2021 - Virtual, Online, United States
Duration: 10 Oct 202114 Oct 2021

Publication series

NameUIST 2021 - Proceedings of the 34th Annual ACM Symposium on User Interface Software and Technology

Conference

Conference34th Annual ACM Symposium on User Interface Software and Technology, UIST 2021
Country/TerritoryUnited States
CityVirtual, Online
Period10/10/2114/10/21

Keywords

  • augmented reality
  • dwell
  • eye blink
  • hands-free
  • head-mounted displays
  • text entry
  • typing

Cite this