A semantic-based approach to digital content placement for immersive environments

Jingyang Liu*, Yunzhi Li, Mayank Goel

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

This paper presents a semantic-based interactive system that enables virtual content placement using natural language. We propose a novel computational framework composed of three components including 3D reconstruction, 3D segmentation, and 3D annotation. Based on the framework, the system can automatically construct a semantic representation of the environment from raw point cloud data. Users can then assign virtual content to a specific physical location by referring to its semantic label. Compared with traditional projection mapping which may involve tedious manual adjustments, the proposed system can facilitate intuitive and efficient manipulation of virtual content in immersive environments through speech inputs. The technical evaluation and user study results show that the system can provide users with accurate semantic information for effective virtual content placement at room scale.

Original languageEnglish
Pages (from-to)5989-6003
Number of pages15
JournalVisual Computer
Volume39
Issue number12
DOIs
Publication statusPublished - Dec 2023
Externally publishedYes

Keywords

  • Digital content placement
  • Scene understanding
  • Semantic-based interaction

Fingerprint

Dive into the research topics of 'A semantic-based approach to digital content placement for immersive environments'. Together they form a unique fingerprint.

Cite this