Semantic Mapping: A Semantics-based Approach to Virtual Content Placement for Immersive Environments

Jingyang Liu*

*Corresponding author for this work

Research output: Chapter in Book or Report/Conference proceedingConference Proceedingpeer-review

4 Citations (Scopus)

Abstract

Semantic Mapping is a semantics-based interactive system that enables intuitive virtual content placement for projection mapping in intelligent environments. Our semantic mapping system embeds semantic information of the environment to provide a user with an easy way to control and place projected virtual items in the physical world. In contrast to traditional projection mapping that involves manual adjustments, this semantic mapping system enables efficient manipulation of virtual content through inputs from users via speech or text. To build the system, we first use a commercial depth camera for scene reconstruction and an end-to-end deep learning framework for semantic segmentation at the instance level. We illustrate the system by developing a prototype for a set of proof-of-concept, room-scale applications. The accuracy study and user study results show that the system can provide users with accurate semantic information for effective virtual content placement.

Original languageEnglish
Title of host publication2021 17th International Conference on Intelligent Environments, IE 2021 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781665403467
DOIs
Publication statusPublished - Jun 2021
Externally publishedYes
Event17th International Conference on Intelligent Environments, IE 2021 - Proceedings - Virtual, Online, United Arab Emirates
Duration: 21 Jun 202124 Jun 2021

Publication series

Name2021 17th International Conference on Intelligent Environments, IE 2021 - Proceedings

Conference

Conference17th International Conference on Intelligent Environments, IE 2021 - Proceedings
Country/TerritoryUnited Arab Emirates
CityVirtual, Online
Period21/06/2124/06/21

Fingerprint

Dive into the research topics of 'Semantic Mapping: A Semantics-based Approach to Virtual Content Placement for Immersive Environments'. Together they form a unique fingerprint.

Cite this