TY - JOUR
T1 - Improving synthetic 3D model-aided indoor image localization via domain adaptation
AU - Li, Qing
AU - Cao, Rui
AU - Zhu, Jiasong
AU - Hou, Xianxu
AU - Liu, Jun
AU - Jia, Sen
AU - Li, Qingquan
AU - Qiu, Guoping
N1 - Publisher Copyright:
© 2021 International Society for Photogrammetry and Remote Sensing, Inc. (ISPRS)
PY - 2022/1
Y1 - 2022/1
N2 - Although the deep learning-based indoor image localization has made significant improvement in terms of accuracy, efficiency, and storage requirement of large indoor scenes, the need for collecting huge labeled training data severely limits its practical application. Recently, the synthetic images rendered from widely available 3D models have shown promising potential to relieve the data collection problem. However, due to the dramatic differences between the synthetic and real images, the localization accuracy of approaches trained on synthetic images is not comparable to the methods trained on real images. In this paper, we propose a domain adaptation-based approach to address this issue. Specifically, the proposed approach mainly contains a model consisting of a multi-level constrained pose regression network and a feature-level discriminator network. The discriminator network forces the pose regression network to generate domain-invariant features from real and synthetic images by adversarial learning and thus reduces the performance gaps. In addition, the multi-level constraints further enhance the localization accuracy of pose regression. We perform extensive experiments on open-source rendering images in different settings. The results show that the proposed method significantly improves the performance. The code for the proposed work is available at https://github.com/lqing900205/BIM_domainadaptation.
AB - Although the deep learning-based indoor image localization has made significant improvement in terms of accuracy, efficiency, and storage requirement of large indoor scenes, the need for collecting huge labeled training data severely limits its practical application. Recently, the synthetic images rendered from widely available 3D models have shown promising potential to relieve the data collection problem. However, due to the dramatic differences between the synthetic and real images, the localization accuracy of approaches trained on synthetic images is not comparable to the methods trained on real images. In this paper, we propose a domain adaptation-based approach to address this issue. Specifically, the proposed approach mainly contains a model consisting of a multi-level constrained pose regression network and a feature-level discriminator network. The discriminator network forces the pose regression network to generate domain-invariant features from real and synthetic images by adversarial learning and thus reduces the performance gaps. In addition, the multi-level constraints further enhance the localization accuracy of pose regression. We perform extensive experiments on open-source rendering images in different settings. The results show that the proposed method significantly improves the performance. The code for the proposed work is available at https://github.com/lqing900205/BIM_domainadaptation.
KW - Deep learning
KW - Domain adaptation
KW - Image localization
KW - Indoor localization
KW - Synthetic images
UR - http://www.scopus.com/inward/record.url?scp=85118875032&partnerID=8YFLogxK
U2 - 10.1016/j.isprsjprs.2021.10.005
DO - 10.1016/j.isprsjprs.2021.10.005
M3 - Article
AN - SCOPUS:85118875032
SN - 0924-2716
VL - 183
SP - 66
EP - 78
JO - ISPRS Journal of Photogrammetry and Remote Sensing
JF - ISPRS Journal of Photogrammetry and Remote Sensing
ER -