Learning Implicit Surface Light Fields

Michael Oechsle, Michael Niemeyer, Christian Reiser, Lars Mescheder, Thilo Strauss, Andreas Geiger

Research output: Chapter in Book or Report/Conference proceedingConference Proceedingpeer-review

26 Citations (Scopus)

Abstract

Implicit representations of 3D objects have recently achieved impressive results on learning-based 3D reconstruction tasks. While existing works use simple texture models to represent object appearance, photo-realistic image synthesis requires reasoning about the complex interplay of light, geometry and surface properties. In this work, we propose a novel implicit representation for capturing the visual appearance of an object in terms of its surface light field. In contrast to existing representations, our implicit model represents surface light fields in a continuous fashion and independent of the geometry. Moreover, we condition the surface light field with respect to the location and color of a small light source. Compared to traditional surface light field models, this allows us to manipulate the light source and relight the object using environment maps. We further demonstrate the capabilities of our model to predict the visual appearance of an unseen object from a single real RGB image and corresponding 3D shape information. As evidenced by our experiments, our model is able to infer rich visual appearance including shadows and specular reflections. Finally, we show that the proposed representation can be embedded into a variational auto-encoder for generating novel appearances that conform to the specified illumination conditions.

Original languageEnglish
Title of host publicationProceedings - 2020 International Conference on 3D Vision, 3DV 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages452-462
Number of pages11
ISBN (Electronic)9781728181288
DOIs
Publication statusPublished - Nov 2020
Externally publishedYes
Event8th International Conference on 3D Vision, 3DV 2020 - Virtual, Fukuoka, Japan
Duration: 25 Nov 202028 Nov 2020

Publication series

NameProceedings - 2020 International Conference on 3D Vision, 3DV 2020

Conference

Conference8th International Conference on 3D Vision, 3DV 2020
Country/TerritoryJapan
CityVirtual, Fukuoka
Period25/11/2028/11/20

Keywords

  • 3D Deep Learning
  • Appearance modelling
  • Implicit Functions
  • Novel View Synthesis

Fingerprint

Dive into the research topics of 'Learning Implicit Surface Light Fields'. Together they form a unique fingerprint.

Cite this