3D Morphable Models and Beyond

Motivation

The Morphable Face Model paper of 1999 was the beginning of what is now a wide research field around the construction, fitting and application of those models. Since then, 3D morphable models have developed into a key concept in the reconstruction and understanding of real-world scenes from images, as well as the controllable synthesis of real-world imagery. The strong growth led to a variety of interpretations and applications and there is still lots of active research in this direction.

Recently, parts of the community shifted towards topics including neural rendering and representation learning which led to new possibilities, including more realistic rendering as well as multi-object models. Whilst those methods are novel and some are more a proof of concept than ready to be used in real-world applications, they have a strong potential since they can be learned from image data and do not rely on handcrafted models or large corpora of scanned 3D data that are hard to capture in practice.

In this Dagstuhl Seminar we aim to bring together researchers from industry and academia that work at the interface of morphable models, neural rendering and implicit representation learning. On the one hand, we will focus on classical methods, namely 3D Morphable Models (3DMMs) based on explicit shape representations and classical computer graphics based rendering pipelines emulating physical light transport. On the other hand, we will explore synergies to novel learned 3D generative models as well as novel neural rendering algorithms that replace or complement established computer graphics concepts with machine learned concepts. The seminar will therefore offer a unique platform to deeply and systematically explore the methodological synergies between these originally separately researched directions in graphics, vision, and machine learning. The seminar is meant to initiate cross-fertilization between those research areas to allow combining their benefits with the goal of building better models of the world.

This seminar aims to capture the key ideas and persons in the field to enable a coordinated approach to current limitations and open research questions as well as establishing stronger collaborations in this research area.