TOP
Search the Dagstuhl Website
Looking for information on the websites of the individual seminars? - Then please:
Not found what you are looking for? - Some of our services have separate websites, each with its own search option. Please check the following list:
Schloss Dagstuhl - LZI - Logo
Schloss Dagstuhl Services
Seminars
Within this website:
External resources:
  • DOOR (for registering your stay at Dagstuhl)
  • DOSA (for proposing future Dagstuhl Seminars or Dagstuhl Perspectives Workshops)
Publishing
Within this website:
External resources:
dblp
Within this website:
External resources:
  • the dblp Computer Science Bibliography


Dagstuhl Seminar 16462

Inpainting-Based Image Compression

( Nov 13 – Nov 18, 2016 )

(Click in the middle of the image to enlarge)

Permalink
Please use the following short url to reference this page: https://www.dagstuhl.de/16462

Organizers

Contact



Motivation

Since the amount of visual data is rapidly increasing, there is a high demand for powerful methods for compressing digital images. A well-known example is the lossy JPEG standard that is based on the discrete cosine transform. Unfortunately ist quality deteriorates substantially for high compression rates, such that better alternatives are needed.

The goal of this seminar is to pursue a completely different strategy than traditional, transform-based codecs: We study approaches that rely on so-called inpainting methods. They store only a small, carefully selected subset of the image data. In the decoding phase, the missing data is reconstructed by interpolation with partial differential equations (PDEs) or by copying information from patches in other image regions. Such codecs allow a very intuitive interpretation, and first experiments show their advantages for high compression rates where they can beat even advanced transform-based methods.

However, inpainting-based codecs are still in an early stage and require to solve a number of challenging fundamental problems, for example:

  1. Which data gives the best reconstruction?
  2. What are the optimal inpainting operators?
  3. How should the selected data be encoded and decoded?
  4. What are the most efficient algorithms for real-time applications?
  5. Can these approaches be tailored towards specific imagery and other data types (e.g. hyperspectral data, HDR images, videos, surface data)?

These problems are highly interrelated. Moreover, they require interdisciplinary expertise from various fields such as image inpainting, data compression and coding, approximation theory, optimisation, and numerical analysis. To design these codecs in an optimal way, one must also understand their connections to other image compression strategies and to closely related areas such as biological vision, perceptually relevant features, texture models, parameter identification, harmonic analysis, sparsity and compressed sensing, scattered data approximation, radial basis functions, subdivision strategies, geometric modeling, and computer graphics.

There has been no workshop on this topic before. Thus, we want to bring together for the first time leading researchers from these different fields and with different background, ranging from computer science over mathematics and electrical engineering to psychophysics. They can benefit from their interactions in the highly inspiring atmosphere of Schloss Dagstuhl. It is also planned to edit the first book that is solely devoted to this topic, and participants are invited to contribute to it.


Summary

Since the amount of visual data is rapidly increasing, there is a high demand for powerful methods for compressing digital images. A well-known example is the lossy JPEG standard that is based on the discrete cosine transform. Unfortunately its quality deteriorates substantially for high compression rates, such that better alternatives are needed.

The goal of this seminar was to pursue a completely different strategy than traditional, transform-based codecs (coders and decoders): We studied approaches that rely on so-called inpainting methods. They store only a small, carefully selected subset of the image data. In the decoding phase, the missing data is reconstructed by interpolation with partial differential equations (PDEs) or by copying information from patches in other image regions. Such codecs allow a very intuitive interpretation, and first experiments show their advantages for high compression rates where they can beat even advanced transform-based methods.

However, inpainting-based codecs are still in an early stage and require to solve a number of challenging fundamental problems, in particular:

  • Which data gives the best reconstruction?
  • What are the optimal inpainting operators?
  • How should the selected data be encoded and decoded?
  • What are the most efficient algorithms for real-time applications?

These problems are highly interrelated. Moreover, they require interdisciplinary expertise from various fields such as image inpainting, data compression and coding, approximation theory, and optimisation. To design these codecs in an optimal way, one must also understand their connections to related areas such as sparsity and compressed sensing, harmonic analysis, scattered data approximation with radial basis functions, and subdivision strategies.

Our seminar constituted the first symposium on this topic. It brought together 29 researchers from 11 countries, covering a broad range of expertise in the different fields mentioned above. Many of them have met for the first time, which resulted in a very fruitful interaction.

In order to have a good basis for joint discussions, first all participants introduced themselves and briefly described their background and interests. Then the seminar proceeded with six tutorial talks (45 minutes plus 15 minutes discussion), given by the four organisers as well as by Simon Masnou and Nira Dyn. In this way all participants could acquire a general overview on the achievements and challenges of inpainting-based image compression and its various aspects such as coding, inpainting, convex optimisation, subdivision, and computational harmonic analysis.

Afterwards we decided to cluster the talks thematically into six sessions, each consisting of 3-4 talks (ca. 30 minutes plus 15 minutes discussion) and lasting half a day:

  1. Harmonic Analysis
    (talks by Gerlind Plonka-Hoch, Naoki Saito, and Hao-Min Zhou)
  2. Approximation Theory
    (talks by Martin Buhmann, Armin Iske, Nira Dyn, and Tomas Sauer)
  3. Inpainting
    (talks by Aurelien Bourquard, Carola-Bibiane Sch"onlieb, and Yann Gousseau)
  4. Compression
    (talks by Gene Cheung, Joan Serra Sagrista, and Claire Mantel)
  5. Optimisation of Data and Operators
    (talks by Zakaria Belhachmi, Laurent Hoeltgen, Peter Ochs, and Pascal Peter)
  6. Algorithms, Biological Vision, and Benchmarking
    (talks by Jalal Fadili, Johannes Ballé, and Sarah Andris)

These sessions triggered interesting discussions during the talks, in the breaks, and in the evening, and they allowed the different communities to learn many new things from each other.

Our program featured also an evening panel discussion on open research questions on the interface between image inpainting and image compression. It was a lively interaction between the five panel members and the audience, involving also controversial statements and views about the future of inpainting-based codecs.

The participants had a very positive impression of this seminar as an inspiring forum to bring together different fields. As a consequence, this symposium also created several new collaborations, e.g. regarding interpolation with radial basis functions, subdivision-based coding, and diffusion-based coding. There was a general consensus that it would be desirable to have another seminar on this topic in 2-3 years. Moreover, it is planned to compile a related monograph which will be the first in its field.

Copyright Joachim Weickert

Participants
  • Sarah Andris (Universität des Saarlandes, DE) [dblp]
  • Johannes Ballé (New York University, US) [dblp]
  • Zakaria Belhachmi (University of Mulhouse, FR) [dblp]
  • Aurélien Bourquard (MIT - Cambridge, US) [dblp]
  • Eva-Maria Brinkmann (Universität Münster, DE) [dblp]
  • Dorin Bucur (University Savoie Mont Blanc - Le Bourget-du-Lac, FR) [dblp]
  • Martin Buhmann (Universität Gießen, DE) [dblp]
  • Gene Cheung (National Institute of Informatics - Tokyo, JP) [dblp]
  • Nira Dyn (Tel Aviv University, IL) [dblp]
  • Gabriele Facciolo (ENS - Cachan, FR) [dblp]
  • Jalal Fadili (Caen University, FR) [dblp]
  • Irena Galic (University of Osijek, HR) [dblp]
  • Yann Gousseau (Telecom ParisTech, FR) [dblp]
  • Christine Guillemot (INRIA - Rennes, FR) [dblp]
  • Laurent Hoeltgen (BTU Cottbus, DE) [dblp]
  • Armin Iske (Universität Hamburg, DE) [dblp]
  • Claire Mantel (Technical University of Denmark - Lyngby, DK) [dblp]
  • Simon Masnou (University Claude Bernard - Lyon, FR) [dblp]
  • Peter Ochs (Universität Freiburg, DE) [dblp]
  • Pascal Peter (Universität des Saarlandes, DE) [dblp]
  • Gerlind Plonka-Hoch (Universität Göttingen, DE) [dblp]
  • Thomas Pock (TU Graz, AT) [dblp]
  • Daniela Rosca (TU of Cluj-Napoca, RO) [dblp]
  • Naoki Saito (University of California - Davis, US) [dblp]
  • Tomas Sauer (Universität Passau, DE) [dblp]
  • Carola-Bibiane Schönlieb (University of Cambridge, GB) [dblp]
  • Joan Serra Sagristà (Autonomus University of Barcelona, ES) [dblp]
  • Joachim Weickert (Universität des Saarlandes, DE) [dblp]
  • Hao-Min Zhou (Georgia Institute of Technology - Atlanta, US) [dblp]

Classification
  • computer graphics / computer vision
  • multimedia
  • optimization / scheduling

Keywords
  • inpainting
  • lossy data compression
  • image processing
  • interpolation
  • approximation
  • partial differential equations (PDEs)
  • optimisation
  • sparsity
  • radial basis functions