https://www.dagstuhl.de/20111

March 8 – 13 , 2020, Dagstuhl Seminar 20111

Tensor Computations: Applications and Optimization

Organizers

Paolo Bientinesi (University of Umeå, SE)
Furong Huang (University of Maryland – College Park, US)
Paul H. J. Kelly (Imperial College London, GB)
P. (Saday) Sadayappan (University of Utah – Salt Lake City, US)

Coordinators

David Ham (Imperial College London, GB)
Christian Lengauer (Köln, DE)

For support, please contact

Dagstuhl Service Team

Documents

Dagstuhl Report, Volume 10, Issue 3 Dagstuhl Report
Aims & Scope
List of Participants

Summary

This seminar was planned for 40 participants, but due to travel restrictions resulting from Covid-19, only 15 were able to attend - though several key talks were delivered via teleconferencing. As a result, the Seminar was very focused, and very productive. Some aspects were lost, perhaps in particular representation in-person of the full breadth of applications communities.

It was very evident from the presentations and lively discussions at the Seminar that the field of "Tensor Computations" is vibrant, multi-faceted, interdisciplinary, and fundamental to progress in a diverse range of important areas which are driving researchers in different fields to search for common foundations and common tools.

One of the communities with an interest in tensor computations can be described as "classical" computational science, focusing, for example, on partial differential equations in fluid dynamics, and electronic structure computations in chemistry and materials science. Tensor contractions have been identified as a powerful way of representing the computational structure in the architecture of compilers for domain-specific languages serving these communities. Exploiting this algebraic intermediate representation in the compiler has enabled important performance optimizations far beyond the scope of conventional compilers based on loop nests and polyhedral techniques.

Another major community is primarily concerned with tensor decomposition - finding low-rank approximations of tensors. This is fundamental to data analytics and machine learning applications. Tensor factorization also provides a powerful framework for deep learning and representation learning, and provides a promising strategy for weight compression in convolutional neural networks.

Tensor contractions, in the form of tensor networks, have enormous importance as a tool for understanding and computation in particle and quantum physics. Indeed mapping the connections between these topics, as exposed through the structure of the tensor network representation, offers an exciting frontier with the potential to underpin these different disciplines with common language and shared software.

The Seminar developed a focus, to some extent as a result of the participants able to attend, on tensor contractions, recognising that this provides a foundation for implementation of numerical methods for tensor decompositions. Revisiting this is a key topic to be addressed in following up this Seminar in the future.

A major focus for progress was identified in characterization of safety and correctness properties - ensuring that tensor contraction expressions are well-formed and meaningful. A related topic that was identified as critical concerns how structure is captured, represented and used. This is not only conceptually valuable, but provides a pathway to exploiting block, band and symmetry structure in generating efficient code.

An open question remains in how to capture, track and exploit the properties of tensors with unstructured (i.e. data-dependent) sparsity.

A key outcome from the Seminar was to recognize the massive replication of efforts in terms of software development. Many tools and libraries are being re-developed within different communities, while failing to share techniques and experience in high-performance implementation. The aim of this Seminar was to address this lack of cohesive, coherent community effort to develop computational building blocks. The results of this effort are being realized in the form of a "white paper", offering a manifesto for how to bridge the discipline divides and realize the potential for tensor computations in the future.

Summary text license
  Creative Commons BY 3.0 Unported license
  Paolo Bientinesi, David Ham, Furong Huang, Paul H. J. Kelly, Christian Lengauer and Saday Sadayappan

Related Dagstuhl Seminar

Classification

  • Data Structures / Algorithms / Complexity
  • Programming Languages / Compiler

Keywords

  • Compilers
  • Numerical methods
  • Linear algebra
  • Machine learning
  • Computational science

Documentation

In the series Dagstuhl Reports each Dagstuhl Seminar and Dagstuhl Perspectives Workshop is documented. The seminar organizers, in cooperation with the collector, prepare a report that includes contributions from the participants' talks together with a summary of the seminar.

 

Download overview leaflet (PDF).

Publications

Furthermore, a comprehensive peer-reviewed collection of research papers can be published in the series Dagstuhl Follow-Ups.

Dagstuhl's Impact

Please inform us when a publication was published as a result from your seminar. These publications are listed in the category Dagstuhl's Impact and are presented on a special shelf on the ground floor of the library.