Probabilistic Numerical Methods – From Theory to Implementation
( 24. Oct – 29. Oct, 2021 )
- Philipp Hennig (Universität Tübingen, DE)
- Ilse Ipsen (North Carolina State University - Raleigh, US)
- Maren Mahsereci (Universität Tübingen, DE)
- Tim Sullivan (University of Warwick - Coventry, GB)
- Michael Gerke (für wissenschaftliche Fragen)
- Susanne Bach-Bernhard (für administrative Fragen)
- Multilevel Bayesian Quadrature - Li, Kaiyu; Giles, Daniel; Karvonen, Toni; Guillas, Serge; Briol, Francois-Xavier - Cornell University : arXiv.org, 2022. - 23 pp..
- ProbNum : Probabilistic Numerics in Python - Wenger, Jonathan; Krämer, Nicholas; Pförtner, Marvin; Schmidt, Jonathan; Bosch, Nathanael; Effenberger, Nina; Zenn, Johannes; Gessner, Alexandra; Hennig, Philipp; Mahsereci, Maren; Briol, Francois-Xavier; Karvonen, Toni - Cornell University : arXiv.org, 2021. - 7 pp..
Numerical methods provide the computational foundation of science and power automated data analysis and inference in its contemporary form of machine learning. Probabilistic numerical methods aim to explicitly represent uncertainty resulting from limited computational resources and imprecise inputs in these models. In recent years, algorithms arising from this formalism have repeatedly shown that they can enrich and improve upon classic methods in tasks where hyperparameter adaptation is not straightforward; where computational stochasticity and low precision play a prominent role; where limited data make uncertainty quantification a key functionality; where related problems have to be solved repeatedly; and where extreme scale or tight budgets call for rough approximations at low cost.
With theoretical analysis well underway, software development is now a key next step to wide-spread success.
From Lapack to SciPy to PyTorch, open-source software libraries have driven scientific advancement in their respective domains. Indeed, the lack of high-quality implementations of probabilistic numerical methods is increasingly a bottleneck for our field. Addressing this issue is a main goal of the seminar. We recently started a community effort to develop an open-source framework named ProbNum. You can track its progress at https://probnum.readthedocs.io/.
The goals of this Dagstuhl Seminar are thus two-fold. First, we want to rekindle our community spirit. The meeting will provide the opportunity to update others on your own research and to discuss new directions and ideas together, after the deadening silence of the Covid years. We have invited a diverse group of people like yourself, hailing from CS/AI/ML, from statistics, optimization, and from numerical analysis. On the other hand, the seminar will act as a milestone for the ProbNum software: we hope some of you may want to join the development effort to shape functionality, structure and interface of the library ahead of a first release, and an associated publication.
Exceptional times require exceptional measures. Dagstuhl seminars have always been about bringing people together in person, in a deliberately remote place. This year, however, some of this interaction will have to take place remotely , after all: All scientific talks of the seminar will be streamed online and even the speakers will not all be present at the castle (a silver lining is that we can increase the number of online participants if necessary). But the last year has also shown painfully that some forms of collaboration do not work without close personal interaction. We thus ask you to decide for yourself: If you are primarily interested in attending the scientific talks – or in giving one of them – we understand if you prefer to attend virtually. The talks will take up only limited hours of the day, typically in the afternoon, local time. If you want to be directly involved in small-group discussions, also after hours, and particularly in the software development effort, we hope you will choose to attend in person. This is, of course, assuming that the course of the pandemic will allow us, legally and practically, to actually meet at the castle in late October. Please let us and Dagstuhl admin know as early as possible if you plan to participate virtually, as this will allow us to invite further people from our waiting list to the castle itself.
Probabilistic Numerical algorithms frame a numerical task as a statistical inference problem, expressed in the language of probabilistic inference. The key advantage of this approach is that it allows quantification of uncertainty arising from finite computational resources, and to combine thus with other forms of uncertainty, in particular those arising from model misspecification, finite observational data, and measurement errors. In recent years, algorithms arising from this formalism have repeatedly shown that they can enrich and improve upon classic methods in tasks where
- hyperparameter adaptation is not straightforward;
- computational stochasticity and low precision play a prominent role;
- limited data make uncertainty quantification a key functionality;
- related problems have to be solved repeatedly;
- and where extreme scale or tight budgets call for rough approximations at low cost.
Probabilistic Numerics lies at the intersection of machine learning within computer science and numerical analysis within applied mathematics. This interdisciplinary nature raises an exciting and challenging set of viewpoints with regards to goals and challenges of the field. The first goal of this seminar was to rekindle our community following two years of pandemic lockdown, to provide an opportunity to update others on one's own research, and to discuss new directions and ideas together. We were lucky to assemble - both in-person and remote - a diverse group of people from computer science, machine learning, from statistics, optimization, and from numerical analysis.
The second key goal of this seminar was to take the next step in the development of probabilistic numerical methods by focusing on their implementation. From Lapack to SciPy to PyTorch, open-source software libraries have driven scientific advancement in their respective domains. Such libraries accelerate research, enable benchmarking and promote the development of new methods via rapid prototyping. Most importantly, they are a necessary step towards their use in applications. While considerable advances in the theoretical understanding of probabilistic numerical methods have been made, the lack of high-quality implementations is holding back their adoption. In response, we recently started a community effort to develop an open-source framework named ProbNum (http://probnum.org).
A central theme of Dagstuhl seminars is the open, collaborative atmosphere with a focus on new ideas and tangible outcomes as opposed to existing work. The seminar stimulated multiple focussed discussions around software and additions to ProbNum. Examples included how to best include automatic differentiation functionality, or how to expand the package's Bayesian quadrature functionalities. It also set the starting point for potential new research collaborations on probabilistic linear solvers and probabilistic numerical methods for PDEs. Even at this point, shortly after the seminar's conclusion, two tangible products are already available: the seminar's participants jointly created a Probabilistic Numerics Wikipedia page https://en.wikipedia.org/wiki/Probabilistic_numerics, and the implementation sessions culminated with a preprint for the community library ProbNum .
As the organizers we want to thank all participants, both physical and virtual, for their interesting talks, the stimulating discussions and the collaborative overall atmosphere. We also want to thank Schloss Dagstuhl for their technical support that made the challenging hybrid format possible.
- Jonathan Wenger, Nicholas Krämer, Marvin Pförtner, Jonathan Schmidt, Nathanael Bosch, Nina Effenberger, Johannes Zenn, Toni Karvonen Alexandra Gessner, François-Xavier Briol, Maren Mahsereci, and Philipp Hennig. ProbNum: Probabilistic numerics in Python. arxiv preprint, 2021. URL http://arxiv.org/abs/2112.02100
- Simon Bartels (University of Copenhagen, DK) [dblp]
- Nathanael Bosch (Universität Tübingen, DE) [dblp]
- François-Xavier Briol (University College London, GB) [dblp]
- Maurizio Filippone (EURECOM - Biot, FR) [dblp]
- Giacomo Garegnani (EPFL - Lausanne, CH) [dblp]
- Roman Garnett (Washington University - St. Louis, US) [dblp]
- Alexandra Gessner (Universität Tübingen, DE)
- Philipp Hennig (Universität Tübingen, DE) [dblp]
- Toni Karvonen (University of Helsinki, FI) [dblp]
- Peter Nicholas Krämer (Universität Tübingen, DE) [dblp]
- Maren Mahsereci (Universität Tübingen, DE) [dblp]
- Katharina Ott (Bosch Center for AI - Renningen, DE)
- Marvin Pförtner (Universität Tübingen, DE)
- Jonathan Schmidt (Universität Tübingen, DE)
- Tomas Teren (TU Dresden, DE)
- Filip Tronarp (Universität Tübingen, DE) [dblp]
- Jonathan Wenger (Universität Tübingen, DE) [dblp]
- Stephen Wright (University of Wisconsin-Madison, US) [dblp]
- Oksana Chkrebtii (Ohio State University, US) [dblp]
- Jon Cockayne (University of Southampton, GB) [dblp]
- Yuhan Ding (Illinois Institute of Technology - Chicago, US)
- Matthew Fisher (Newcastle University, GB) [dblp]
- Fred J. Hickernell (Illinois Institute of Technology - Chicago, US) [dblp]
- Nick Higham (Manchester University, GB)
- Ilse Ipsen (North Carolina State University - Raleigh, US) [dblp]
- Motonobu Kanagawa (EURECOM - Biot, FR) [dblp]
- Hans Kersting (INRIA - Paris, FR) [dblp]
- Tadashi Matsumoto (University of Warwick - Coventry, GB)
- Michael McKerns (Los Alamos National Laboratory, US) [dblp]
- Masha Naslidnyk (Amazon Research Cambridge, GB) [dblp]
- Chris Oates (Newcastle University, GB) [dblp]
- Michael A. Osborne (University of Oxford, GB) [dblp]
- Houman Owhadi (California Institute of Technology - Pasadena, US) [dblp]
- Andrei Paleyes (University of Cambridge, GB)
- Kamran Pentland (University of Warwick - Coventry, GB)
- Geoff Pleiss (Columbia University - New York, US) [dblp]
- Jagadeeswaran Rathinavel (Illinois Institute of Technology - Chicago, US)
- Timothy Reid (North Carolina State University - Raleigh, US)
- Simo Särkkä (Aalto University, FI) [dblp]
- Florian Schäfer (Georgia Institute of Technology - Atlanta, US) [dblp]
- Aleksei Sorokin (Illinois Institute of Technology - Chicago, US)
- Tim Sullivan (University of Warwick - Coventry, GB) [dblp]
- Aretha Teckentrup (University of Edinburgh, GB) [dblp]
- Onur Teymur (University of Kent, GB)
- Zi Wang (Google - Cambridge, US) [dblp]
- Machine Learning
- Mathematical Software
- Numerical Analysis
- Machine Learning
- Numerical Analysis
- Probabilistic Numerics