Dagstuhl Seminar 25112
PETs and AI: Privacy Washing and the Need for a PETs Evaluation Framework
( Mar 09 – Mar 14, 2025 )
(Click in the middle of the image to enlarge)
Permalink
Please use the following short url to reference this page:
https://www.dagstuhl.de/25112
Organizers
- Emiliano De Cristofaro (University of California - Riverside, US)
- Kris Shrishak (Irish Council for Civil Liberties - Dublin, IE)
- Thorsten Strufe (KIT - Karlsruher Institut für Technologie, DE)
- Carmela Troncoso (MPI-SP - Bochum, DE)
Contact
- Michael Gerke (for scientific matters)
- Simone Schilke (for administrative matters)
Shared Documents
- Dagstuhl Materials Page (Use personal credentials as created in DOOR to log in)
Schedule
Privacy is a fundamental human right. Article 12 of the Universal Declaration of Human Rights (UDHR) states that everyone has the right to potection from interference with their privacy. One part of protecting people's privacy is data protection. Laws such as the EU's General Data Protection Regulation (GDPR) have been drafted to protect personal data, which can be exploited to interfere with people's private life. Numerous countries around the world have adopted laws similar to the GDPR. These laws along with an increased awareness of personal data collection have contributed to the appeal of technological solutions known broadly as privacy enhancing technologies (PETs).
The premise of PETs is that these techniques allow data processing while protecting the underlying data from being revealed unnecessarily. PETs make it possible to analyse data from multiple sources without having to see the data. There are two major kinds of PETs: one that offers input privacy and another that offers output privacy. Input privacy allows different people to pour-in their individual data to combine and generate an insight, while no one learns anyone else's individual data. For example, a group of friends can learn who earns the highest without revealing their individual salary to each other. Techniques such as homomorphic encryption and secure multiparty computation (SMPC) fall into this category. These powerful techniques allow two or more entities to compute an agreed upon function on encrypted data. They are useful when the participating entities do not trust each other with their private inputs, but see mutual benefit in the output of the function. Output privacy allows for the release of aggregate data and statistical information while preventing the identification of individuals. Techniques such as differential privacy fall into this category. In many practical use cases, both input and output privacy is desired, and these techniques are combined.
One such use case is AI and in particular machine learning (ML). A huge volume of data is a key component of some of the machine learning techniques, especially those relying on deep neural networks. Personal data and those with sensitive attributes are also used to develop AI models. However, this has contributed to privacy risks. There are a range of attacks in the literature that aim to extract personal data from trained models. PETs have been proposed as the way to protect the functionality of AI while protecting against these privacy attacks. In fact, an entire research field known as privacy-preserving machine learning (PPML) has been formed. PPML incorporates various PETs techniques at various stages of the machine learning to (a) train over encrypted data (e.g., with homomorphic encryption or SMPC), (b) anonymize training process (e.g., DP-SGD), and (c) protect the outputs using differential privacy.
Despite the abundance of works in the area of PETs, AI, and their intersection, there are many remaining challenges. Addressing these challenges is crucial to understand the drawbacks and to reap the benefits of PETs. A range of research questions in Computer Science (protocol design, privacy guarantees, feasibility, scalability, efficiency, etc.) need to be addressed. There are also questions that are interdisciplinary and require expertise from NGOs, ethicists, policy making, law, and regulators. And these research questions are not merely to satisfy academic curiosity but have practical ramifications. They could affect policy making and the work of regulators.
In this Dagstuhl Seminar, a multidisciplinary group of computer science and legal academics and practitioners from industry, human rights groups, and regulators discussed two challenges:
- Privacy washing through PETs: In the recent years, PETs have been used in surveillance applications as in the case of Apple’s proposed (and then retracted) approach to scan images on people’s phones when uploading photos to iCloud. They have also been used in applications where the personal data is seemingly protected but the privacy threats faced by people are amplified, for example in targeted advertising. Such applications show that PETs can be used for “privacy washing”. At the heart of the issue is that most works fail to protect against the interference with privacy as laid down in Article 12 of the UDHR. These works are agnostic to the application context or too generic or limited to the cryptographic protocol without considering the privacy threats due to the system where it is embedded. The imbalances and asymmetries of power between the stakeholders, the role of infrastructures and their providers, and the control of the computing infrastructure are not accounted for. Technical measures to protect data are discussed as being equivalent to privacy, when they are not. Privacy violations can take many other forms including economic and discrimination harms. When the goal of the application is to harm privacy, such technical measures to protect data cannot protect the interference with privacy. The threat models in the literature are inadequate, and thus, systems designed under such models continue to cause privacy harms.
- Evaluation framework to detect privacy washing: If PETs are to protect against interference with privacy, as laid down in the UDHR, then we require standard evaluation methods and frameworks that allow us to compare the degree of protection. While the literature is filled with ways to measure PETs, they are hard to compare. Limitations of PETs should be well documented so that privacy washing through PETs is stopped. A lack of an independent evaluation framework allows privacy washing. Addressing this challenge is timely and this seminar took the initial steps towards an evaluation framework.
Seminar Structure
Since the participants came from diverse backgrounds ranging from different topics in computer science to legal and regulatory work, the seminar began with several introductory talks and two panel discussions to bring everyone up to speed. Then, we brainstormed in small groups about all the aspects that could influence whether the deployment of a PET could be considered privacy washing. We subsequently grouped these aspects into four topics: Functionality and Framing, Infrastructure for PETs, Accountability, and Detection of Fake PETs. We split the group into four subgroups to discuss these aspects further and develop criteria by which to evaluate the deployment of a PET leading to a vast catalogue of factors that influence the efficacy of PET deployments. During the plenary meetings after group discussions, the rapporteurs from each group shared the progress made during the group discussions. Finally, we spent the remaining time to merge the results of the four subgroups into a draft for a position paper. The position paper describes what privacy washing is, who is involved in its deployment, who can be affected by it, and the considerations that help to detect privacy washing in deployed systems.
Creative Commons BY 4.0
Emiliano De Cristofaro, Kris Shrishak, Thorsten Strufe, and Carmela Troncoso
An increased awareness of personal data collection and of data protection regulations has contributed to the appeal of privacy enhancing technologies (PETs). The premise of PETs is that the techniques such as syntactic mechanisms for statistical disclosure control, differential privacy, homomorphic encryption and secure multiparty computation facilitate data processing while protecting individuals from unwanted disclosures. PETs have been proposed as the way to protect the functionality of artificial intelligence (AI) while protecting against these privacy attacks. This field, known as privacy-preserving machine learning (PPML), incorporates various PETs techniques at various stages of the machine learning process to (a) train over encrypted data, (b) anonymize the training process, and (c) protect the outputs using differential privacy.
Despite the abundance of works in the area of PETs, AI, and their intersection, there are many remaining challenges. Addressing these challenges is crucial to understand the drawbacks and to reap the benefits of PETs. Recent works have raised concerns about efficacy and deployment of PETs, observing that fundamental rights of people are continually being harmed, including, paradoxically, privacy. PETs have been used in surveillance applications and as a privacy washing tool.
How PETs address privacy threats needs a rethink. Protecting personal data is only a first step, and is insufficient in many cases to protect people from interference with their privacy. Furthermore, an independent evaluation framework is required to assess the level of privacy protection offered by deployed PETs solutions and to protect against privacy washing. However, most computer scientists are not well equipped to address social problems on their own. Thus, this Dagstuhl Seminar aims to bring together a group of computer science and legal scholars working on privacy and AI along with industry, policy experts, and regulators to explore the role of PETs and the challenge of private and accountable AI.
Emiliano De Cristofaro, Kris Shrishak, Thorsten Strufe, and Carmela Troncoso
Please log in to DOOR to see more details.
- Frederik Armknecht (Universität Mannheim, DE) [dblp]
- Aurélien Bellet (INRIA - Montpellier, FR) [dblp]
- Robin Berjon (Princeton, US) [dblp]
- Asia Biega (MPI-SP - Bochum, DE) [dblp]
- Paul Comerford (Information Commissioner’s Office - Wilmslow, GB)
- Ana-Maria Cretu (EPFL - Lausanne, CH) [dblp]
- Emiliano De Cristofaro (University of California - Riverside, US) [dblp]
- Yves-Alexandre de Montjoye (Imperial College London, GB) [dblp]
- Sébastien Gambs (UQAM - Montreal, CA) [dblp]
- Georgi Ganev (University College London, GB) [dblp]
- Patricia Guerra-Balboa (KIT - Karlsruher Institut für Technologie, DE) [dblp]
- Johanna Gunawan (Maastricht University, NL) [dblp]
- Seda F. Gürses (TU Delft, NL) [dblp]
- Bailey Kacsmar (University of Alberta - Edmonton, CA) [dblp]
- Felix Morsbach (KIT - Karlsruher Institut für Technologie, DE) [dblp]
- Lucy Qin (Georgetown University - Washington, DC, US) [dblp]
- Kris Shrishak (Irish Council for Civil Liberties - Dublin, IE) [dblp]
- Thorsten Strufe (KIT - Karlsruher Institut für Technologie, DE) [dblp]
- Hinako Sugiyama (University of California - Irvine, US)
- Vanessa Teague (Australian National University - Acton, AU) [dblp]
- Carmela Troncoso (MPI-SP - Bochum, DE) [dblp]
- Michael Veale (University College London, GB) [dblp]
- Rui-Jie Yew (Brown University - Providence, US)
Classification
- Computers and Society
- Cryptography and Security
- Machine Learning
Keywords
- Privacy
- Privacy Enhancing Technologies
- Machine Learning
- Artificial Intelligence
- Interdisciplinary
