04. – 09. Februar 2018, Dagstuhl Seminar 18061

Evidence About Programmers for Programming Language Design


Stefan Hanenberg (Universität Duisburg-Essen, DE)
Brad A. Myers (Carnegie Mellon University – Pittsburgh, US)
Bonita Sharif (Youngstown State University, US)
Andreas Stefik (Univ. of Nevada – Las Vegas, US)

Auskunft zu diesem Dagstuhl Seminar erteilen

Annette Beyer zu administrativen Fragen

Andreas Dolzmann zu wissenschaftlichen Fragen


Programm des Dagstuhl Seminars (Hochladen)

(Zum Einloggen bitte Seminarnummer und Zugangscode verwenden)


In computer science as it stands today, there exists thousands of competing programming language products, with new ones (including both domain-specific and general purpose languages) being created regularly. These technologies are important, with applications ranging from educational or business tools to critical infrastructure in society or the military. While programming is by its nature mathematical, meaning that these products have a rigorous foundation to check that they work on a technical level, they are programmed by people that may have challenges and are prone to mistakes. While one might assume that existing communities, like the International Conference on Software Engineering (ICSE), Empirical Software Engineering (ESE), or the programming language conferences themselves (e.g., ICFP, OOPSLA, POPL) might be suitable venues for discussions about human-factors evidence to guide language design, this has not historically been the case. Notably, recent studies by Kaijanaho and others has shown that these venues contain almost no scientific data on the issue, with only 22 studies from the 1950s to 2012. Further, one might reasonably assume that smaller workshops, like the Psychology of Programmers Interest Group (PPIG), older workshops like Empirical Studies on Programmers (ESP), or newer workshops like PLATEAU cover this space, but rigorous investigations into their evidence practices show that these venues rarely include reports with rigorous scientific evidence.

In this Dagstuhl Seminar, we have four primary objectives. First, we are 1) striving to form a diverse community of scholars around programming language design based on evidence of programmer impacts. Given that top-tier venues in the area have not carefully investigated this long-standing and important issue in the field, we want to form a new community that will. Second, given that the design space for programming languages is very large, it makes sense to 2) create an organized set of priorities for new studies in the area along with 3) the evidence standard to be used. Notably, if a new community that contains evidence were to appear, it makes sense to create rigorous standards so that scholars can "test" each other's claims in replication, as opposed to the "trust" model that exists today. Finally, research can often make progress if it adheres to guiding principles, which brings us to our fourth objective, 4) to discuss or create theories that can guide the community in understanding and framing the evidence gathered over time. Historically, scholars have relied on somewhat subjective guiding theories. In contrast, we strive for theories and data that emphasize precision and objectivity.

During the seminar, participants will engage in a variety of activities. First, we will have sessions on the existing literature to provide attendees with historical context, a shared sense of what evidence exists, and methods that have been used to gather that evidence. Second, as there are considerable holes in the scholarly literature, we will generate lists of possible studies and try to prioritize them as a group. Third, to better understand what evidence standard could be used for these future studies, we will discuss evidence standards as they exist in other scholarly disciplines and their respective pros and cons. In summary, seminar participants will investigate the impact of programming language design on the productivity of programmers using systematic science with established standards.

  Creative Commons BY 3.0 DE
  Stefan Hanenberg, Brad A. Myers, Bonita Sharif, and Andreas Stefik


  • Programming Languages / Compiler
  • Society / Human-computer Interaction
  • Software Engineering


  • Human Factors
  • Programming Language Design
  • Community Evidence Standards
  • Domain Specific Languages


Bücher der Teilnehmer 

Buchausstellung im Erdgeschoss der Bibliothek

(nur in der Veranstaltungswoche).


In der Reihe Dagstuhl Reports werden alle Dagstuhl-Seminare und Dagstuhl-Perspektiven-Workshops dokumentiert. Die Organisatoren stellen zusammen mit dem Collector des Seminars einen Bericht zusammen, der die Beiträge der Autoren zusammenfasst und um eine Zusammenfassung ergänzt.


Download Übersichtsflyer (PDF).


Es besteht weiterhin die Möglichkeit, eine umfassende Kollektion begutachteter Arbeiten in der Reihe Dagstuhl Follow-Ups zu publizieren.

Dagstuhl's Impact

Bitte informieren Sie uns, wenn eine Veröffentlichung ausgehend von
Ihrem Seminar entsteht. Derartige Veröffentlichungen werden von uns in der Rubrik Dagstuhl's Impact separat aufgelistet  und im Erdgeschoss der Bibliothek präsentiert.