TOP
Search the Dagstuhl Website
Looking for information on the websites of the individual seminars? - Then please:
Not found what you are looking for? - Some of our services have separate websites, each with its own search option. Please check the following list:
Schloss Dagstuhl - LZI - Logo
Schloss Dagstuhl Services
Seminars
Within this website:
External resources:
  • DOOR (for registering your stay at Dagstuhl)
  • DOSA (for proposing future Dagstuhl Seminars or Dagstuhl Perspectives Workshops)
Publishing
Within this website:
External resources:
dblp
Within this website:
External resources:
  • the dblp Computer Science Bibliography


Dagstuhl Seminar 18061

Evidence About Programmers for Programming Language Design

( Feb 04 – Feb 09, 2018 )

(Click in the middle of the image to enlarge)

Permalink
Please use the following short url to reference this page: https://www.dagstuhl.de/18061

Organizers

Contact



Schedule

Motivation

In computer science as it stands today, there exists thousands of competing programming language products, with new ones (including both domain-specific and general purpose languages) being created regularly. These technologies are important, with applications ranging from educational or business tools to critical infrastructure in society or the military. While programming is by its nature mathematical, meaning that these products have a rigorous foundation to check that they work on a technical level, they are programmed by people that may have challenges and are prone to mistakes. While one might assume that existing communities, like the International Conference on Software Engineering (ICSE), Empirical Software Engineering (ESE), or the programming language conferences themselves (e.g., ICFP, OOPSLA, POPL) might be suitable venues for discussions about human-factors evidence to guide language design, this has not historically been the case. Notably, recent studies by Kaijanaho and others has shown that these venues contain almost no scientific data on the issue, with only 22 studies from the 1950s to 2012. Further, one might reasonably assume that smaller workshops, like the Psychology of Programmers Interest Group (PPIG), older workshops like Empirical Studies on Programmers (ESP), or newer workshops like PLATEAU cover this space, but rigorous investigations into their evidence practices show that these venues rarely include reports with rigorous scientific evidence.

In this Dagstuhl Seminar, we have four primary objectives. First, we are 1) striving to form a diverse community of scholars around programming language design based on evidence of programmer impacts. Given that top-tier venues in the area have not carefully investigated this long-standing and important issue in the field, we want to form a new community that will. Second, given that the design space for programming languages is very large, it makes sense to 2) create an organized set of priorities for new studies in the area along with 3) the evidence standard to be used. Notably, if a new community that contains evidence were to appear, it makes sense to create rigorous standards so that scholars can "test" each other's claims in replication, as opposed to the "trust" model that exists today. Finally, research can often make progress if it adheres to guiding principles, which brings us to our fourth objective, 4) to discuss or create theories that can guide the community in understanding and framing the evidence gathered over time. Historically, scholars have relied on somewhat subjective guiding theories. In contrast, we strive for theories and data that emphasize precision and objectivity.

During the seminar, participants will engage in a variety of activities. First, we will have sessions on the existing literature to provide attendees with historical context, a shared sense of what evidence exists, and methods that have been used to gather that evidence. Second, as there are considerable holes in the scholarly literature, we will generate lists of possible studies and try to prioritize them as a group. Third, to better understand what evidence standard could be used for these future studies, we will discuss evidence standards as they exist in other scholarly disciplines and their respective pros and cons. In summary, seminar participants will investigate the impact of programming language design on the productivity of programmers using systematic science with established standards.

Copyright Stefan Hanenberg, Brad A. Myers, Bonita Sharif, and Andreas Stefik

Press Room

Press Reviews


Summary

Programming languages underlie and have significant impact on software development, especially in terms of the ability of programmers to achieve their goals. Although designers of programming languages can already reason about the formal properties of their languages, few tools are available to assess the impact of design decisions on programmers and software engineers.

At Dagstuhl Seminar 18061, a diverse set of participants gathered to review the existing body of evidence about programmers that has implications on programming language design. Participants also reviewed existing research methods, such as eye tracking, that may help better understand the impact of language design decisions on programmers. Participants brainstormed a long list of possible research questions for investigation (§4), and then divided into working groups (§5) to focus on several areas of research interest, including novices, context switching and cognitive load, language features, emotional attachment to languages, and representativeness of subjects in studies. In each area, participants proposed research methods and questions that they felt would be valuable to address in the future. Then, the group discussed and prioritized these research questions.

The seminar included a discussion of the need for an evidence standard in empirical studies of programming languages, focusing on content of the evidence standard, adoption mechanisms, and criteria for what it might include in our field. Finally, the seminar concluded with a discussion of future directions for research, including a list of research questions that the participants were planning on collaborating on in the near future.

Copyright Michael Coblenz

Participants
  • Jonathan Aldrich (Carnegie Mellon University - Pittsburgh, US) [dblp]
  • Craig Anslow (Victoria University - Wellington, NZ) [dblp]
  • Ameer Armaly (University of Notre Dame, US) [dblp]
  • Johannes Bechberger (KIT - Karlsruher Institut für Technologie, DE)
  • Brett A. Becker (University College Dublin, IE) [dblp]
  • Andrew Begel (Microsoft Research - Redmond, US) [dblp]
  • Tanja Blascheck (INRIA Saclay - Orsay, FR) [dblp]
  • Neil Brown (King's College London, GB) [dblp]
  • Michael Coblenz (Carnegie Mellon University - Pittsburgh, US) [dblp]
  • Igor Crk (Southern Illinois Univ. - Edwardsville, US) [dblp]
  • John M. Daughtry (Google - Seattle, US) [dblp]
  • Fabian Deitelhoff (Fachhochschule Dortmund, DE) [dblp]
  • Rob DeLine (Microsoft Corporation - Redmond, US) [dblp]
  • Brian Dorn (University of Nebraska - Omaha, US) [dblp]
  • Andrew Duchowski (Clemson University, US) [dblp]
  • Scott Fleming (University of Memphis, US) [dblp]
  • Baker Franke (Code.org - Seattle, US) [dblp]
  • Reiner Hähnle (TU Darmstadt, DE) [dblp]
  • Matthias Hauswirth (University of Lugano, CH) [dblp]
  • Felienne Hermans (TU Delft, NL) [dblp]
  • Johannes C. Hofmeister (Universität Passau, DE) [dblp]
  • Ciera Jaspan (Google Inc. - Mountain View, US) [dblp]
  • Antti-Juhani Kaijanaho (University of Jyväskylä, FI) [dblp]
  • A. J. Ko (University of Washington - Seattle, US) [dblp]
  • Thomas D. LaToza (George Mason University - Fairfax, US) [dblp]
  • Andrew Macvean (Google - Seattle, US) [dblp]
  • Jonathan I. Maletic (Kent State University, US) [dblp]
  • Amelia A. McNamara (Smith College - Northampton, US) [dblp]
  • Briana B. Morrison (University of Nebraska - Omaha, US) [dblp]
  • Brad Myers (Carnegie Mellon University - Pittsburgh, US) [dblp]
  • Lutz Prechelt (FU Berlin, DE) [dblp]
  • Sibylle Schupp (TU Hamburg-Harburg, DE) [dblp]
  • Bonita Sharif (Youngstown State University, US) [dblp]
  • Andreas Stefik (Univ. of Nevada - Las Vegas, US) [dblp]
  • Walter F. Tichy (KIT - Karlsruher Institut für Technologie, DE) [dblp]
  • Phillip Merlin Uesbeck (Univ. of Nevada - Las Vegas, US) [dblp]
  • Lea Verou (MIT - Cambridge, US) [dblp]

Classification
  • programming languages / compiler
  • society / human-computer interaction
  • software engineering

Keywords
  • Human Factors
  • Programming Language Design
  • Community Evidence Standards
  • Domain Specific Languages