Evidence About Programmers for Programming Language Design

Press Room

• Designing and evaluating programming languages: Dagstuhl trip report
  Blog entry by Andy J. Ko in Bits and Behaviour on Medium, February 9, 2018
• Dagstuhl reflections
  Blog entry by Amelia McNamara in Letters from a young statistician, February 11, 2018
• Dagstuhl Seminar 18061: Evidence about programmers for programming language design
  Blog entry by Brett Becker in CS0, February 11, 2018
• Tag Archive: dagstuhl
  Live blog entries by Felienne in Programming and Education and Programming Education, February 5 - 8, 2018

Summary

Programming languages underlie and have significant impact on software development, especially in terms of the ability of programmers to achieve their goals. Although designers of programming languages can already reason about the formal properties of their languages, few tools are available to assess the impact of design decisions on programmers and software engineers.

At Dagstuhl Seminar 18061, a diverse set of participants gathered to review the existing body of evidence about programmers that has implications on programming language design. Participants also reviewed existing research methods, such as eye tracking, that may help better understand the impact of language design decisions on programmers. Participants brainstormed a long list of possible research questions for investigation (§4), and then divided into working groups (§5) to focus on several areas of research interest, including novices, context switching and cognitive load, language features, emotional attachment to languages, and representativeness of subjects in studies. In each area, participants proposed research methods and questions that they felt would be valuable to address in the future. Then, the group discussed and prioritized these research questions.

The seminar included a discussion of the need for an evidence standard in empirical studies of programming languages, focusing on content of the evidence standard, adoption mechanisms, and criteria for what it might include in our field. Finally, the seminar concluded with a discussion of future directions for research, including a list of research questions that the participants were planning on collaborating on in the near future.