Automated programming refers to techniques that suggest newly written code, e.g., in the form of code completion tools. Techniques for automated programming include large language models that predict code based on natural language specifications of the intended behavior. The recent development of technologies like Codex and ChatGPT have made us examine the possibility of automated programming in the future.

Automated program repair refers to a suite of techniques for automated rectification of errors or vulnerabilities in programs. Automated program repair technologies were originally developed for reducing the debugging effort for manually written code. However, interestingly these techniques can be adapted to improve automatically generated code.

This Dagstuhl Seminar will explore the intersection of these two fields. We plan to discuss recent work and potential future work in the following directions

1. Apply program repair to fix code generated by code completion models. The code generated by large language models often has significant room for improvement in terms of correctness. We can also consider repair of automatically generated code in general.
2. Apply the generate-and-validate paradigm from program repair to the code completion problem.
3. Apply language model-based code generators to the program repair problem.
4. Use the ability of large language models to infer the intended behavior of code from natural language information embedding in the code.
5. In addition to predicting (fixed) code, generate evidence that the final code is trustworthy. Such evidence may take the form of tests generated along with the code, or other certificates obtained from formal reasoning.

This seminar will also help build greater connections and cross community understanding across formal verification researchers and researchers in program repair / automated programming. Formal verification or symbolic analysis techniques can be suitably adapted to give certificates of correctness, which may be useful in integrating large language model generated code into a software project. This can help in the coexistence of manually written code with auto-code from large language models. This possibility will be discussed and examined during the seminar.

Creative Commons BY 4.0

Claire Le Goues, Michael Pradel, Abhik Roychoudhury, and Shin Hwei Tan