Sat 22 June 2024
Innopolis University, Russia
The 4th International Conference on Code Quality (ICCQ) was a one-day computer science event focused on static and dynamic analysis, program verification, programming languages design, software bug detection, and software maintenance. ICCQ was organized in cooperation with Innopolis University and IEEE Computer Society.
Keynote
Xin Xia
SEAT Lab, Huawei, China
Prior to joining Huawei, Dr. Xia was an ARC DECRA Fellow and an assistant professor at the Faculty of Information Technology, Monash University (Australia). He received the ACM SIGSOFT Early Career Researcher Award in 2022. He co-authored multiple papers, among others, in TSE and TOSEM journals, in ICSE, ESEC/FSE, and ASE conferences.
Is LLM a Silver Bullet to Automated Software Engineering?
Recently, Large Language Models (LLMs) such as GPT-3 and ChatGPT have attracted great attention from both academia and industry. They have shown substantial gains in solving a variety of problems ranging from Q&A to text summarization. Existing studies also found that some LLMs can be applied to the source code, such as code generation or debugging. However, their performance on various software engineering tasks has not been systematically investigated, and the understanding of LLMs is arguably fairly limited. Also, it is unclear how we can build software engineering capability based on LLM. In this talk, I will discuss the performance of LLMs on various software engineering tasks, including code generation, test generation, program repair, code translation, and documentation generation, and present some software engineering applications based on LLM (e.g., . vulnerability management, code search, and code idioms mining).
Program Committee
Anatoly Shalyto (Chair)
ITMO University
And in alphabetical order:
Vahid Alizadeh
DePaul University
Vijay Anant Athavale
Walchand Institute of Technology
Elisa Baniassad
University of British Columbia
Pietro Braione
University of Milano-Bicocca
Bernhard Egger
Seoul National University
Eduardo Fernandes
University of Southern Denmark
Yusuke Izawa
Tokyo Institute of Technology
Javier Luis Cánovas Izquierdo
Universitat Oberta de Catalunya
Ranjit Jhala
University of California, San Diego
ACM Fellow
Narges Khakpour
Newcastle University
Kais Klai
University Sorbonne Paris Nord
Antoine Miné
Sorbonne Université
Mohamed Wiem Mkaouer
Rochester Institute of Technology
Magnus Myreen
Chalmers University of Technology
Francis Palma
University of New Brunswick
Henrique Rebêlo
Universidade Federal de Pernambuco
Ilya Sergey
National University of Singapore
Yudai Tanabe
Tokyo Institute of Technology
Philip Wadler
University of Edinburgh
ACM Fellow
Vadim Zaytsev
University of Twente
Steve Zdancewic
University of Pennsylvania
Accepted Papers
We received 23 submissions. 10 papers were desk rejected. 4 papers were accepted. Each paper received at least three reviews from PC members.
Free Foil: Generating Efficient and Scope-Safe Abstract Syntax
Nikolai Kudasov,
Renata Shakirova,
Egor Shalagin,
and
Karina Tyulebaeva
Handling bound identifiers correctly and efficiently is critical in implementations of compilers, proof assistants, and theorem provers. When choosing a representation for abstract syntax with binders, implementors face a trade-off between type safety with intrinsic scoping, efficiency, and generality. The “foil” by Maclaurin, Radul, and Paszke combines an efficient implementation of the Barendregt convention with intrinsic scoping through advanced type system features in Haskell, such as rank-2 polymorphism and generalized algebraic data types. Free scoped monads of Kudasov, on the other hand, combine intrinsic scoping with de Bruijn indices as nested data types with Sweirstra’s data types à la carte approach to allow generic implementation of algorithms such as higher-order unification. In this paper, we suggest two approaches of making the foil more affordable. First, we marry free scoped monads with the foil, allowing to generate efficient, type-safe, and generic abstract syntax representation with binders for any language given its second-order signature. Second, we provide Template Haskell functions that allow generating the scope-safe representation from a naïve one under some modest assumptions. The latter approach enables us to use existing tools like BNF Converter to very quickly prototype complete implementation of languages, including parsing, pretty-printing, and efficient intrinsically scoped abstract syntax. We demonstrate both approaches using λπ with pairs and patterns as our example object language. Finally, we provide benchmarks comparing our implementation against the foil, free scoped monads with nested de Bruijn indices, and some traditional implementations.
Replication of a Study about the Impact of Method Chaining and Comments on Readability and Comprehension
Isabel Sampaio
and
Alberto Sampaio
It is well known that readability is an essential feature of quality code, and that many factors can affect the readability of code. This paper presents a conceptual replication of a previous experimental study that evaluated two practices associated with source code readability, namely, use of comments and method chaining. The replication study has the same research questions as the original study and involved almost all students of an OOP course of an informatics engineering program. The research process is presented, alongside decisions made and differences from the original study. Concerning perceived readability, our study found no significant differences between method chaining variants and between comment variants. The original study did find a significant difference for comments variants. In the case of comprehension, we found no significant differences between the method chaining variants but found a significant difference for an α = 0.1% among comments variants, both as opposed to the original study.
Exploring the Effectiveness of Abstract Syntax Tree Patterns for Algorithm Recognition
Denis Neumüller,
Florian Sihler,
Raphael Straub,
and
Matthias Tichy
The automated recognition of algorithm implementations can support many software maintenance and re-engineering activities by providing knowledge about the concerns present in the code base. Moreover, recognizing inefficient algorithms like Bubble Sort and suggesting superior alternatives from a library can help in assessing and improving the quality of a system. Approaches from related work suffer from usability as well as scalability issues and their accuracy is not evaluated. In this paper, we investigate how well our approach based on the abstract syntax tree of a program performs for automatic algorithm recognition. To this end, we have implemented a prototype consisting of: A domain-specific language designed to capture the key features of an algorithm and used to express a search pattern on the abstract syntax tree, a matching algorithm to find these features, and an initial catalog of ready to use patterns. To create our search patterns we performed a web search using the algorithm name and described key features of the found reference implementations with our domain-specific language. We evaluate our prototype on a subset of the BigCloneEval benchmark containing algorithms like Fibonacci, Bubble Sort, and Binary Search. We achieve an average F1-score of 0.68 outperforming the large language model Codellama which attains 0.35. Additionally, we use multiple code clone detection tools as a baseline for comparison, achieving a recall of 0.54 while the best-performing tool reaches 0.20.
Assessing the Code Clone Detection Capability of Large Language Models
Zixian Zhang
and
Takfarinas Saber
This study aims to assess the performance of two advanced Large Language Models (LLMs), GPT-3.5 and GPT-4, in the task of code clone detection. The evaluation involves testing the models on a variety of code pairs of different clone types and levels of similarity, sourced from two datasets: BigCloneBench (human-made) and GPTCloneBench (LLM-generated). Findings from the study indicate that GPT-4 consistently surpasses GPT-3.5 across all clone types. A correlation was observed between the GPTs’ accuracy at identifying code clones and code similarity, with both GPT models exhibiting low effectiveness in detecting the most complex Type-4 code clones. Additionally, GPT models demonstrate a higher performance identifying code clones in LLM-generated code compared to humans-generated code. However, they do not reach impressive accuracy. These results emphasize the imperative for ongoing enhancements in LLM capabilities, particularly in the recognition of code clones and in mitigating their predisposition towards self-generated code clones—which is likely to become an issue as software engineers are more numerous to leverage LLM-enabled code generation and code refactoring tools.
Partners
Organizers
These people were making ICCQ 2024:
Yegor
Bugayenko (Chair)
Irina
Grashkina
Tatiana
Kudryashova
