The 4th International Conference on Code Quality

Sat 22 June 2024
Innopolis University, Russia

The 4th International Conference on Code Quality (ICCQ) was a one-day computer science event focused on static and dynamic analysis, program verification, programming languages design, software bug detection, and software maintenance. ICCQ was organized in cooperation with IEEE Computer Society and Innopolis University.

The Proceedings of ICCQ were published by IEEE Xplore.

ieee

Keynote

xin xia Xin Xia
SEAT Lab, Huawei, China
Prior to joining Huawei, Dr. Xia was an ARC DECRA Fellow and an assistant professor at the Faculty of Information Technology, Monash University (Australia). He received the ACM SIGSOFT Early Career Researcher Award in 2022. He co-authored multiple papers, among others, in TSE and TOSEM journals, in ICSE, ESEC/FSE, and ASE conferences.

Is LLM a Silver Bullet to Automated Software Engineering? youtube icon

Recently, Large Language Models (LLMs) such as GPT-3 and ChatGPT have attracted great attention from both academia and industry. They have shown substantial gains in solving a variety of problems ranging from Q&A to text summarization. Existing studies also found that some LLMs can be applied to the source code, such as code generation or debugging. However, their performance on various software engineering tasks has not been systematically investigated, and the understanding of LLMs is arguably fairly limited. Also, it is unclear how we can build software engineering capability based on LLM. In this talk, I will discuss the performance of LLMs on various software engineering tasks, including code generation, test generation, program repair, code translation, and documentation generation, and present some software engineering applications based on LLM (e.g., . vulnerability management, code search, and code idioms mining).

Program Committee

anatoly shalyto Anatoly Shalyto (Chair)
ITMO University

And in alphabetical order:

vahid alizadeh Vahid Alizadeh
DePaul University

vijay anant athavale Vijay Anant Athavale
Walchand Institute of Technology

elisa baniassad Elisa Baniassad
University of British Columbia

pietro braione Pietro Braione
University of Milano-Bicocca

stephen chang Stephen Chang
UMass Boston

bernhard egger Bernhard Egger
Seoul National University

eduardo fernandes Eduardo Fernandes
University of Southern Denmark

yusuke izawa Yusuke Izawa
Tokyo Institute of Technology

javier luis canovas izquierdo Javier Luis Cánovas Izquierdo
Universitat Oberta de Catalunya

ranjit jhala Ranjit Jhala
University of California, San Diego
ACM Fellow

tetsuo kamina Tetsuo Kamina
Oita University

narges khakpour Narges Khakpour
Newcastle University

kais klai Kais Klai
University Sorbonne Paris Nord

antoine-mine Antoine Miné
Sorbonne Université

mkaouer mohamed Mohamed Wiem Mkaouer
Rochester Institute of Technology

magnus myreen Magnus Myreen
Chalmers University of Technology

francis palma Francis Palma
University of New Brunswick

henrique rebelo Henrique Rebêlo
Universidade Federal de Pernambuco

ilya sergey Ilya Sergey
National University of Singapore

yudai tanabe Yudai Tanabe
Tokyo Institute of Technology

didier verna Didier Verna
EPITA

guannan wei Guannan Wei
Purdue University

philip wadler Philip Wadler
University of Edinburgh
ACM Fellow

vadim zaytsev Vadim Zaytsev
University of Twente

steve zdancewic Steve Zdancewic
University of Pennsylvania

Accepted Papers

We received 23 submissions. 10 papers were desk rejected. 4 papers were accepted. Each paper received at least three reviews from PC members.

Free Foil: Generating Efficient and Scope-Safe Abstract Syntax youtube icon pdf icon Nikolai Kudasov, Renata Shakirova, Egor Shalagin, and Karina Tyulebaeva

Handling bound identifiers correctly and efficiently is critical in implementations of compilers, proof assistants, and theorem provers. When choosing a representation for abstract syntax with binders, implementors face a trade-off between type safety with intrinsic scoping, efficiency, and generality. The “foil” by Maclaurin, Radul, and Paszke combines an efficient implementation of the Barendregt convention with intrinsic scoping through advanced type system features in Haskell, such as rank-2 polymorphism and generalized algebraic data types. Free scoped monads of Kudasov, on the other hand, combine intrinsic scoping with de Bruijn indices as nested data types with Sweirstra’s data types à la carte approach to allow generic implementation of algorithms such as higher-order unification. In this paper, we suggest two approaches of making the foil more affordable. First, we marry free scoped monads with the foil, allowing to generate efficient, type-safe, and generic abstract syntax representation with binders for any language given its second-order signature. Second, we provide Template Haskell functions that allow generating the scope-safe representation from a naïve one under some modest assumptions. The latter approach enables us to use existing tools like BNF Converter to very quickly prototype complete implementation of languages, including parsing, pretty-printing, and efficient intrinsically scoped abstract syntax. We demonstrate both approaches using λπ with pairs and patterns as our example object language. Finally, we provide benchmarks comparing our implementation against the foil, free scoped monads with nested de Bruijn indices, and some traditional implementations.

Replication of a Study about the Impact of Method Chaining and Comments on Readability and Comprehension youtube icon pdf icon Isabel Sampaio and Alberto Sampaio

It is well known that readability is an essential feature of quality code, and that many factors can affect the readability of code. This paper presents a conceptual replication of a previous experimental study that evaluated two practices associated with source code readability, namely, use of comments and method chaining. The replication study has the same research questions as the original study and involved almost all students of an OOP course of an informatics engineering program. The research process is presented, alongside decisions made and differences from the original study. Concerning perceived readability, our study found no significant differences between method chaining variants and between comment variants. The original study did find a significant difference for comments variants. In the case of comprehension, we found no significant differences between the method chaining variants but found a significant difference for an α = 0.1% among comments variants, both as opposed to the original study.

Exploring the Effectiveness of Abstract Syntax Tree Patterns for Algorithm Recognition youtube icon pdf icon Denis Neumüller, Florian Sihler, Raphael Straub, and Matthias Tichy

The automated recognition of algorithm implementations can support many software maintenance and re-engineering activities by providing knowledge about the concerns present in the code base. Moreover, recognizing inefficient algorithms like Bubble Sort and suggesting superior alternatives from a library can help in assessing and improving the quality of a system. Approaches from related work suffer from usability as well as scalability issues and their accuracy is not evaluated. In this paper, we investigate how well our approach based on the abstract syntax tree of a program performs for automatic algorithm recognition. To this end, we have implemented a prototype consisting of: A domain-specific language designed to capture the key features of an algorithm and used to express a search pattern on the abstract syntax tree, a matching algorithm to find these features, and an initial catalog of ready to use patterns. To create our search patterns we performed a web search using the algorithm name and described key features of the found reference implementations with our domain-specific language. We evaluate our prototype on a subset of the BigCloneEval benchmark containing algorithms like Fibonacci, Bubble Sort, and Binary Search. We achieve an average F1-score of 0.68 outperforming the large language model Codellama which attains 0.35. Additionally, we use multiple code clone detection tools as a baseline for comparison, achieving a recall of 0.54 while the best-performing tool reaches 0.20.

Assessing the Code Clone Detection Capability of Large Language Models youtube icon pdf icon Zixian Zhang and Takfarinas Saber

This study aims to assess the performance of two advanced Large Language Models (LLMs), GPT-3.5 and GPT-4, in the task of code clone detection. The evaluation involves testing the models on a variety of code pairs of different clone types and levels of similarity, sourced from two datasets: BigCloneBench (human-made) and GPTCloneBench (LLM-generated). Findings from the study indicate that GPT-4 consistently surpasses GPT-3.5 across all clone types. A correlation was observed between the GPTs’ accuracy at identifying code clones and code similarity, with both GPT models exhibiting low effectiveness in detecting the most complex Type-4 code clones. Additionally, GPT models demonstrate a higher performance identifying code clones in LLM-generated code compared to humans-generated code. However, they do not reach impressive accuracy. These results emphasize the imperative for ongoing enhancements in LLM capabilities, particularly in the recognition of code clones and in mitigating their predisposition towards self-generated code clones—which is likely to become an issue as software engineers are more numerous to leverage LLM-enabled code generation and code refactoring tools.

Partners

innopolis university Innopolis University

spbu St. Petersburg University

hse Higher School of Economics

Organizers

These people were making ICCQ 2024:

yegor bugayenko Yegor
Bugayenko
(Chair)

sergey belov Sergey
Belov

irina grashkina Irina
Grashkina

Tatiana Kudryashova Tatiana
Kudryashova

sergei prokhorov Sergei
Prokhorov