GraphPyRec: A novel graph-based approach for fine-grained Python code recommendation

• Published in: Science of computer programming Vol. 238; p. 103166
• Main Authors: Zong, Xing, Zheng, Shang, Zou, Haitao, Yu, Hualong, Gao, Shang
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.12.2024
• Subjects: Abstract syntax tree; Code graph; Code recommendation; Software development; Software engineering; Code graph; Code recommendation; Software development; Abstract syntax tree; Software engineering
• ISSN: 0167-6423
• DOI: 10.1016/j.scico.2024.103166

Artificial intelligence has been widely applied in software engineering areas such as code recommendation. Significant progress has been made in code recommendation for static languages in recent years, but it remains challenging for dynamic languages like Python as accurately determining data flows before runtime is difficult. This limitation hinders data flow analysis, affecting the performance of code recommendation methods that rely on code analysis. In this study, a graph-based Python recommendation approach (GraphPyRec) is proposed by converting source code into a graph representation that captures both semantic and dynamic information. Nodes represent semantic information, with unique rules defined for various code statements. Edges depict control flow and data flow, utilizing a child-sibling-like process and a dedicated algorithm for data transfer extraction. Alongside the graph, a bag of words is created to include essential names, and a pre-trained BERT model transforms it into vectors. These vectors are integrated into a Gated Graph Neural Network (GGNN) process of the code recommendation model, enhancing its effectiveness and accuracy. To validate the proposed method, we crawled over a million lines of code from GitHub. Experimental results show that GraphPyRec outperforms existing mainstream Python code recommendation methods, achieving Top-1, 5, and 10 accuracy rates of 68.52%, 88.92%, and 94.05%, respectively, along with a Mean Reciprocal Rank (MRR) of 0.772. •Custom node definition rules are proposed to achieve a full representation of the code semantics.•A data flow analysis algorithm is given to analyze variable transfer pathways.•A graph-based Python code recommendation model was constructed based on GGNN and BERT.•A Python code corpus was constructed by collecting a million lines of Python programs from GitHub.