-
Automatic Semantic Augmentation of Language Model Prompts (for Code Summarization), (ICSE2024)
- Abstract: Large Language Models (LLM) are a new class of computation engines, "programmed" via prompt engineering. Researchers are still learning how to best "program" these LLMs to help developers. We start with the intuition that developers tend to consciously and unconsciously collect semantics facts, from the code, while working. Mostly these are shallow, simple facts arising from a quick read. For a function, such facts might include parameter and local variable names, return expressions, simple pre-...
- Labels: static analysis, code summarization, agent design, prompt strategy, retrieval-augmented generation
-
DroidCoder: Enhanced Android Code Completion with Context-Enriched Retrieval-Augmented Generation, (ASE2024)
- Abstract: Android is the most popular mobile operating system. However, Android development requires extensive coding, especially for unique features such as lifecycle callbacks and UI widgets. Existing code completion methods typically utilize Retrieval-Augmented Generation (RAG) to provide contextual information for pre-trained code large language models (Code LLMs) to perform completion. Despite considerable progress in these methods, their effectiveness in Android development remains limited. This is ...
- Labels: code generation, code completion, agent design, prompt strategy, retrieval-augmented generation
-
EvoR: Evolving Retrieval for Code Generation, (EMNLP2024)
- Abstract: Recently the retrieval-augmented generation (RAG) has been successfully applied in code generation. However, existing pipelines for retrieval-augmented code generation (RACG) employ static knowledge bases with a single source, limiting the adaptation capabilities of Large Language Models (LLMs) to domains they have insufficient knowledge of. In this work, we develop a novel pipeline, EVOR, that employs the synchronous evolution of both queries and diverse knowledge bases. On two realistic settin...
- Labels: code generation, code completion, source code model, agent design, prompt strategy, retrieval-augmented generation
-
Hierarchical Repository-Level Code Summarization for Business Applications Using Local LLMs, (arXiv2025)
- Abstract: In large-scale software development, understanding the functionality and intent behind complex codebases is critical for effective development and maintenance. While code summarization has been widely studied, existing methods primarily focus on smaller code units, such as functions, and struggle with larger code artifacts like files and packages. Additionally, current summarization models tend to emphasize low-level implementation details, often overlooking the domain and business context that ...
- Labels: static analysis, code summarization, agent design, prompt strategy, retrieval-augmented generation
-
Instructive Code Retriever: Learn from Large Language Model's Feedback for Code Intelligence Tasks, (ASE2024)
- Abstract: Recent studies proposed to leverage large language models (LLMs) with In-Context Learning (ICL) to handle code intelligence tasks without fine-tuning. ICL employs task instructions and a set of examples as demonstrations to guide the model in generating accurate answers without updating its parameters. While ICL has proven effective for code intelligence tasks, its performance heavily relies on the selected examples. Previous work has achieved some success in using BM25 to retrieve examples for ...
- Labels: agent design, prompt strategy, retrieval-augmented generation
-
Leandojo: Theorem proving with retrieval-augmented language models, (NeurIPS2023)
- Abstract: Large language models (LLMs) have shown promise in proving formal theorems using proof assistants such as Lean. However, existing methods are difficult to reproduce or build on, due to private code, data, and large compute requirements. This has created substantial barriers to research on machine learning methods for theorem proving. This paper removes these barriers by introducing LeanDojo: an open-source Lean playground consisting of toolkits, data, models, and benchmarks. LeanDojo extracts da...
- Labels: agent design, prompt strategy, retrieval-augmented generation
-
LocAgent: Graph-Guided LLM Agents for Code Localization, (ACL2025)
- Abstract: Code localization--identifying precisely where in a codebase changes need to be made--is a fundamental yet challenging task in software maintenance. Existing approaches struggle to efficiently navigate complex codebases when identifying relevant code sections. The challenge lies in bridging natural language problem descriptions with the appropriate code elements, often requiring reasoning across hierarchical structures and multiple dependencies. We introduce LocAgent, a framework that addresses ...
- Labels: agent design, prompt strategy, retrieval-augmented generation, planning
-
On the Impacts of Contexts on Repository-Level Code Generation, (NAACL2025)
- Abstract: CodeLLMs are widely used for code generation, yet their ability to handle repository-level dependencies remains underexplored. We introduce RepoExec, a benchmark for evaluating repository-level code generation, focusing on executability, functional correctness, and dependency utilization. Our study evaluates 18 models, revealing that retaining full dependency context yields the best performance, while smaller context sizes can be misleading. Pretrained LLMs excel in correctness but often reimple...
- Labels: code generation, program synthesis, agent design, prompt strategy, retrieval-augmented generation
-
RACONTEUR: A Knowledgeable, Insightful, and Portable LLM-Powered Shell Command Explainer, (NDSS2025)
- Abstract: Malicious shell commands are linchpins to many cyber-attacks, but may not be easy to understand by security analysts due to complicated and often disguised code structures. Advances in large language models (LLMs) have unlocked the possibility of generating understandable explanations for shell commands. However, existing general-purpose LLMs suffer from a lack of expert knowledge and a tendency to hallucinate in the task of shell command explanation. In this paper, we present Raconteur, a knowl...
- Labels: static analysis, code summarization, agent design, prompt strategy, retrieval-augmented generation
-
RepoAudit: An Autonomous LLM-Agent for Repository-Level Code Auditing, (ICML2025)
- Abstract: Code auditing is a code review process with the goal of finding bugs. Large Language Models (LLMs) have shown substantial potential in this task, offering the ability to analyze programs without compilation and enabling customized bug detection following specified prompts. However, applying LLMs to repository-level code auditing presents notable challenges. The inherent context limits and hallucinations of LLMs can lead to the low quality of bug reports. Meanwhile, the large size of software rep...
- Labels: static analysis, bug detection, agent design, prompt strategy, retrieval-augmented generation, planning
-
RepoGraph: Enhancing AI Software Engineering with Repository-level Code Graph, (ICLR2025)
- Abstract: Large Language Models (LLMs) excel in code generation yet struggle with modern AI software engineering tasks. Unlike traditional function-level or file-level coding tasks, AI software engineering requires not only basic coding proficiency but also advanced skills in managing and interacting with code repositories. However, existing methods often overlook the need for repository-level code understanding, which is crucial for accurately grasping the broader context and developing effective solutio...
- Labels: agent design, prompt strategy, retrieval-augmented generation, planning
-
Repository-Level Prompt Generation for Large Language Models of Code, (ICML2023)
- Abstract: With the success of large language models (LLMs) of code and their use as code assistants (e.g. Codex used in GitHub Copilot), techniques for introducing domain-specific knowledge in the prompt design process become important. In this work, we propose a framework called Repo-Level Prompt Generator that learns to generate example-specific prompts using prompt proposals. The prompt proposals take context from the entire repository, thereby incorporating both the structure of the repository and the...
- Labels: code generation, code completion, agent design, prompt strategy, retrieval-augmented generation
-
Retrieval-Based Prompt Selection for Code-Related Few-Shot Learning, (ICSE2023)
- Abstract: Large language models trained on massive code corpora can generalize to new tasks without the need for task-specific fine-tuning. In few-shot learning, these models take as input a prompt, composed of natural language instructions, a few instances of task demonstration, and a query and generate an output. However, the creation of an effective prompt for code-related tasks in few-shot learning has received little attention. We present a technique for prompt creation that automatically retrieves c...
- Labels: general coding task, code model, code model training, agent design, prompt strategy, retrieval-augmented generation
-
{C}odex{G}raph: Bridging Large Language Models and Code Repositories via Code Graph Databases, (NAACL2025)
- Abstract: Large Language Models (LLMs) excel in stand-alone code tasks like HumanEval and MBPP, but struggle with handling entire code repositories. This challenge has prompted research on enhancing LLM-codebase interaction at a repository scale. Current solutions rely on similarity-based retrieval or manual tools and APIs, each with notable drawbacks. Similarity-based retrieval often has low recall in complex tasks, while manual tools and APIs are typically task-specific and require expert knowledge, red...
- Labels: agent design, prompt strategy, retrieval-augmented generation