Agents in Software Engineering: Survey, Landscape, and Vision
A survey of 115 studies proposes a Perception–Memory–Action framework for LLM-based software-engineering agents.
Key Findings
Methodology
The authors filtered and quality-assessed 115 papers on LLM-based agents for software engineering. They organize the field into Perception, Memory, and Action: perception handles text, code, vision, or audio; memory comprises semantic, episodic, and procedural knowledge; action covers reasoning, retrieval, learning, and interaction with humans or digital environments, including multi-agent collaboration.
Key Results
- The principal result is a taxonomy rather than a new accuracy number: 115 studies are mapped to three modules. Existing work concentrates on token-based text, documentation/API retrieval, Chain-of-Thought reasoning, and execution-environment interaction.
- Representative mechanisms include DocPrompting, RepoCoder, De-Hallucinator, CodeAgent, SCoT, CodeCoT, and TOOLGEN, spanning code generation, completion, debugging, vulnerability repair, repository-level development, and tool use.
- The survey identifies consistent gaps: little agent research uses vision, audio, or hybrid structural inputs; software engineering lacks an authoritative code knowledge base; and multi-agent collaboration introduces substantial computation, synchronization, and communication overhead.
Significance
This work gives the fragmented LLM-for-SE literature a common vocabulary. It argues that an agent is not merely a chatbot that invokes tools, but a closed-loop system that observes software state, consults memory, acts, and uses feedback to revise decisions. Academically, the framework supports systematic comparison. Industrially, it connects code models with retrieval, interpreters, testing platforms, repository navigation, and developer workflows.
Technical Contribution
The main technical contribution is a software-engineering-specific taxonomy. Perception distinguishes tokens, AST/control-flow representations, and hybrid inputs; memory separates external knowledge, case history, and knowledge embedded in model parameters or agent code; action separates reasoning, retrieval, learning, and external interaction. The framework also places single agents and multi-agent collaboration in one conceptual model.
Novelty
The paper presents itself as the first in-depth survey of LLM-based agents in software engineering. Its novelty is not a new learning algorithm, but a unifying interpretation of prompting, retrieval, tool calling, self-checking, and agent collaboration as a Perception–Memory–Action loop, followed by a concrete research agenda.
Limitations
- This is a survey and vision paper, not a controlled benchmark study. It provides no common dataset, metric, or end-to-end experiment that can establish one architecture as superior in accuracy, reliability, or cost.
- The evidence is concentrated on code generation and textual inputs. Vision, audio, structural code representations, and multi-agent efficiency are underexplored, while category boundaries depend on how broadly “agent” is defined.
Future Work
Future work should build authoritative code knowledge bases; support visual, auditory, AST/graph, and multimodal perception; and develop role-adaptive agents with multiple capabilities. Evaluation should jointly measure test-passing correctness, repair validity, cost, latency, security, and human effort. The authors also call for hallucination reduction, efficient multi-agent communication, and stronger integration of program analysis, testing, and formal verification into agent systems.
AI Executive Summary
Large language models now support code summarization, generation, translation, vulnerability repair, and debugging. Yet a single prompt rarely handles the long context, dependencies, execution feedback, and iterative changes of real software engineering. Many papers already use agent-like mechanisms, but under inconsistent terminology. Wang and colleagues survey 115 filtered studies and ask what an LLM-based software-engineering agent is made of and how it tackles multi-step work.
Their framework has three modules. Perception converts text, source code, UML, execution results, or potentially audio into model-usable information. Memory contains semantic knowledge such as documentation and APIs, episodic information such as prior interactions and examples, and procedural knowledge stored in model parameters or agent code. Action includes internal reasoning, retrieval, and learning, plus external interaction with developers, other agents, code interpreters, testing platforms, and digital environments. DocPrompting and RepoCoder show how retrieval grounds generation; SCoT and CodeCoT specialize reasoning for code; CodeAgent connects repository navigation and testing tools.
The survey’s central contribution is a map rather than a single performance claim. Current work is strongest in token-based text, external documentation/API memory, and CoT-style reasoning. Visual, auditory, structural, and hybrid inputs remain largely unexplored, and the field lacks a trusted code knowledge base. Multi-agent systems may enable specialization but create synchronization and compute costs. The authors therefore envision agents that observe, remember, act, verify, and learn, while software-engineering technologies such as testing and program analysis in turn advance the broader agent field.
Deep Analysis
Background
LLMs have spread across code summarization, generation, translation, vulnerability detection, and repair. Representative systems include DocPrompting, RepoCoder, AceCoder, De-Hallucinator, and CodeAgent; reasoning methods include CoT, SCoT, CodeCoT, and Brainstorm. However, papers often describe agent behavior as prompting, retrieval, tool use, or self-correction without a shared conceptual structure.
Core Problem
Real software tasks require repository understanding, API selection, execution, error interpretation, and iterative modification. One-shot generation lacks reliable environmental feedback and is vulnerable to hallucination, forgotten history, and context limits. Multi-agent decomposition may help, but it adds inference calls, synchronization, and communication overhead.
Innovation
- �� Surveys and quality-assesses 115 studies.
- �� Introduces a Perception–Memory–Action framework for single and multi-agent SE systems.
- �� Splits memory into semantic, episodic, and procedural forms, and action into reasoning, retrieval, learning, and external interaction.
- �� Identifies neglected modalities, missing knowledge infrastructure, hallucination, and collaboration efficiency as research priorities.
Methodology
- �� Perception: token sequences encode code; ASTs and control-flow graphs expose syntax and structure; visual inputs may include UI sketches or UML; audio supports speech interaction.
- �� Memory: DocPrompting retrieves documentation; KPC uses API exception knowledge; AceCoder retrieves similar programs; RepoCoder iterates retrieval and generation.
- �� Reasoning: CodeChain performs self-revision; CodeCoT combines test generation with self-examination; SCoT structures reasoning in code-like form.
- �� Action: CodeAgent invokes symbol navigation and testing tools; TOOLGEN learns when to trigger completion tools; learning updates model knowledge, external memory, or agent code.
- �� Collaboration: multiple agents divide responsibilities and share state.
Experiments
The paper primarily performs literature collection, filtering, and quality assessment, retaining 115 papers rather than running a unified benchmark. The reviewed systems cover repositories, API documentation, runtime environments, and digital software environments. CODEAGENTBENCH is cited as a manually curated repository-level generation benchmark, but the survey reports no cross-paper accuracy, pass-rate, cost, or ablation aggregate.
Results
The taxonomy shows that token text, documentation/API retrieval, CoT reasoning, and code-execution interaction are the most mature patterns. Vision and audio remain scarcely explored in LLM-based SE agents. Although AST and graph representations exist in conventional SE, they are not systematically integrated into agent perception. Hallucination, knowledge-base construction, and multi-agent efficiency emerge as common bottlenecks.
Applications
The framework can guide repository-level completion, requirements-to-code generation, automated debugging, vulnerability repair, test generation, code review, and developer question answering. Practical deployment requires reliable indexing, sandboxed execution, testing feedback, permission control, rollback, and human approval rather than unconstrained text generation.
Limitations & Outlook
The survey lacks a common experimental protocol, so it cannot rank algorithms rigorously; its corpus may also depend on search coverage and the definition of “agent.” Future work needs realistic repository benchmarks, standardized metrics, and cost-aware comparisons of single agents, multi-agent systems, retrieval augmentation, and tool use. Security, privacy, recovery from tool errors, and verifiable execution remain essential.
Plain Language Accessible to non-experts
Imagine an agent as an engineer repairing a complicated factory machine. Perception is the engineer’s eyes and ears: it reads the request, examines the machine’s diagrams, studies its parts, and watches what happens when the machine runs. Memory is the toolbox and repair notebook. The toolbox contains manuals and part specifications; the notebook records similar breakdowns and earlier attempts; practiced skills are already in the engineer’s mind.
Action is the repair process. The engineer first reasons about the fault, searches the manual, changes a component, switches the machine on, and uses the result to decide what to do next. DocPrompting is like checking the manual before choosing a part. RepoCoder is like searching the factory’s old repairs for a similar solution. CodeCoT is like writing down the repair steps while testing each one.
The paper organizes 115 studies as if it were cataloguing a huge repair company. It finds that most systems can read written instructions, search documents, and use testing tools. Far fewer can understand interface pictures or sound. There is also no universally trusted catalogue of software parts, and teams of automated engineers may waste time reporting to one another. Better agents must observe more kinds of evidence, remember reliable facts, test every repair, and cooperate efficiently.
ELI14 Explained like you're 14
Picture a game where your teammate must repair a broken city. It does not simply answer, “Here is some code.” It reads the mission, checks the map, searches its backpack, enters the city to test a machine, notices the failure, and plans again. That teammate is a software-engineering agent.
The authors examined 115 papers and describe three parts: perception, memory, and action. Perception is the teammate’s eyes, handling text, code, diagrams, and results from running programs. Memory is the save file: it stores manuals, previous missions, useful examples, and skills learned inside the model. Action is what the teammate does—think, search, edit code, run tests, and talk to humans or other teammates.
RepoCoder searches the whole project for similar code. DocPrompting checks documentation before writing an answer. CodeCoT links coding with testing and self-checking. These ideas are stronger than blindly guessing because the program can actually run and reveal mistakes.
But this teammate is not an unbeatable character! Most systems mainly read words, not pictures or sound. They may invent an API that does not exist, and several agents working together can use lots of computer power. The exciting next level is an agent that sees more, remembers trustworthy information, checks every move, and works smoothly with its team. Would you trust it with your homework project yet? Probably only with a backup copy!
Glossary
LLM-based Agent
A system using a large language model as its cognitive core to perceive an environment, reason, and act. It commonly relies on tools, memory, and feedback for multi-step tasks.
The paper studies such agents for code generation, debugging, retrieval, and software environments.
Perception
The module that converts external text, code, images, or audio into information the model can process. It defines which software-engineering states the agent can observe.
The survey distinguishes token, tree/graph, hybrid, visual, and auditory inputs.
Semantic, Episodic, and Procedural Memory
Semantic memory stores facts such as documentation and APIs; episodic memory stores cases, messages, and examples; procedural memory stores operating knowledge in model parameters or agent code.
These three forms constitute the framework’s memory module.
Chain-of-Thought (CoT)
A prompting or reasoning strategy that decomposes a problem into intermediate steps. For code, it can include structured reasoning, self-revision, and test-based checking.
The paper discusses SCoT, CodeCoT, Tree CoT, Brainstorm, and related variants.
Retrieval-Augmented Generation
A process that retrieves relevant documentation, APIs, or code before adding it to the generation context. It grounds outputs in project-specific evidence and can reduce unsupported guesses.
DocPrompting, RepoCoder, and De-Hallucinator exemplify this mechanism.
Open Questions Unanswered questions from this research
- 1 Unified evaluation remains unresolved: papers use different tasks, datasets, and metrics. The field needs joint measurement of correctness, test passing, cost, latency, security, and developer effort.
- 2 It is unclear how tokens, code structure, images, and audio should be fused effectively. Progress requires multimodal datasets, structure-aware models, and validation in realistic development workflows.
- 3 Efficient multi-agent collaboration lacks reproducible mechanisms and theory. Systems must reduce communication and synchronization while preserving specialization and solution quality.
Applications
Immediate Applications
Repository-level coding assistant
Teams can combine RepoCoder-style iterative retrieval, CodeAgent tool use, and project API documentation for completion, code navigation, and testing. Deployment needs private indexing, sandbox execution, permissions, and review; expected benefits include fewer irrelevant completions and fewer API hallucinations.
Automated debugging and repair
A system can combine CodeCoT’s test-and-self-examine loop with De-Hallucinator’s API grounding to locate defects, propose patches, and run tests from a bug report. It is best suited to low-risk, version-controlled repositories, with developers approving consequential changes.
Long-term Vision
Multi-role software engineering team
Requirement, coding, testing, and security agents could share semantic, episodic, and procedural memory inside a verified digital environment. Such teams might transform development productivity, but require solutions for cost, permissions, security, state synchronization, and accountability.
Abstract
In recent years, Large Language Models (LLMs) have achieved remarkable success and have been widely used in various downstream tasks, especially in the tasks of the software engineering (SE) field. We find that many studies combining LLMs with SE have employed the concept of agents either explicitly or implicitly. However, there is a lack of an in-depth survey to sort out the development context of existing works, analyze how existing works combine the LLM-based agent technologies to optimize various tasks, and clarify the framework of LLM-based agents in SE. In this paper, we conduct the first survey of the studies on combining LLM-based agents with SE and present a framework of LLM-based agents in SE which includes three key modules: perception, memory, and action. We also summarize the current challenges in combining the two fields and propose future opportunities in response to existing challenges. We maintain a GitHub repository of the related papers at: https://github.com/DeepSoftwareAnalytics/Awesome-Agent4SE.