ISSN (Online): 2321-3418
server-injected
Engineering and Computer Science
Open Access

Automated Vulnerability Patching in Legacy Code Using LLMs and Multi-AI Agents

DOI: 10.18535/ijsrm/v14i04.ec05· Pages: 2875-2885· Vol. 14, No. 04, (2026)· Published: April 30, 2025
PDFAuto
Views: 11 PDF downloads: 1

Abstract

This report synthesizes peer-reviewed and archival research published between 2019 and 2024 on the use of large language models and multi-agent architectures for automated vulnerability repair, with a specific focus on how these techniques apply, or fail to apply, to legacy software systems. The review traces a clear methodological progression across this five-year window: from template and heuristic program repair, through neural and transfer-learning models, to fine-tuned transformer-based repair, zero-shot and conversational prompting of general-purpose LLMs, and finally to multi-agent pipelines in which specialized agents divide the work of detecting, localizing, patching, and validating a vulnerability. Two figures accompany the discussion. The first depicts a generalized multi-agent repair pipeline synthesized from the architectures described across the reviewed literature, showing how detection, localization, patch generation, and validation agents typically interact, including the validation feedback loop that distinguishes conversational and agentic systems from earlier single-shot approaches. The second traces the field's methodological evolution chronologically, from 2019's template-based baselines through 2024's agentic systems. A comparative table consolidates fifteen of the most relevant studies discussed in the body of the report, listing the approach, target language, and publication venue for each. The report concludes that while multi-agent LLM pipelines show real promise for legacy code specifically, since their capacity for autonomous context reconstruction and iterative, feedback-driven refinement directly addresses the sparse documentation and thin test coverage that define legacy environments, the literature to date has been validated almost exclusively on actively maintained, comparatively well-organized open-source repositories. The gap between these benchmarks and the scale and disorganization of real enterprise legacy systems has not yet been closed empirically, and closing it, together with building trust and explainability mechanisms suitable for security-critical maintenance work, is identified as the field's most consequential open problem.

Keywords

LLMs and Multi-AI Agents

1. Introduction

Legacy software systems remain the operational backbone of banking, government, healthcare, telecommunications, and industrial control environments, even as the tools used to build and secure them have fallen decades behind current practice. These systems are typically written in aging languages such as C, C++, COBOL, or early Java, maintained by shrinking teams, and burdened by sparse documentation, minimal automated test coverage, and deeply intertwined dependencies that make even small modifications risky. At the same time, the volume of publicly disclosed software vulnerabilities has continued to climb year over year, placing sustained pressure on security teams who must triage, prioritize, and remediate defects far faster than manual code review allows. This combination of aging code and accelerating threat disclosure has created a widening gap between the rate at which vulnerabilities are discovered and the rate at which they can be responsibly patched.

Automated Program Repair (APR) emerged as a research discipline intended to close part of this gap by generating candidate fixes for defective code without full manual intervention. Early APR techniques relied on hand-crafted fix templates, heuristic search over mutation operators, or constraint-based synthesis, and while these approaches demonstrated that automated repair was feasible in principle, they struggled to generalize beyond narrow bug classes and rarely addressed the deeper semantic reasoning that security vulnerabilities often require. The arrival of large language models (LLMs) trained on vast corpora of source code changed the calculus considerably. Because these models internalize statistical regularities in how developers actually write and fix code, they are able to propose plausible, context-aware patches for defect types that template-based systems could never anticipate.

More recently, the field has moved beyond single-model prompting toward multi-agent architectures, in which several specialized LLM-driven agents collaborate across the stages of the repair pipeline: locating the vulnerable code, reasoning about the underlying flaw, generating candidate patches, and validating those patches against compilers, test suites, and security sanitizers before a human ever sees the result. This architectural shift mirrors a broader trend in applied artificial intelligence, in which complex tasks are decomposed across cooperating agents rather than delegated to a single monolithic model. For legacy systems in particular, this decomposition offers a promising way to compensate for missing documentation and thin test coverage, since different agents can be tasked with reconstructing context, cross-referencing related commits, or synthesizing missing test oracles.

This report surveys the trajectory of automated vulnerability repair from its template-based origins through fine-tuned neural models, zero-shot LLM prompting, conversational repair, and finally multi-agent orchestration, with particular attention to how each generation of technique does or does not address the distinctive difficulties of legacy code. It draws on peer-reviewed and archival literature published between 2019 and 2024, synthesizes the methodological choices and evaluation practices used across this body of work, and concludes with an assessment of the practical and research challenges that remain before autonomous or semi-autonomous patching can be trusted with production legacy systems.

2. Background and Foundational Concepts

2.1 Automated Program Repair: Template, Heuristic, and Constraint-Based Methods

Before the deep learning era, APR research was dominated by three complementary strategies. Template-based approaches encoded common bug-fixing patterns mined from historical commit histories and attempted to match a defective code fragment against one of these templates; Liu, Koyuncu, Kim, and Bissyandé's TBar system exemplifies this approach, systematically applying a curated catalogue of fix patterns to the Defects4J benchmark and, at the time, establishing one of the strongest baselines in the field (Liu et al., 2019). Heuristic and search-based approaches instead treated repair as an optimization problem, using genetic programming or other stochastic search procedures to explore a space of candidate mutations until one satisfied the available test suite. Constraint-based approaches synthesized a fix directly from a formal specification of correct behavior, typically derived from failing and passing test cases. Each of these families produced valuable tooling, but all three shared a common limitation: repair quality was bounded by the completeness of the templates, mutation operators, or specifications the researchers had anticipated in advance, which made these systems brittle when confronted with the long tail of real-world defects.

2.2 From General-Purpose APR to Vulnerability-Specific Repair

Security vulnerabilities differ from ordinary functional bugs in that a successful patch must not only restore intended behavior but also eliminate an exploitable condition, such as a buffer overflow, use-after-free, injection flaw, or improper input validation, often without introducing a new weakness in the process. This distinction motivated a dedicated line of research into Automated Vulnerability Repair (AVR), which adapts APR techniques to security-specific datasets and evaluation criteria. Because labeled vulnerability-fix pairs are comparatively scarce relative to the general bug-fix commits available for ordinary APR training, much of the early AVR literature focused on data efficiency: transferring knowledge learned from large bug-fix corpora to the smaller vulnerability-specific setting, as in the transfer-learning design of VRepair (Chen et al., 2023), or fine-tuning pretrained sequence-to-sequence models such as T5 on curated vulnerability datasets, as in VulRepair (Fu et al., 2022).

2.3 Large Language Models for Code

The generation of LLMs trained specifically on source code, and later the much larger general-purpose conversational models such as GPT-3.5 and GPT-4, introduced two capabilities that proved decisive for vulnerability repair. First, because these models are pretrained on enormous quantities of publicly available code and natural-language documentation, they encode broad prior knowledge of common vulnerability patterns and their conventional fixes, which allows them to propose reasonable patches even without any task-specific fine-tuning, a capability studied directly by Pearce, Tan, Ahmad, Karri, and Dolan-Gavitt in their examination of zero-shot vulnerability repair (Pearce et al., 2023). Second, because these models can be prompted iteratively and can incorporate feedback from compilers, static analyzers, or test results within a conversational exchange, they support repair workflows that more closely resemble how a human developer debugs code: propose a fix, observe the outcome, and refine. Xia and Zhang formalized this conversational pattern in ChatRepair, demonstrating that iterative, feedback-conditioned prompting substantially outperforms single-shot patch generation on standard benchmarks (Xia & Zhang, 2024).

2.4 Multi-Agent Systems in Software Engineering

A multi-agent system, in the context relevant to this report, refers to an architecture in which multiple LLM-driven components, each configured with a distinct role, tool access, or prompting strategy, communicate and coordinate to accomplish a task that would be difficult for any single model instance to complete reliably in one pass. In software engineering specifically, this has translated into agents responsible for issue triage, code search, patch drafting, and test execution operating in a loop with explicit hand-offs between them. Talebirad and Nadiri outline the general coordination patterns that make such collaboration effective, including role specialization, shared memory, and negotiation protocols between agents (Talebirad & Nadiri, 2023), while systems such as MAGIS demonstrate this pattern concretely for GitHub issue resolution, assigning distinct agents to planning, coding, and review functions within a single repair episode (Tao et al., 2024).

2.5 Why Legacy Code Raises the Stakes for Each of These Foundations

Each of the foundational concepts introduced above interacts differently with legacy code than with the modern, actively maintained repositories most APR research has historically targeted. Template-based methods depend on patterns mined from large commit histories, which legacy repositories frequently lack in usable form, since decades of undocumented, ad hoc changes rarely produce the clean, attributable fix commits that pattern mining requires. Fine-tuned neural models depend on labeled training data drawn from the same population of modern repositories, which means their learned priors may not reflect the coding idioms, error-handling conventions, or memory-management patterns common in older C or COBOL codebases. Zero-shot and conversational LLM approaches are comparatively more robust to this distribution shift, since they draw on broad pretraining rather than a narrow fine-tuning corpus, but their effectiveness still degrades for languages and idioms underrepresented in that pretraining data, a concern discussed further in Section 8.3. Multi-agent systems, finally, depend on the availability of tools, such as compilers, test runners, and static analyzers, that must actually function against the legacy build environment, which is often the first and most mundane obstacle any deployment of this technology encounters in practice.

3. Evolution of LLM-Based Vulnerability Repair

The progression from template-based repair to modern agentic pipelines did not occur as a single leap but unfolded across a sequence of overlapping methodological generations, each addressing limitations exposed by its predecessor. Figure 1 summarizes this trajectory across the period covered by this review.

Figure 1
Figure 1 Evolution of automated vulnerability repair approaches, 2019 to 2024, from template-based systems to agentic multi-agent pipelines.

3.1 Neural and Transfer-Learning Approaches

The first wave of neural AVR systems adapted sequence-to-sequence architectures originally developed for machine translation to the task of transforming vulnerable code into a repaired version. Because vulnerability-labeled training data was scarce, Chen, Kommrusch, and Monperrus proposed training such a model first on a large general bug-fix corpus and only then fine-tuning it on the smaller vulnerability-specific dataset, a transfer-learning strategy that allowed the model to inherit broad code-editing competence before specializing (Chen et al., 2023). This approach demonstrated that neural repair could outperform template-based baselines on realistic C code vulnerabilities, but it remained constrained by the relatively small model capacities and limited context windows available at the time, which made it difficult to reason about vulnerabilities that spanned multiple functions or files.

3.2 Fine-Tuned Transformer Approaches

As pretrained transformer models such as T5 became widely available, researchers shifted toward fine-tuning these larger, pretrained backbones directly on vulnerability-repair datasets rather than training smaller models from scratch. Fu, Tantithamthavorn, Le, Nguyen, and Phung's VulRepair exemplifies this generation, combining a pretrained T5 encoder-decoder with byte-pair encoding tuned to source code tokens, and reporting substantial improvements in perfect-prediction accuracy over prior neural and template-based baselines on a large corpus of real-world vulnerability fixes drawn from open-source projects (Fu et al., 2022). This period established fine-tuning on domain-specific corpora as the dominant paradigm and produced the benchmarks and datasets that subsequent LLM-based approaches would continue to use for comparison.

3.3 Zero-Shot and Prompted LLM Repair

The emergence of very large, general-purpose language models trained on code and natural language at unprecedented scale made it possible to attempt vulnerability repair without any task-specific fine-tuning at all. Pearce, Tan, Ahmad, Karri, and Dolan-Gavitt conducted one of the first large-scale studies of this zero-shot setting, evaluating several commercially available and open-source LLMs on synthetic and hand-crafted vulnerability benchmarks and finding that, while these models could often produce plausible repairs, prompt design had an outsized effect on success rates, and performance varied considerably across vulnerability classes (Pearce et al., 2023). This sensitivity to prompt phrasing motivated follow-on work into more disciplined prompting strategies, including Ahmed and Devanbu's use of self-consistency, in which multiple independently sampled patches are generated and the most consistent among them is selected, an approach the authors showed measurably improved repair reliability over single-sample prompting (Ahmed & Devanbu, 2023). Around the same period, Xia, Wei, and Zhang conducted a broader empirical study of pretrained language models across the general APR task, establishing benchmarks and methodology that the vulnerability-specific literature would subsequently draw upon (Xia et al., 2023).

3.4 Conversational and Feedback-Conditioned Repair

A further refinement recognized that human developers rarely produce a correct fix on the first attempt; instead, they iterate, informed by compiler errors, failing tests, or static analysis warnings. Xia and Zhang operationalized this insight in ChatRepair, a fully automated conversational repair approach that interleaves patch generation with immediate feedback from the build and test pipeline, allowing the model to revise unsuccessful patches within the same dialogue rather than starting over, and reporting substantial cost and accuracy advantages over non-conversational baselines on the Defects4J benchmark (Xia & Zhang, 2024). Kulsum, Zhu, Xu, and d'Amorim extended this feedback-conditioned paradigm specifically to vulnerability repair with VRpilot, combining chain-of-thought reasoning prior to patch generation with iterative refinement driven by compiler and sanitizer output, and demonstrating measurable gains over prior vulnerability-repair baselines in both C and Java (Kulsum et al., 2024).

3.5 Toward Agentic Repair

The most recent phase in this progression moves beyond a single model engaged in a feedback loop toward architectures in which multiple cooperating agents divide the repair task among themselves. Zhang, Ruan, Fan, and Roychoudhury's AutoCodeRover exemplifies this shift, autonomously exploring a repository's structure to localize relevant code before generating a patch, effectively automating the context-gathering work that earlier approaches assumed would be supplied externally (Zhang et al., 2024). This agentic direction, discussed at length in the following section, represents the current frontier of the field and is where legacy-code-specific challenges are most actively being addressed.

4. Multi-Agent Architectures for Repair and Issue Resolution

Multi-agent repair systems generalize the single-model feedback loop into a pipeline of specialized roles, each of which can be independently prompted, equipped with different tools, or even instantiated with a different underlying model. Figure 2 depicts a generalized version of this pipeline, synthesized from the architectures reported across the literature reviewed in this report.

Figure 2
Figure 2 A generalized multi-agent LLM pipeline for vulnerability patching, showing detection, localization, patch generation, validation, and orchestration stages with a feedback loop.

4.1 Role Specialization

The central design decision in any multi-agent repair architecture is how responsibilities are divided among agents. A detection or triage agent is typically responsible for surfacing candidate vulnerabilities, often by wrapping conventional static or dynamic analysis tools; a localization agent narrows this down to the specific function, file, or code region responsible; a patch generation agent, almost always the LLM component doing the heaviest reasoning, proposes one or more candidate fixes; and a validation agent checks these candidates against compilers, existing test suites, and, where available, security-specific sanitizers. Lee, Xia, Yang, Huang, Zhu, Zhang, and Lyu's unified debugging framework demonstrates the value of this decomposition concretely, coordinating multiple LLM-based agents through a synergistic workflow that the authors show refines fault localization in ways a single monolithic prompt struggles to replicate (Lee et al., 2024).

4.2 Coordination and Communication Protocols

Once responsibilities are divided across agents, the system must define how information flows between them: whether agents share a common memory, whether they communicate through structured messages or free-form natural language, and how conflicts or disagreements between agents are resolved. Talebirad and Nadiri's general treatment of multi-agent LLM collaboration lays out several of the coordination patterns that recur across the applied software-engineering literature, including hierarchical delegation, in which a coordinating agent assigns subtasks to specialist agents, and peer negotiation, in which agents iteratively critique one another's output before converging on a final answer (Talebirad & Nadiri, 2023). MAGIS applies a hierarchical variant of this pattern specifically to GitHub issue resolution, assigning distinct agents to planning, code editing, and quality control roles within a single automated pull request workflow (Tao et al., 2024).

4.3 Autonomous Repository Exploration

A distinguishing feature of the most capable agentic systems is their ability to explore a codebase autonomously rather than relying on a human or an upstream tool to supply the relevant context. AutoCodeRover equips its agents with structured code-search capabilities, allowing them to navigate a repository's abstract syntax tree and identify the code elements most likely relevant to a given issue before any patch is attempted, which the authors argue is a key reason the system can resolve realistic GitHub issues without the exhaustive context that earlier repair tools required to be handed manually (Zhang et al., 2024). This autonomous exploration capacity is particularly consequential for legacy systems, where the institutional knowledge needed to identify relevant code by hand has often been lost.

4.4 Static Analysis as an Agent Tool

Rather than treating static analysis and LLM-based reasoning as competing approaches, several recent systems integrate conventional static analysis directly as a tool that an agent can invoke. Li, Dutta, and Naik's IRIS combines LLM-assisted reasoning with static analysis specifically for security vulnerability detection, using the static analyzer to constrain and verify the hypotheses the language model generates rather than relying on the model's judgment in isolation (Li et al., 2024). This hybrid pattern, in which deterministic tools anchor probabilistic reasoning, recurs throughout the agentic literature and is one of the more promising mitigations against the hallucination risk inherent to LLM-generated patches.

4.5 Benchmarking Agentic Systems

Evaluating multi-agent repair systems requires benchmarks that reflect the complexity of realistic software engineering tasks rather than isolated, single-function bugs. Jimenez and colleagues' SWE-bench addresses this need directly, curating a large collection of real-world GitHub issues paired with their corresponding human-authored pull requests, and has become a common evaluation target for agentic systems including AutoCodeRover and MAGIS, allowing the field to compare architectures on a shared, realistic task rather than on narrower academic benchmarks alone (Jimenez et al., 2023).

4.6 Handling Disagreement and Failure Between Agents

Coordinating multiple agents introduces a class of failure mode that single-model systems do not encounter: agents can disagree, and a downstream agent can propagate an upstream agent's mistake without any single component being individually at fault. A localization agent that misidentifies the vulnerable function will send a patch generation agent down an entirely unproductive path no matter how capable that generation agent is in isolation, and a validation agent with an incomplete test suite may certify an incorrect patch as successful. The hierarchical and peer-negotiation coordination patterns described by Talebirad and Nadiri are, in part, attempts to guard against this kind of error propagation by giving agents the ability to challenge or double-check one another's outputs before a candidate patch is finalized, though the authors are candid that no coordination protocol yet eliminates the risk entirely, only reduces its likelihood (Talebirad & Nadiri, 2023). This remains one of the more actively studied problems in agentic software engineering, and its resolution matters disproportionately for legacy code, where the thin test coverage discussed in Section 5.1 gives a validation agent comparatively little opportunity to catch an error introduced upstream.

5. Applying These Methods to Legacy Code Specifically

5.1 Distinctive Challenges of Legacy Systems

Legacy codebases present a cluster of difficulties that do not appear, or appear only mildly, in the modern, well-tested repositories most academic APR benchmarks are drawn from. Documentation is frequently absent or outdated, meaning an LLM-based agent cannot rely on docstrings or design documents to understand intended behavior. Test coverage is often minimal or entirely absent for the modules most in need of repair, which removes the automated oracle that most validation agents depend on to distinguish a correct patch from a merely plausible one. Dependencies on obsolete libraries, compilers, or even hardware assumptions can make it difficult to reproduce a build environment at all, let alone execute a test suite within it. Finally, legacy systems are disproportionately written in languages such as C, C++, or older Java dialects that are heavily represented in memory-safety vulnerability classes, which raises the stakes of an incorrect or incomplete patch considerably.

5.2 Reconstructing Missing Context

Because legacy code frequently lacks the documentation and test infrastructure that repair agents elsewhere depend on, a growing thread of the literature focuses on reconstructing this missing context automatically. IRIS's combination of static analysis with LLM reasoning is directly relevant here, since it allows an agent to derive structural and data-flow context about a legacy function even when no natural-language documentation exists to describe it (Li et al., 2024). Similarly, AutoCodeRover's autonomous repository exploration capability, originally designed for actively maintained open-source projects, has clear applicability to legacy systems, where a repair agent may need to trace usage of a vulnerable function across dozens of undocumented call sites before it can safely propose a change (Zhang et al., 2024).

5.3 Reducing Incorrect and Overfit Patches

A patch that merely satisfies an incomplete or poorly designed test suite, without addressing the underlying vulnerability in a generalizable way, is a well-documented failure mode in the APR literature, and this risk is amplified in legacy code where test suites are often thin. Le-Cong, Le, and Murray's semantic-guided search approach addresses this by narrowing the space of candidate patches using semantic constraints derived from the program's behavior rather than surface-level test outcomes alone, which the authors show improves the efficiency and correctness of LLM-generated repairs relative to naive sampling (Le-Cong et al., 2024). Techniques of this kind are particularly valuable in the legacy setting, where the alternative, namely trusting a sparse or absent test suite as the sole arbiter of correctness, is simply not viable.

5.4 Iterative, Feedback-Driven Repair for Under-Tested Code

Where legacy systems do retain some build tooling, even without comprehensive tests, conversational and feedback-conditioned repair strategies remain applicable and arguably become more important, since they allow an agent to use whatever partial signal is available, a successful compilation, a sanitizer warning, a single regression test, to iteratively refine a candidate patch rather than relying on a single-shot generation. VRpilot's use of compiler and sanitizer feedback alongside chain-of-thought reasoning illustrates how this iterative refinement can substitute, at least partially, for the comprehensive test suites that legacy code typically lacks (Kulsum et al., 2024), while ChatRepair's broader demonstration that conversational refinement outperforms single-shot prompting suggests the same pattern should transfer to legacy vulnerability repair even where the available feedback signal is weaker than in well-tested modern repositories (Xia & Zhang, 2024).

5.5 Organizational and Economic Pressures Behind Legacy Modernization

The technical difficulties discussed above do not exist in a vacuum; they are compounded by organizational pressures that make legacy systems especially attractive candidates for automated repair despite the added risk. Institutions that depend on decades-old codebases frequently face a shrinking pool of engineers familiar with the original languages and architectures, which means the manual review capacity needed to safely evaluate a proposed patch is itself scarce. This creates a difficult tension: the systems most in need of automated assistance are also the systems where human oversight of that assistance is hardest to staff. Multi-agent pipelines that surface a clear, reasoned rationale for each proposed patch, in the manner demonstrated by VRpilot's chain-of-thought design, partially mitigate this tension by lowering the expertise bar required to review a patch, even if they cannot eliminate the need for review altogether (Kulsum et al., 2024). Any realistic account of how this technology will be adopted in legacy environments must therefore weigh not only technical accuracy but also how well a given system reduces the specialized review burden placed on an already-stretched maintenance team.

6. Evaluation Methodologies

6.1 Benchmarks and Datasets

The literature reviewed in this report draws on a relatively small set of recurring benchmarks. Defects4J remains the dominant benchmark for general Java program repair and continues to anchor comparisons between conversational, agentic, and earlier neural approaches. Vulnerability-specific work has instead relied on curated corpora of real-world commit-linked vulnerability fixes, such as the dataset of over eight thousand vulnerability fixes drawn from more than seventeen hundred real-world projects that VulRepair was evaluated against (Fu et al., 2022). More recently, SWE-bench has become the standard for evaluating agentic systems on realistic, repository-scale issues rather than isolated single-function bugs, reflecting the field's broader shift toward tasks that better approximate real maintenance work (Jimenez et al., 2023). Table 1 summarizes the principal studies discussed in this report alongside their target languages and publication venues, offering a consolidated view of how the field has diversified across both approach and evaluation context.

Table 1 Selected studies in LLM-based and multi-agent automated vulnerability repair, 2019 to 2024.
Study Year System / Approach Language(s) Venue
Chen, Kommrusch, & Monperrus 2023 VRepair (transfer learning) C IEEE TSE
Fu, Tantithamthavorn, Le, Nguyen, & Phung 2022 VulRepair (fine-tuned T5) C/C++ ESEC/FSE
Pearce, Tan, Ahmad, Karri, & Dolan-Gavitt 2023 Zero-shot LLM prompting C/C++ IEEE S&P
Ahmed & Devanbu 2023 Self-consistency prompting Java/C ASE
Xia, Wei, & Zhang 2023 Pre-trained LLM repair benchmarking Multi-language ICSE
Xia & Zhang 2024 ChatRepair (conversational repair) Java ISSTA
Zhang, Ruan, Fan, & Roychoudhury 2024 AutoCodeRover (autonomous agent) Python ISSTA
Kulsum, Zhu, Xu, & d'Amorim 2024 VRpilot (reasoning + validation feedback) C/Java AIware
Lee, Xia, Yang, Huang, Zhu, Zhang, & Lyu 2024 Unified multi-agent debugging synergy Multi-language arXiv
Tao, Zhou, Zhang, & Cheng 2024 MAGIS (multi-agent issue resolution) Multi-language arXiv
Li, Dutta, & Naik 2024 IRIS (LLM-assisted static analysis) Java arXiv
Le-Cong, Le, & Murray 2024 Semantic-guided search repair Multi-language arXiv
Jimenez et al. 2023 SWE-bench (evaluation benchmark) Python arXiv
Liu, Koyuncu, Kim, & Bissyandé 2019 TBar (template-based baseline) Java ISSTA

6.2 Metrics

Reported metrics vary somewhat across studies but converge on a small number of recurring measures. Perfect prediction, meaning that a generated patch is textually identical or semantically equivalent to the developer's actual fix, is the strictest and most commonly reported accuracy measure. Plausible patch rate, meaning the patch passes all available tests without necessarily matching the reference fix, is reported alongside perfect prediction precisely because it can overstate success on codebases with weak test suites, a concern especially relevant to legacy systems as discussed in Section 5.3. Increasingly, studies also report the practical cost of repair in monetary or computational terms, following the precedent set by ChatRepair's headline finding that a majority of evaluated bugs could be fixed for well under a dollar in API costs (Xia & Zhang, 2024), a framing that has made cost-efficiency a first-class evaluation criterion alongside raw accuracy.

6.3 Human-in-the-Loop Validation

Despite the push toward autonomy, nearly every system reviewed in this report retains, or explicitly recommends, a human review step before a generated patch is merged into a production codebase. This is partly a matter of caution given the imperfect correlation between test-suite success and true correctness discussed above, and partly a recognition that security patches carry consequences, both technical and organizational, that automated systems are not yet trusted to bear alone. The orchestration and deployment stage depicted in Figure 1 reflects this convention, positioning human review as the final gate between an agent-generated patch and its deployment.

6.4 Reproducibility and Reporting Practices

A methodological concern that cuts across nearly every study reviewed in this report is the variability in how results are reported and how easily they can be independently reproduced. Some systems, including VulRepair and TBar, publish reproduction packages alongside their datasets and evaluation scripts, which has allowed subsequent researchers to build directly on their reported baselines with confidence (Fu et al., 2022; Liu et al., 2019). Other results, particularly those reported for proprietary, API-gated LLMs such as GPT-4, are inherently harder to reproduce exactly, since the underlying model can change behavior between API versions in ways that are not always documented by the provider, and the exact prompts and sampling parameters used are not always disclosed in full. This report treats reproducibility as an important, if secondary, criterion when weighing the strength of evidence behind any given claim, and readers evaluating these systems for adoption should likewise prioritize studies that provide sufficient methodological detail to be independently verified against their own legacy codebases.

7. Open Challenges and Risks

7.1 Patch Correctness Versus Plausibility

The gap between a patch that merely passes existing tests and a patch that genuinely eliminates a vulnerability without introducing new defects remains the most consequential open problem in this field. This gap widens for legacy code precisely because the test suites available to validate a patch are often the weakest link in the pipeline, and no amount of sophistication in the patch-generation agent can fully compensate for an inadequate oracle. Semantic-guided approaches such as Le-Cong, Le, and Murray's work represent one mitigation strategy, but the underlying tension between automatable verification and true semantic correctness is unlikely to be fully resolved without parallel investment in better test and specification generation for legacy systems (Le-Cong et al., 2024).

7.2 Scalability to Large and Poorly Structured Repositories

Much of the agentic literature demonstrates strong results on repositories that, while realistic, are still comparatively well-organized open-source projects with conventional directory structures and reasonably modern tooling. Legacy enterprise codebases are frequently far larger, more tangled, and less consistently structured, which stresses the context-retrieval and repository-exploration capabilities that systems such as AutoCodeRover depend on (Zhang et al., 2024). Whether the autonomous exploration strategies validated on curated open-source benchmarks generalize gracefully to codebases an order of magnitude larger, with decades of undocumented architectural drift, remains an open empirical question that the current literature has not yet directly addressed.

7.3 Security Risks Introduced by the Repair Process Itself

An automated patching pipeline is itself a piece of software with its own attack surface and failure modes. An LLM-generated patch can introduce a new vulnerability while appearing to fix the original one, particularly if the validation agent's test coverage does not exercise the new code path; a multi-agent system can also amplify this risk if a downstream agent trusts an upstream agent's assessment without independent verification. This risk motivates the layered validation architecture depicted in Figure 1, in which compiler checks, test suites, and sanitizers are applied jointly rather than any single signal being treated as sufficient, though the literature reviewed here also makes clear that no combination of automated checks has yet eliminated this risk entirely.

7.4 Trust, Explainability, and Developer Adoption

Even a technically correct patch will not be adopted if the developers responsible for a legacy system cannot understand why it was proposed or trust that it accounts for institutional knowledge the automated system was never given access to. The reasoning-first design adopted by several of the systems reviewed here, including VRpilot's use of chain-of-thought reasoning prior to patch generation, is partly motivated by this need for explainability, since a documented rationale gives a human reviewer something concrete to evaluate beyond the diff itself (Kulsum et al., 2024). Building this trust at organizational scale, however, is as much a socio-technical challenge as a technical one, and the current literature has focused predominantly on the latter.

7.5 Benchmark Bias Toward Modern, Well-Maintained Repositories

A subtler but equally important risk is that the benchmarks used to validate these systems, Defects4J, SWE-bench, and the curated vulnerability corpora discussed in Section 6.1, are drawn overwhelmingly from actively maintained, comparatively well-documented open-source projects, precisely because these are the repositories where clean commit histories and reliable test suites make rigorous evaluation possible in the first place. This creates a form of evaluation bias in which the reported success rates of agentic repair systems may not transfer proportionally to the legacy environments this report is ultimately concerned with, since the very properties that make a repository suitable for benchmarking, active maintenance, thorough tests, clear documentation, are the properties legacy systems most conspicuously lack. Readers of the agentic repair literature should therefore treat reported accuracy figures as an upper bound on what current systems can achieve in genuinely under-documented, under-tested legacy environments rather than as a direct estimate of legacy-specific performance.

8. Future Directions

8.1 Fully Autonomous Multi-Agent Pipelines

The trajectory traced in Section 3 points toward increasingly autonomous pipelines in which detection, localization, patch generation, and validation are handled end-to-end with minimal human intervention, reserving human review for exceptional or high-risk cases rather than every proposed patch. Realizing this vision responsibly for legacy systems will likely require the kind of layered, tool-integrated agent design demonstrated by IRIS and AutoCodeRover, extended with legacy-specific tooling for context reconstruction and test generation that the current literature has only begun to explore (Li et al., 2024; Zhang et al., 2024).

8.2 Integration with CI/CD and DevSecOps Workflows

For automated patching to have practical impact on legacy systems, it must integrate with the continuous integration and deployment pipelines that organizations already operate, triggering repair agents automatically when a new vulnerability is disclosed, routing generated patches through existing code review tooling, and logging the reasoning trail an agent followed for audit purposes. None of the systems reviewed in this report were evaluated within a live DevSecOps pipeline, and closing this gap between research prototype and operational tooling represents a substantial opportunity for near-term applied work.

8.3 Domain Adaptation for Legacy-Heavy Languages

Much of the LLM-based repair literature concentrates on Java, Python, and C/C++, languages well represented in the public code corpora that large models are pretrained on. Legacy systems in domains such as finance, government, and industrial control frequently rely on COBOL, Fortran, or vendor-specific dialects that are comparatively underrepresented in these training corpora, which may degrade the zero-shot repair quality that Pearce and colleagues documented for better-represented languages (Pearce et al., 2023). Targeted domain adaptation, whether through continued pretraining on available legacy-language corpora or through retrieval-augmented approaches that supply relevant examples at inference time, represents an underexplored direction with direct practical relevance to the legacy-modernization problem this report is concerned with.

8.4 Standardized Benchmarks for Legacy-Specific Vulnerability Repair

Finally, the field currently lacks a benchmark analogous to SWE-bench or Defects4J that is purpose-built around the specific difficulties of legacy code: sparse tests, missing documentation, and outdated toolchains. Constructing such a benchmark, ideally drawn from real legacy vulnerability disclosures with careful attention to reproducible build environments, would allow the community to measure progress on the legacy-specific challenges discussed throughout Sections 5 and 7 with the same rigor that SWE-bench has brought to general repository-level issue resolution (Jimenez et al., 2023).

8.5 Hybrid Human-AI Review Workflows

Rather than framing the goal as full autonomy, a substantial portion of near-term progress is likely to come from designing better hybrid workflows in which agents handle the labor-intensive work of context gathering, candidate generation, and preliminary validation, while human reviewers are presented with a small number of well-justified candidate patches accompanied by clear, auditable reasoning rather than a raw diff. This reframes the human role from patch author to patch auditor, a shift that plays to the comparative strengths of both parties: agents excel at exhaustively searching a large candidate space and applying consistent validation checks, while human reviewers retain institutional knowledge and risk judgment that no system trained solely on public code repositories can be expected to replicate. Designing the interfaces, audit trails, and escalation criteria that make this division of labor effective is likely to matter as much for real-world adoption as further improvements to the underlying language models themselves.

9. Conclusion

This report has traced the development of automated vulnerability repair from its origins in template-based and heuristic program repair through fine-tuned neural models, zero-shot and conversational large language model prompting, and finally toward the multi-agent architectures that currently define the state of the art. Across this trajectory, a consistent pattern emerges: each generation of technique has addressed a limitation exposed by its predecessor, from the narrow generalization of hand-crafted templates, to the data hunger of early neural models, to the prompt sensitivity of zero-shot LLMs, to the single-shot brittleness that conversational and feedback-conditioned approaches were designed to overcome. Multi-agent systems represent the current synthesis of these lessons, distributing the repair task across specialized agents for detection, localization, patch generation, and validation, and increasingly equipping these agents with the ability to explore a codebase and gather context autonomously rather than depending on a human to supply it.

For legacy systems specifically, the promise of this trajectory is significant but not yet fully realized. The very features that make multi-agent LLM pipelines powerful, autonomous context reconstruction, iterative feedback-driven refinement, and tool-integrated reasoning, are also the features best suited to compensating for the sparse documentation, thin test coverage, and outdated tooling that define legacy environments. Systems such as AutoCodeRover and IRIS demonstrate that autonomous exploration and hybrid static-analysis-plus-LLM reasoning can already recover meaningful context in modern, if imperfectly documented, repositories, and there is good reason to expect these techniques to transfer, at least partially, to older and more poorly structured systems. At the same time, the literature reviewed in this report has been evaluated almost exclusively on benchmarks drawn from actively maintained, comparatively well-tested open-source projects, and the gap between these benchmarks and the scale, disorganization, and toolchain obsolescence of real enterprise legacy code has not yet been closed empirically.

Three considerations should guide how this technology is adopted in practice. First, the distinction between a plausible patch and a genuinely correct one remains unresolved at a fundamental level, and this distinction matters more, not less, in legacy code precisely because the test suites available to catch an incorrect patch are weakest where the risk is highest; organizations adopting these tools should treat automated validation as a necessary but not sufficient condition for trust and should retain meaningful human review, particularly for security-critical modules. Second, the multi-agent paradigm introduces its own attack surface, since a repair pipeline that autonomously modifies production code is itself security-relevant infrastructure, and the layered validation architecture described in this report, compiler checks, test suites, and sanitizers operating jointly, should be regarded as a baseline requirement rather than an optional enhancement. Third, the field's current concentration on Java, Python, and C/C++ leaves a meaningful blind spot around the COBOL, Fortran, and vendor-specific languages that disproportionately populate the systems most in need of this technology, and closing that gap through targeted domain adaptation and dedicated legacy-focused benchmarks should be treated as a priority rather than an afterthought.

Taken together, the literature surveyed here supports a measured but genuinely optimistic conclusion. Automated vulnerability patching has progressed, within a remarkably short span of years, from brittle template matching to agentic systems capable of autonomously navigating a codebase, reasoning about a vulnerability's root cause, and iteratively refining a fix in response to real feedback. The next phase of progress will depend less on further scaling of the underlying language models and more on the harder, less glamorous work of building legacy-appropriate benchmarks, integrating these pipelines into real DevSecOps workflows, and developing the trust and explainability mechanisms that will let human maintainers hand meaningful authority to these systems without abdicating responsibility for the code they are ultimately accountable for. Achieving that balance, rather than any single architectural breakthrough, is likely to determine how much of this research translates into safer legacy software in practice.

References

  1. Ahmed, T., & Devanbu, P. (2023). Better patching using LLM prompting, via self-consistency. In 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE) (pp. 1742–1746). IEEE. DOI ↗ Google Scholar ↗
  2. Chen, Z., Kommrusch, S., & Monperrus, M. (2023). Neural transfer learning for repairing security vulnerabilities in C code. IEEE Transactions on Software Engineering, 49(1), 147–165. DOI ↗ Google Scholar ↗
  3. Verma, Harsh. (2025). AI-driven cybersecurity in software engineering. World Journal of Advanced Research and Reviews. 27. 2012-2025. . DOI ↗ Google Scholar ↗
  4. Verma, Harsh. (2025). Ethical challenges and bias mitigation in Artificial Intelligence systems. World Journal of Advanced Research and Reviews. 28. 2364-2373. . DOI ↗ Google Scholar ↗
  5. Verma, Harsh. (2026). Cloud-based AI systems for scalable and intelligent software applications. World Journal of Advanced Research and Reviews. 29. 2041-2051. . DOI ↗ Google Scholar ↗
  6. Fu, M., Tantithamthavorn, C., Le, T., Nguyen, V., & Phung, D. (2022). VulRepair: A T5-based automated software vulnerability repair. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2022) (pp. 935–947). ACM. DOI ↗ Google Scholar ↗
  7. Jimenez, C. E., Yang, J., Wettig, A., Yao, S., Pei, K., Press, O., & Narasimhan, K. (2023). SWE-bench: Can language models resolve real-world GitHub issues? arXiv. DOI ↗ Google Scholar ↗
  8. Kulsum, U., Zhu, H., Xu, B., & d'Amorim, M. (2024). A case study of LLM for automated vulnerability repair: Assessing impact of reasoning and patch validation feedback. In Proceedings of the 1st ACM International Conference on AI-Powered Software (AIware 2024) (pp. 103–111). ACM. DOI ↗ Google Scholar ↗
  9. Le-Cong, T., Le, B., & Murray, T. (2024). Semantic-guided search for efficient program repair with large language models. arXiv. https://arxiv.org/abs/2410.16655 Google Scholar ↗
  10. Lee, C., Xia, C. S., Yang, L., Huang, J., Zhu, Z., Zhang, L., & Lyu, M. R. (2024). A unified debugging approach via LLM-based multi-agent synergy. arXiv. DOI ↗ Google Scholar ↗
  11. Li, Z., Dutta, S., & Naik, M. (2024). IRIS: LLM-assisted static analysis for detecting security vulnerabilities. arXiv. https://arxiv.org/abs/2405.17238 Google Scholar ↗
  12. Liu, K., Koyuncu, A., Kim, D., & Bissyandé, T. F. (2019). TBar: Revisiting template-based automated program repair. In Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2019) (pp. 31–42). ACM. DOI ↗ Google Scholar ↗
  13. Pearce, H., Tan, B., Ahmad, B., Karri, R., & Dolan-Gavitt, B. (2023). Examining zero-shot vulnerability repair with large language models. In 2023 IEEE Symposium on Security and Privacy (SP) (pp. 2339–2356). IEEE. DOI ↗ Google Scholar ↗
  14. Verma, H. (2025). Future Trends in AI, Machine Learning, and Big Data: Implications for Technical Leadership. International Journal of Engineering & Extended Technologies Research (IJEETR), 7(4), 10448-10460. Google Scholar ↗
  15. Verma, H. (2019). Secure Real-Time Heterogeneous IoT Data Management System. Available at SSRN 6573879. Verma, H., & Krishnan, A. Analytics Performance Load Test. International Journal of Computer Applications, 975, 8887. Google Scholar ↗
  16. Verma, H. (2024). AI Agentic Architectures for Autonomous Data Engineering Pipelines. International Journal of Research and Applied Innovations, 7(6), 11984-11994. Google Scholar ↗
  17. Talebirad, Y., & Nadiri, A. (2023). Multi-agent collaboration: Harnessing the power of intelligent LLM agents. arXiv. https://arxiv.org/abs/2306.03314 Google Scholar ↗
  18. Tao, W., Zhou, Y., Zhang, W., & Cheng, Y. (2024). MAGIS: LLM-based multi-agent framework for GitHub issue resolution. arXiv. https://arxiv.org/abs/2403.17927 Google Scholar ↗
  19. Xia, C. S., Wei, Y., & Zhang, L. (2023). Automated program repair in the era of large pre-trained language models. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) (pp. 1482–1494). IEEE. DOI ↗ Google Scholar ↗
  20. Xia, C. S., & Zhang, L. (2022). Less training, more repairing please: Revisiting automated program repair via zero-shot learning. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2022) (pp. 959–971). ACM. DOI ↗ Google Scholar ↗
  21. Xia, C. S., & Zhang, L. (2024). Automated program repair via conversation: Fixing 162 out of 337 bugs for $0.42 each using ChatGPT. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2024) (pp. 819–831). ACM. DOI ↗ Google Scholar ↗
  22. Zhang, Y., Ruan, H., Fan, Z., & Roychoudhury, A. (2024). AutoCodeRover: Autonomous program improvement. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2024) (pp. 1592–1604). ACM. Google Scholar ↗
  23. Verma, Harsh. (2025). Explainable AI (XAI) for Software Engineering Decision-Making. . DOI ↗ Google Scholar ↗
  24. Islam, M. S., Verma, H., Khan, L., & Kantarcioglu, M. (2019, December). Secure real-time heterogeneous iot data management system. In 2019 first IEEE international conference on trust, privacy and security in intelligent systems and applications (TPS-ISA) (pp. 228-235). IEEE. Shrestha, A. K., Singha, S., Sural, S., Sutton, S., Tahiri, S., Tipper, D., ... & Yu, L. Yu, Xiaoyuan 46 Zhao, Zhilong 236 Zou, Xukai 46. Google Scholar ↗
  25. Verma, H. (2024). Autonomous Multi-Agent Systems for Enterprise Decision-Making. International Journal of Engineering & Extended Technologies Research (IJEETR), 6(5), 8867-8880. Google Scholar ↗
  26. Verma, H. (2026). Security in Multi-Agent AI Systems: Modeling Emergent Vulnerabilities via Trust Graphs. International Journal of Scientific Research and Management (IJSRM), 14(04), 2863-2874. Google Scholar ↗
Author details
Harsh Verma
Palo Alto Networks, Artificial Intelligence, United States
✉ Corresponding Author
👤 View Profile →🔗 Is this you? Claim this publication