When Models Edit Too Much: On the Fidelity of Minimal Code Edits

The evolution of automated code repair has transitioned from rule-based systems like GenProg to deep learning models such as CodeX and GPT variants, which leverage large-scale training data for high success rates. Prior works primarily evaluate whether the bug is fixed, using metrics like Pass@k, but neglect the extent of code changes. As models grow larger, their capacity to generate complex modifications increases, raising concerns about over-editing and maintainability. Recent efforts focus on improving patch quality, but lack systematic quantification of unnecessary edits. This gap limits the practical deployment of AI-based repair tools, especially in maintenance scenarios where minimal, faithful patches are preferred. The need for metrics and training strategies that explicitly address edit fidelity remains pressing, motivating the current study.

Despite high functional success, large language models often produce repairs that involve excessive modifications, complicating review and risking regressions. This over-editing phenomenon is especially problematic in real-world maintenance, where minimal, faithful patches are crucial. Existing benchmarks do not quantify the size or complexity of edits, making it difficult to evaluate and improve models' fidelity. The core challenge is to develop metrics that measure unnecessary changes and training methods that teach models to prefer minimal edits, balancing repair success with simplicity. Addressing this problem requires a controlled evaluation framework, precise metrics, and effective training strategies to mitigate over-editing while maintaining repair effectiveness.

This work introduces a controlled benchmark based on injecting AST corruptions into reference solutions, enabling precise measurement of over-editing via excess Levenshtein distance and cognitive complexity. It proposes a preservation prompt that guides models to make minimal, faithful edits, significantly reducing unnecessary changes. Additionally, it employs reinforcement learning to train models to learn minimal editing preferences, demonstrating improved out-of-domain fidelity without sacrificing repair success. These innovations collectively establish a new paradigm for quantifying and learning edit fidelity, moving beyond traditional correctness metrics. The combination of controlled corruption, explicit prompting, and RL training offers a comprehensive approach to enhancing repair quality.

• �� Construct a benchmark from 400 BigCodeBench problems, injecting localized AST corruptions to generate known minimal repairs.
• �� Use multiple models (GPT-5.5, Claude Opus 4.7) with different prompting strategies (generic vs. preservation) to perform repairs.
• �� Measure repair quality using Pass@1, normalized token Levenshtein distance, and added cognitive complexity, validated by human annotation.
• �� Introduce a preservation prompt that explicitly instructs models to minimize unnecessary changes.
• �� Employ reinforcement learning (PRIME-RL) to fine-tune models, optimizing a reward function combining execution success and minimal edits.
• �� Conduct ablation studies across model sizes, corruption types, and training methods to analyze effects on edit fidelity.
• �� Evaluate out-of-domain generalization and the impact of prompt and training strategies on over-editing behavior.

The experiments use 400 Python repair tasks from BigCodeBench with injected AST corruptions, ensuring each task's minimal fix is known. Models include GPT-5.5, DeepSeek, Gemini, and open-weight models of varying sizes. They compare generic prompts with preservation prompts, and supervised fine-tuning versus reinforcement learning training. Metrics include Pass@1, normalized Levenshtein distance, and cognitive complexity, with human validation for metric reliability. Ablation studies analyze prompt effects, model size, and training methods, focusing on out-of-domain generalization. The training involves applying corruptions, optimizing reward functions, and evaluating repairs across multiple settings to assess fidelity improvements.

Models without prompts often exhibit excess Levenshtein distances averaging 0.195, indicating significant over-editing. Incorporating preservation prompts reduces this to 0.131, with a 26.6% decrease in cognitive complexity and a 2.3-point increase in Pass@1. Reinforcement learning further enhances out-of-domain fidelity, achieving minimal edit distances around 0.05 and Pass@1 above 78%. Larger models do not necessarily produce smaller edits; over-editing persists across scales. Corruption types like boundary and control flow bugs are more prone to excessive modifications. The combination of prompts and RL training effectively mitigates over-editing, demonstrating that edit fidelity can be learned and improved systematically.

The findings support developing automated code repair tools that prioritize minimal, faithful patches, reducing review and maintenance effort. These methods can be integrated into CI/CD pipelines, offering more reliable fixes. In the long term, AI-assisted programming could evolve into a standard component of software engineering, enabling faster, safer development cycles with less manual intervention, especially in legacy system maintenance and large-scale refactoring.

Current evaluation focuses on local, function-level repairs; applicability to large-scale architectural changes remains untested. Over-editing patterns are more evident in certain corruption types, and the generalization to more ambiguous or complex corruptions needs further validation. RL training incurs high computational costs, limiting scalability. Future work should explore multi-task learning, human-in-the-loop training, and broader corruption scenarios to address these limitations.