AIChilles: Automatically Uncovering Hidden Weaknesses in AI-Evolved Systems
AIChilles finds 49 hidden regressions across 30 AI-evolved system programs using differential, constraint-aware search.
Key Findings
Methodology
AIChilles compares a human baseline P with an evolved program P′. It deterministically extracts workload parameters from evaluators and generators, uses an agent to infer ranges and cross-parameter constraints, then constructs valid workloads. Four type-specific agents search for correctness, runtime, memory, and quality regressions. P is the differential oracle; execution-frequency trajectories of P′ provide behavioral-diversity guidance, while deduplication and agentic root-cause analysis consolidate witnesses.
Key Results
- Across 30 program settings produced by Engram, AdaEvolve, and OpenEvolve, spanning five system applications and GPT-5/Claude-Opus-4.6, AIChilles found 49 distinct weaknesses: 25 runtime, 11 memory, 7 correctness, and 6 optimality instances. In Prism, OpenEvolve changes complexity from O(MG) to O(MG²).
- Against random fuzzing, mutation fuzzing, property-based testing, and a baseline LLM agent, AIChilles consistently finds more diverse weaknesses under the same time budget. Engram exposes fewer weaknesses than AdaEvolve and OpenEvolve, but no family is uniformly robust.
- Adding AIChilles feedback to evolution removes several weaknesses by penalizing vulnerable candidates. The trade-off is reduced benchmark improvement: the original 12–60% gains shrink, and some robust winners move closer to the human baseline.
Significance
The paper shows that benchmark leadership does not imply deployment robustness. AI-generated system algorithms may overfit evaluator workloads, increase asymptotic cost, consume excessive memory, or fail on valid unseen inputs. For academia, AIChilles supplies a cross-application framework for studying these regressions. For industry, it proposes a practical CI/CD safety layer for schedulers, model-placement controllers, cache systems, and other components on the critical path of production infrastructure.
Technical Contribution
The main technical contribution is a unified formulation of heterogeneous testing as relative divergence between P and P′. Deterministic parameter extraction stabilizes workload discovery; agentic constraint inference recovers application semantics; type-specific agents prevent a monolithic search from collapsing onto easy crashes; differential checks ground decisions in execution; and code-frequency trajectories promote behavioral diversity. The taxonomy is explicit: P′ crashes while P succeeds, t′>Ft·t, m′>Fm·m, or q′<q.
Novelty
Unlike conventional fuzzers that primarily target crashes, and performance tools designed for one program or metric, AIChilles searches for regressions relative to a baseline across correctness, runtime, memory, and solution quality. Its fundamental novelty is the composition of deterministic program analysis, LLM-based semantic constraint recovery, weakness-specific search, differential oracles, and trajectory-based diversity for AI-evolved systems.
Limitations
- The method assumes that P is an executable and sufficiently trustworthy oracle. If the baseline is itself incorrect, inefficient, or based on a different objective, shared bugs and legitimate improvements may be misclassified or missed.
- Constraint inference remains agent-dependent, while timing and memory measurements are noisy. Randomized programs, external services, and changing system state can create invalid or irreproducible witnesses.
- The evaluation covers five application classes and 30 settings, not the full space of production systems or long-term traffic distributions.
Future Work
Future work should combine multiple baselines, formal specifications, real production traces, repeated statistical measurements, and richer resource or security objectives. Trajectory coverage could be extended with data-flow, state, energy, and semantic coverage. Integrating these signals directly into evolutionary fitness may produce programs that are both high-scoring and robust rather than merely benchmark-specialized.
AI Executive Summary
LLM-driven system evolution is turning algorithm design into an iterative search process. Frameworks such as AlphaEvolve, OpenEvolve, AdaEvolve, and Engram rewrite programs and retain candidates that score well on fixed workloads; AdaEvolve reports 12–60% gains over human-designed algorithms. Yet a fixed evaluator can reward specialization rather than robustness. An evolved scheduler may excel on benchmark cases while becoming slower, memory-hungry, incorrect, or inferior on valid unseen workloads.
Zhou and colleagues introduce AIChilles, which treats the human program P as a differential oracle for the evolved program P′. It extracts workload parameters deterministically from evaluators and generators, asks an agent to infer ranges and cross-parameter constraints, and builds valid workload grammars. Four specialized agents search separately for correctness, runtime, memory, and optimality regressions. Execution-frequency vectors from P′ act as behavioral fingerprints, steering the search away from repeated failures. A weakness exists when P′ crashes while P succeeds, or when t′>Ft·t, m′>Fm·m, or q′<q.
Across 30 settings from Engram, AdaEvolve, and OpenEvolve, covering transaction scheduling, expert-parallel load balancing, multi-cloud scheduling, LLM prefix-cache optimization, and model placement, AIChilles found 49 distinct weaknesses: 25 runtime, 11 memory, 7 correctness, and 6 optimality. In the Prism case, OpenEvolve raises placement complexity from O(MG) to O(MG²), while Engram’s elaborate local search can regress in scale and quality. Feedback mitigates several weaknesses but reduces benchmark gains. The broader lesson is that AI-evolved infrastructure needs adversarial regression testing before deployment.
Deep Analysis
Background
AlphaEvolve popularized LLM-guided algorithm evolution. OpenEvolve uses manually tuned mutation and retention, AdaEvolve reallocates search resources dynamically, and Engram supports long-horizon exploration through persistent archives and research digests. These systems target scheduling, load balancing, caching, and model placement, reporting substantial benchmark gains. What remains unresolved is whether the resulting code generalizes, scales, and fails safely outside evaluator workloads.
Core Problem
Given baseline P, evolved program P′, and valid workload set W, AIChilles seeks an input where P′ is worse than P. An execution returns 〈q,t,m〉 or abnormal termination ⊥. The challenge is not merely generating hard inputs: inputs must satisfy hidden application constraints, expose a discriminative gap, cover multiple weakness types, and reveal diverse root causes rather than repeated variants of one crash.
Innovation
- ��Relative divergence replaces isolated bug finding as the central target.
- ��Deterministic parameter extraction plus agentic constraint inference produces semantic workload grammars.
- ��Correctness, runtime, memory, and optimality receive separate search agents.
- ��Execution-frequency trajectories guide behavioral diversity and deduplication.
- ��Root-cause analysis and warm-start workloads feed discoveries back into future evolution.
- ��The design spans heterogeneous applications without a manually engineered harness for every target.
Methodology
- ��Input: baseline P, evolved P′, evaluator, and workload examples.
- ��Extraction: parse evaluators and generators to identify workload variables such as GPUs, nodes, experts, replicas, or operation sequences.
- ��Inference: an agent estimates ranges and cross-parameter constraints, producing a valid sampler.
- ��Search: four agents mutate workloads for crashes, runtime ratios, memory ratios, or quality gaps.
- ��Validation: execute P and P′ on the same workload and check ⊥, t′>Ft·t, m′>Fm·m, or q′<q.
- ��Diversity: record line-execution frequencies in P′ and prioritize novel trajectories.
- ��Analysis: cluster witnesses, explain root causes, and reuse adversarial workloads as warm starts.
Experiments
The study evaluates Engram, AdaEvolve, and OpenEvolve with GPT-5 and Claude-Opus-4.6 across transaction scheduling, expert-parallel load balancing, multi-cloud job scheduling, LLM prefix-cache optimization, and Prism model placement. Baselines include random fuzzing, mutation fuzzing, property-based testing, and a baseline LLM agent. Evaluation counts distinct weaknesses and compares mitigation behavior. The paper reports 30 AI-evolved program settings; the abstract and main results emphasize five application categories.
Results
AIChilles discovers 49 distinct weaknesses: runtime regressions occur in 25 program instances, memory regressions in 11, correctness weaknesses in 7, and optimality regressions in 6. Engram exposes fewer weaknesses than AdaEvolve and OpenEvolve, but none is uniformly robust. In Prism, the human policy is O(MG), whereas OpenEvolve’s nested scan is O(MG²); Engram’s local search can improve benchmark scores while becoming slower and locally trapped. Mitigation improves robustness but reduces score gains.
Applications
AIChilles can be placed in CI/CD after each AI-generated revision, testing schedulers, GPU model-placement controllers, cache managers, and cloud orchestration policies. It requires executable P and P′, an accessible evaluator or workload generator, and measurement of quality, wall-clock time, and peak memory. It is particularly valuable for online controllers, where a slow recomputation can leave a cluster using stale resource assignments or create a denial-of-service vector.
Limitations & Outlook
The framework depends on a reliable executable baseline and meaningful shared metrics. Agentic constraint inference adds cost and may struggle with undocumented environments; timing and memory results require repeated measurements. Randomized algorithms, external dependencies, and evolving production state complicate reproducibility. The current evidence spans five application classes and 30 settings, so it offers no universal coverage guarantee. Future work should add multiple oracles, formal constraints, production traces, state-aware coverage, and security or energy objectives.
Plain Language Accessible to non-experts
Imagine a factory with two workers. Worker P is the original human-designed worker: not flashy, but reliable across many kinds of orders. Worker P′ is an AI-improved worker. The factory tests it on a small set of sample orders, and P′ learns to earn excellent scores there. The danger is that it may be secretly optimized for those samples rather than for the factory’s real variety of orders.
AIChilles is an unusually persistent quality inspector. It first reads the factory handbook to learn which orders are legal. Then it creates many unusual but valid orders and gives exactly the same order to both workers. If P′ crashes, takes much longer, uses far more storage, or produces a worse product, the inspector records the case.
The inspector also watches how each worker moves through the factory. If ten tests trigger the same routine, it stops repeating them and searches for a different route. In the study, it found 49 hidden problems among 30 AI-modified programs: 25 involved becoming slower, 11 used more memory, 7 failed outright, and 6 produced worse solutions.
The lesson is simple: winning a fixed examination is not the same as being dependable in the real world. AIChilles can be added to the improvement loop, but stronger safety often costs some benchmark score. A good system therefore needs both impressive performance and predictable behavior under unfamiliar orders.
ELI14 Explained like you're 14
Think of an AI that upgrades a game’s matchmaking bot. It keeps rewriting the bot and testing it on the same collection of maps. Eventually, its score beats the old bot by 12–60%. Awesome, right? But maybe it has memorized those maps. On a new map it could freeze, take forever, use loads of memory, or make worse matches.
AIChilles is like a clever player whose job is to find embarrassing glitches. It takes the old bot P and the new bot P′, invents lots of legal but tricky game situations, and makes both bots face the same one. If the new bot crashes while the old one survives, runs much slower, eats more computer memory, or gets a lower score, AIChilles saves that challenge.
It also watches which parts of the code each challenge activates. Why? Because finding the same glitch twenty times is not as useful as finding twenty different glitches. In the paper, it tested 30 AI-evolved programs and found 49 hidden weaknesses. The biggest group—25 cases—made programs slower as the problem grew. Eleven used extra memory, seven caused correctness failures, and six performed worse on new cases.
For example, a Prism model-placement program originally worked roughly like O(MG), but an AI version added another scan and became O(MG²). More GPUs could therefore make it dramatically slower. The takeaway is like judging a game character: high attack power is not enough if its armor breaks instantly. AIChilles helps developers search for the armor problems before launch!
Glossary
AI-driven system optimization
A workflow in which an AI agent repeatedly rewrites a system algorithm and an evaluator selects high-scoring candidates. It turns program design into evolutionary search.
The paradigm studied through OpenEvolve, AdaEvolve, and Engram.
Differential oracle
A reference used to judge one program by comparing its behavior with another on identical inputs. Here, the human baseline P is the oracle for P′.
It checks correctness, runtime, memory, and solution quality.
Workload grammar
A compact description of legal input parameters, ranges, and cross-parameter constraints. It prevents the search from wasting effort on invalid workloads.
AIChilles builds it through deterministic extraction and agentic inference.
Code-frequency coverage
A behavioral fingerprint recording how frequently program lines execute during a run. Different fingerprints suggest different paths or failure mechanisms.
It drives diverse search and witness deduplication.
Optimality regression
A case where P′ receives a lower solution-quality score than P on a valid workload. It often indicates overfitting to the evaluator distribution.
The Prism study compares placement balance scores.
Scalability regression
A disproportionate increase in runtime or memory as workload size grows. It can arise from added loops, exhaustive search, or expensive proxies.
OpenEvolve’s Prism implementation illustrates the O(MG) to O(MG²) risk.
Open Questions Unanswered questions from this research
- 1 The study does not establish coverage across all system domains. It remains unclear how reliably the method handles production traces, nondeterministic execution, external services, and rapidly changing resource states while producing reproducible witnesses.
- 2 Treating P as the oracle can hide shared defects or undervalue legitimate improvements. Multiple baselines, formal specifications, and confidence intervals are needed to separate true regressions from measurement noise.
Applications
Immediate Applications
AI-code release gate
Systems teams can run P and P′ on adversarial workloads after every AI-generated revision. Crashes, latency ratios, peak-memory growth, and quality losses can become CI failure criteria, preventing fragile candidates from reaching production.
Scheduler stress testing
Cloud and GPU operators can generate combinations of nodes, models, requests, and replicas before deployment. This can expose complexity regressions such as O(MG²), helping controllers react quickly when demand or available hardware changes.
Long-term Vision
Robustness-aware evolution
Future optimizers could include quality, latency, memory, security, and energy in one fitness objective. The goal would shift from maximizing a narrow benchmark to evolving algorithms that remain efficient and safe across changing real-world workloads.
Abstract
The computer systems community has recently seen growing interest in AI-driven system evolution, where AI agents iteratively rewrite systems. Frameworks such as AdaEvolve and Engram report 12-60% score improvements over human-designed algorithms. While these results are promising, there are practical concerns if these AI-evolved programs can perform worse on unseen workloads and exhibit scalability regressions. Given the speed and scale of AI-generated code, we need automated mechanisms to uncover such identify hidden weaknesses in AI-evolved systems programs. To this end, we develop AIChilles that takes as input a baseline program $P$ and an AI-evolved program $P'$, AIChilles searches for valid workloads where $P'$ regresses relative to $P$ in correctness, runtime, memory usage, or output quality. To tackle the diversity in system applications, weakness types and potential bugs, AIChilles combines deterministic workload-parameter extraction, agent-based constraint inference, differential oracles, and code-frequency coverage to discover diverse failures. Across five system applications and 30 AI-evolved programs, AIChilles finds 49 distinct hidden weaknesses. We also show that explicitly including AIChilles in the AI-driven development lifecycle can mitigate several of these weaknesses.