Steerable Instruction Following Coding Data Synthesis with Actor-Parametric Schema Co-Evolution

The evolution of large language models (LLMs) like GPT-3, Codex, and AlphaCode has demonstrated remarkable progress in code synthesis and reasoning tasks. These models rely heavily on large-scale instruction-annotated datasets, which are expensive and labor-intensive to produce manually. Recent efforts have explored automatic data augmentation, including static brute-force sampling and adversarial generation, but these approaches often suffer from logical inconsistency, limited diversity, and difficulty in verification. As models grow larger and more capable, the need for high-quality, diverse, and challenging training data becomes critical. Addressing this gap, researchers seek dynamic, scalable methods that can generate data with controllable complexity and validity, fostering models that can reason, generalize, and adapt more effectively.

The core challenge is how to automatically generate vast amounts of instruction-code pairs that are both logically valid and sufficiently challenging to improve model reasoning. Static sampling methods tend to produce repetitive, low-diversity data, while adversarial approaches risk generating unsolvable or ill-defined problems. Ensuring logical consistency and verifiability, especially when scaling up, remains a significant obstacle. Moreover, balancing difficulty and validity is non-trivial, as overly hard or invalid samples can hinder training. Developing a method that can adaptively control data complexity, ensure correctness, and promote diversity is essential for advancing automatic programming systems.

This paper introduces a pioneering framework combining actor models with parametric instruction schemas, guided by Monte Carlo Tree Search (MCTS). Key innovations include:

• �� Parametric instruction schemas that encode diverse constraints with configurable parameters, enabling flexible and expansive instruction representation.
• �� MCTS-driven exploration of instruction combinations, efficiently navigating the combinatorial space to generate challenging yet valid problems.
• �� Feedback-driven, proof-by-construction validation, ensuring logical correctness through AST analysis and execution verification.
• �� Co-evolution of the instruction schema library via pattern composition and mutation, driven by sampler success statistics, fostering continuous difficulty escalation.
• �� An iterative, self-improving cycle where the actor model is post-trained on generated data, progressively increasing problem complexity.

This integrated approach surpasses static or adversarial methods by providing a controllable, scalable, and logically sound data synthesis pipeline that adapts to model capabilities.

• �� Construct a diverse instruction schema library with variable slots and parameter ranges.
• �� Use MCTS to sample instruction combinations, guided by estimated difficulty and success metrics.
• �� Generate code via an LLM conditioned on current instructions, validating correctness through AST and execution checks.
• �� Collect feedback from the actor model’s success rate, adjusting sampling and guiding early stopping.
• �� Evolve instruction schemas through pattern composition, merging synergistic schemas based on sampler statistics.
• �� Mutate underperforming schemas using adversarial prompts and validation, tightening constraints to increase difficulty.
• �� Iterate this process over multiple rounds, continuously expanding and refining the dataset with challenging, valid instruction-code pairs.

The authors curated a dataset from MBPP, LeetCode, and ClassEval, totaling 1,565 problems. They trained models of various sizes (1.3B to 32B) using data generated by IFCodeEvolve. Evaluation was performed on IFEvalCode and CodeIF benchmarks, measuring instruction satisfaction and task success rates. Hyperparameters included C=3, α=10, η=0.5, with multiple evolution rounds. Ablation studies examined the impact of schema composition, mutation, and feedback routing. Results showed consistent improvements, with the largest models achieving performance comparable to proprietary SOTA models, especially on complex instructions.

Models trained with IFCodeEvolve data achieved over 20% higher instruction compliance and success rates. For example, the 32B model's instruction satisfaction increased from 0.68 to 0.87, and success rate from 0.58 to 0.85. Multi-round evolution further boosted performance, with successive generations showing incremental gains. Ablation studies confirmed that schema composition and mutation significantly contributed to data diversity and difficulty. The generated data also exhibited high logical correctness, validated via AST and execution checks, leading to improved robustness and reasoning ability.

This framework enables automatic generation of high-quality training data for code synthesis, benefiting large language models, automated programming tools, and intelligent tutoring systems. It can be applied in scenarios requiring diverse, challenging, and verifiable code problems, reducing manual effort and enabling scalable data augmentation. In industry, it supports the development of more reliable AI coding assistants and automated code review systems. Long-term, the methodology can be extended to scientific reasoning, mathematical problem solving, and multi-modal tasks, fostering autonomous AI systems capable of self-improvement.

The approach depends on predefined schema libraries, which may limit instruction diversity unless continuously expanded. Computational costs are high due to iterative sampling, validation, and schema mutation, especially for larger models. Its effectiveness on multi-modal or highly complex instructions remains to be validated. Future work should focus on schema expressiveness, efficiency improvements, and broader applicability to other reasoning domains.