AutoPyVerifier: Learning Compact Executable Verifiers for Large Language Model Outputs

Recent progress in large language models (LLMs) such as GPT-4 and PaLM has significantly advanced AI reasoning, code synthesis, and instruction following. Despite these breakthroughs, the reliability and correctness of model outputs remain critical issues. Traditional verification approaches rely heavily on manual rule design or simple neural classifiers, which are either labor-intensive or lack interpretability. Recent efforts have explored using LLMs as judges, but their inherent variability and surface-level reasoning limit robustness. Programmatic verifiers, like Python functions, offer transparency and executability but require extensive manual engineering and often lack coverage for complex tasks. The challenge is to develop automated methods that can learn verification logic from data, ensuring both coverage and interpretability, thus enabling scalable verification for diverse AI applications.

The core problem is to automatically induce a small, effective set of Python verifiers that collectively approximate a target evaluation metric (e.g., correctness) over model outputs. Existing solutions are either manual, which is time-consuming and inflexible, or rely on fixed heuristics that cannot adapt to task complexity. The difficulty lies in balancing verification coverage, interpretability, and computational efficiency. Moreover, verification logic must generalize across models and data distributions to be truly useful. Achieving this requires a systematic search over candidate verifier sets, guided by performance metrics and structural considerations, which has not been adequately addressed in prior work.

This paper introduces a DAG-based search framework that automates the learning of verification logic. Key innovations include:

• �� Leveraging LLMs to synthesize diverse verifier candidates based on task descriptions;
• �� Organizing the search space as a DAG, enabling iterative refinement and efficient exploration;
• �� Combining task-specific metrics (like F1), exploration bonuses, and size penalties in a unified acquisition function;
• �� Employing a critic-modifier loop where the critic diagnoses verification gaps and the modifier proposes targeted improvements;
• �� Analyzing verifier categories to shift from superficial to structural and semantic checks.

These innovations collectively enable automatic induction of compact, high-quality verification sets that generalize across models and tasks.

• �� Initial verifier synthesis: Use LLM prompts to generate multiple sets of Python verification functions, covering structural, formatting, and correctness checks.
• �� DAG construction: Each verifier set forms a node; edges represent transformations like adding, removing, or modifying verifiers.
• �� Node scoring: Compute a utility combining task performance (F1), exploration bonus (UCB-inspired), size penalty, and feasibility (balanced TP/TN ratios).
• �� Node selection: Pick the highest-scoring node for expansion.
• �� Critic analysis: An LLM critic examines false positives/negatives, diagnoses gaps, and suggests improvements.
• �� Modifier generation: An LLM proposes refined verifier sets based on critic feedback, focusing on adding, replacing, or removing verifiers.
• �� Iterative search: Repeat scoring, selection, critic analysis, and modification until convergence or budget exhaustion.
• �� Final selection: Choose the node with the best utility as the learned verifier set, which is then deployed for evaluation or inference.

The framework was evaluated on four benchmarks: AIME (mathematical reasoning), LiveCodeBench (code generation), ComplexFuncBench (multi-step reasoning), and IFBench (instruction following). Using GPT-4.1 as the backbone, the search ran for 20 steps, generating up to 3 verifier sets per node. Hyperparameters (α, β, γ) were tuned via grid search. Performance was measured by F1 scores on both in-distribution and out-of-distribution datasets, comparing initial versus refined verifier sets. Transferability was tested by applying learned verifiers to different models (e.g., Gemini-2.5). Ablation studies analyzed the impact of each search component and category analysis of learned verifiers.

Results show that the search process consistently improves verification accuracy, with gains up to 55 points F1, while maintaining small verifier sets. The learned verifiers generalize well across models and datasets, with improvements up to 54.4 points OOD. Category analysis indicates a shift toward structural and semantic checks, enhancing interpretability. When used as external tools during inference, these verifiers boost downstream accuracy by up to 17 points, demonstrating practical benefits. The approach outperforms baseline heuristics and manual rules, establishing a new standard for automated verification.

The learned verifier sets can serve as reliable proxies for correctness evaluation, improve model reranking, and guide self-correction during inference. They are applicable in high-stakes domains like education, code verification, and safety-critical systems, where interpretability and robustness are essential. Additionally, the methodology can be integrated into training pipelines for reinforcement learning or fine-tuning, fostering more trustworthy AI systems. Future work may extend to multi-modal verification and adaptive, task-aware logic induction.

The reliance on multiple large model calls increases computational costs, limiting scalability. The predefined verifier categories may restrict capturing complex or domain-specific checks. Transferability, while promising, is not universal; some tasks or models show limited improvement. Further research is needed to reduce costs, expand verifier types, and enhance robustness in diverse scenarios.