Test-Time Verification for Text-to-SQL via Outcome Reward Models
GradeSQL framework enhances Text-to-SQL reliability using ORM, achieving a 4.33% gain on BIRD.
Key Findings
Methodology
This paper introduces the GradeSQL framework, utilizing Outcome Reward Models (ORMs) for test-time verification. Training is achieved via automated candidate generation and execution-based labeling, eliminating manual annotation. ORMs are integrated into a verification-driven Best-of-N pipeline and evaluated on BIRD and Spider benchmarks.
Key Results
- On the BIRD dataset, ORM selection outperformed execution-based Best-of-N and Majority Voting by 4.33%.
- On the Spider dataset, ORM selection improved by 2.10%, showing stronger performance on complex queries.
- ORM scales effectively with larger candidate sets, particularly for complex queries.
Significance
The study demonstrates the potential of ORM in structured query generation, offering a simple, effective, and scalable alternative to existing heuristic selection strategies.
Technical Contribution
Technical contributions include an automated data generation and labeling process without manual annotation, and semantic scoring via ORM that surpasses traditional execution success and output frequency.
Novelty
This is the first application of ORM to structured query generation, providing a new verification mechanism that surpasses existing heuristic methods.
Limitations
- While ORM excels in complex queries, its improvement is limited in simpler queries.
- Requires an additional training phase, increasing computational cost.
- Limited generalizability to specific LLM architectures.
Future Work
Future work includes exploring ORM applications in other structured tasks and optimizing the training process to reduce computational cost.
AI Executive Summary
Improving the reliability of large language models in structured reasoning tasks like Text-to-SQL is a central challenge. Existing test-time inference strategies, such as Best-of-N sampling and Majority Voting, rely on heuristic signals, which fail to provide semantic discrimination across candidate outputs. This paper explores Outcome Reward Models (ORMs) as learned semantic scoring functions for test-time verification in Text-to-SQL. We introduce GradeSQL, a scalable framework for training task-specific ORMs via automated candidate generation and execution-based labeling, enabling verifier training without manual annotation. We integrate ORMs into a verification-driven Best-of-N pipeline and evaluate our approach on the BIRD and Spider benchmarks. ORM-based selection consistently outperforms execution-based Best-of-N and Majority Voting, with gains of up to 4.33% on BIRD and 2.10% on Spider. Overall, our results demonstrate that ORM-based verification provides a simple, effective, and scalable alternative to heuristic test-time selection strategies. Code, datasets, and models are publicly available.
Deep Analysis
Background
The Text-to-SQL task aims to translate natural language questions into executable SQL queries, providing intuitive access to structured databases. Despite significant progress with large language models, performance remains limited on complex queries.
Core Problem
Existing test-time inference strategies rely on heuristic signals like execution success or output frequency, failing to provide semantic discrimination across candidate outputs.
Innovation
Introduced the GradeSQL framework, which trains task-specific ORMs via automated candidate generation and execution-based labeling, eliminating manual annotation.
Methodology
- �� Automated Candidate Generation: Generate diverse candidate SQL queries based on natural language questions and database schema.
- �� Data Labeling: Label candidate queries' correctness via execution equivalence.
- �� Supervised Fine-Tuning: Fine-tune a verifier LLM using labeled data.
Experiments
Evaluated on BIRD and Spider benchmarks using multiple open-source LLM families. Results show ORM-based selection improves execution accuracy over heuristic baselines.
Results
ORM-based selection improved by 4.33% on BIRD and 2.10% on Spider, showing stronger performance on complex queries.
Applications
Applicable to structured query generation tasks requiring high semantic accuracy, such as database query optimization.
Limitations & Outlook
Requires an additional training phase, increasing computational cost. Limited generalizability to specific LLM architectures.
Plain Language Accessible to non-experts
Imagine you're in a library searching for a book. You have many title candidates but don't know which one you really need. ORM acts like a smart librarian, recommending the most suitable book based on your needs and the book's content, rather than just choosing based on the cover or frequency of the title.
ELI14 Explained like you're 14
Imagine you're playing a game where you need to choose the right path to win. You have many choices but don't know which one is correct. ORM acts like a smart guide, recommending the most suitable route based on your goals and the path's features, rather than just choosing based on appearance or name. Isn't that cool?
Glossary
Outcome Reward Models (ORMs)
A learned semantic scoring function used for test-time verification.
Used for test-time verification in Text-to-SQL.
Best-of-N
A test-time inference strategy that generates multiple candidates and selects the best output.
Used for candidate output selection.
Majority Voting
A test-time inference strategy that selects the best output based on output frequency.
Used for candidate output selection.
GradeSQL
A framework for training task-specific ORMs via automated candidate generation and execution-based labeling.
Used for ORM training.
BIRD
A benchmark dataset for evaluating Text-to-SQL tasks.
Used for evaluating ORM selection methods.
Open Questions Unanswered questions from this research
- 1 How to apply ORM in other structured tasks?
- 2 How to optimize the training process to reduce computational cost?
Applications
Immediate Applications
Database Query Optimization
Improve query semantic accuracy using ORM selection methods.
Long-term Vision
Intelligent Data Analysis
Enhance accuracy and efficiency of complex data analysis tasks using ORM.
Abstract
Improving the reliability of large language models (LLMs) at inference time is a central challenge in structured reasoning tasks such as Text-to-SQL. Common test-time inference strategies, including Best-of-N sampling and Majority Voting, rely on heuristic signals such as execution success or output frequency, which provide limited semantic discrimination across candidate outputs. In this work, we study Outcome Reward Models (ORMs) as learned semantic scoring functions for test-time verification in Text-to-SQL. While ORMs have been previously explored for test-time scaling and alignment, their application to structured query generation remains underexplored. We introduce GradeSQL, a scalable framework for training task-specific ORMs via automated candidate generation and execution-based labeling, enabling verifier training without manual annotation. We integrate ORMs into a verification-driven Best-of-N pipeline and evaluate our approach on the BIRD and Spider benchmarks across multiple open-source LLM families. ORM-based selection consistently outperforms execution-based Best-of-N and Majority Voting, with gains of up to +4.33% on BIRD and +2.10% on Spider. We further show that ORMs scale effectively with larger candidate sets and yield stronger improvements on complex queries. Overall, our results demonstrate that ORM-based verification provides a simple, effective, and scalable alternative to heuristic test-time selection strategies for Text-to-SQL. Code datasets and models are publicly available.