Can QPP Choose the Right Query Variant? Evaluating Query Variant Selection for RAG Pipelines

Retrieval-Augmented Generation (RAG) has rapidly become a dominant architectural paradigm for modern information systems. Traditional ad-hoc retrieval methods typically provide a ranked list for user consumption, whereas RAG inserts a Large Language Model (LLM) between retrieval and the user, delegating answer synthesis to a generative model conditioned on retrieved evidence. This shift fundamentally alters both the objective and the economics of search. In this setting, the role of query reformulation is significantly emphasized. A user's original query may fail to retrieve passages that adequately ground generation, exacerbating vocabulary mismatch, intent drift, and underspecification issues. LLM-based query reformulation has become common practice to mitigate this problem by generating multiple semantically equivalent query variants to improve recall and coverage.

Executing the full pipeline for every query reformulation is computationally expensive, especially in production settings where exhaustive execution is often infeasible. This necessitates a more efficient alternative: can we identify the best query variant before incurring the downstream generation cost? Query Performance Prediction (QPP) offers a natural mechanism for this problem. Traditionally, QPP estimates retrieval effectiveness without relevance judgments and has been used for tasks such as selective query expansion, system routing, and risk-sensitive retrieval. However, its evaluation has largely relied on correlation with ranking metrics such as nDCG or Average Precision.

This study is the first to apply QPP to query variant selection in RAG pipelines, proposing a new evaluation framework that significantly enhances generation quality without increasing computational costs. This contrasts sharply with traditional QPP methods, which mainly focus on estimating retrieval effectiveness. The study assesses these predictors using correlation- and decision-based metrics, revealing a systematic divergence between retrieval and generation objectives: variants maximizing ranking metrics like nDCG often fail to produce the best generated answers, exposing a 'utility gap' between retrieval relevance and generation fidelity.

• �� Conduct large-scale experiments on the TREC-RAG dataset, evaluating pre-retrieval and post-retrieval predictors under sparse and dense retrievers.

• �� Use correlation- and decision-based metrics to assess these predictors' effectiveness.

• �� Reveal a systematic divergence between retrieval and generation objectives, where variants maximizing ranking metrics like nDCG often fail to produce the best generated answers.

• �� Demonstrate that lightweight pre-retrieval predictors often match or outperform more expensive post-retrieval methods, providing a latency-efficient approach to robust RAG.

Experiments were conducted on the TREC-RAG 2024 benchmark, explicitly designed to evaluate RAG systems and provide evaluation protocols for retrieval and RAG tasks separately. The benchmark consists of 56 queries constructed over the MS MARCO v2.1 corpus, which contains over 138 million passages. Importantly, these queries have been carefully and thoroughly judged across retrieval and generative dimensions by both human assessors and LLM-based judges, enabling a fair comparison of performance under different pipeline configurations. The study specifically utilizes human annotations for both retrieval and nugget-based evaluations.

The study reveals a systematic divergence between retrieval and generation objectives: variants that maximize ranking metrics such as nDCG often fail to produce the best generated answers, exposing a 'utility gap' between retrieval relevance and generation fidelity. Nevertheless, QPP can reliably identify variants that improve end-to-end quality over the original query. Notably, lightweight pre-retrieval predictors frequently match or outperform more expensive post-retrieval methods, offering a latency-efficient approach to robust RAG.

This research holds significant implications for both academia and industry. It highlights the importance of selecting the best query variant in RAG pipelines, particularly when there is a utility gap between generation quality and retrieval relevance. By using QPP, systems can enhance the quality of generated answers without significantly increasing computational costs, which is crucial for real-world applications that require efficient handling of large volumes of queries.

The study is primarily conducted on the TREC-RAG dataset, and its performance on other datasets may vary. While pre-retrieval predictors perform well in many cases, they may not be as effective as post-retrieval methods for certain complex queries. Additionally, the study reveals a systematic divergence between retrieval and generation objectives, where variants maximizing ranking metrics like nDCG often fail to produce the best generated answers, exposing a 'utility gap' between retrieval relevance and generation fidelity.