QueStER: Query Specification for Generative keyword-based Retrieval
QueStER uses lightweight LLMs to generate keyword queries, combining traditional BM25 retrieval with learned query reformulation, achieving +4.0 nDCG@10 over BM25.
Key Findings
Methodology
The approach employs a lightweight LLM (e.g., Qwen-4B) trained via reinforcement learning (GRPO) to generate explicit keyword queries from user inputs. The model outputs keywords, which are then used by BM25 for retrieval. Rewards based on SoftRank and nDCG guide training, with cross-encoder distillation providing relevance signals. Experiments on MS MARCO and TREC datasets demonstrate that QueStER significantly outperforms baseline BM25 and neural IR models, maintaining high efficiency (~28ms/query). The system balances effectiveness with computational cost, leveraging reinforcement learning to optimize query reformulation policies.
Key Results
- On MS MARCO dev set, QueStER improves nDCG@10 by 4.0 points (from 50.6 to 54.6), and on TREC DL’19/’20, it surpasses neural baselines like ColBERTv2 and SPLADEv2. In out-of-domain BEIR datasets, it exceeds all neural models with a +5.3 point gain, demonstrating strong generalization. The model achieves a retrieval latency of approximately 28ms per query, comparable to traditional methods but with enhanced effectiveness. Ablation studies confirm that larger models (up to 4B parameters) and carefully designed rewards contribute to performance gains.
- The experiments reveal that the reinforcement learning strategy, reward functions, and model size are critical factors. SoftNDCG rewards better reflect ranking quality, leading to more stable training. Cross-encoder distillation effectively mitigates false negatives in datasets like MS MARCO. The results establish that explicit keyword generation, guided by learned policies, can effectively bridge the gap between traditional lexical retrieval and neural models, offering a scalable, interpretable, and high-performing retrieval framework.
- Overall, QueStER demonstrates that combining generative query reformulation with classical retrieval engines yields superior results, especially in cross-domain scenarios. Its efficiency and scalability make it suitable for real-world large-scale search systems, providing a promising direction for future research integrating language models with traditional IR pipelines.
Significance
This work addresses core limitations of neural IR models, notably their high storage and update costs, by integrating learned keyword generation with established lexical retrieval techniques. It leverages reinforcement learning to optimize query reformulation, resulting in a system that is both effective and scalable. The approach enhances cross-domain robustness and interpretability, aligning with industry needs for efficient, transparent search engines. By bridging generative models with traditional IR, it paves the way for more adaptable and cost-effective large-scale retrieval solutions, impacting academia and industry alike.
Technical Contribution
The key technical innovation lies in training a lightweight LLM via reinforcement learning (GRPO) to generate explicit keyword queries, guided by SoftRank-based rewards. This enables effective policy optimization without extensive supervision. The system combines the strengths of generative models and lexical retrieval, reducing dependency on large indexes and enabling easy updates. The use of cross-encoder distillation further refines relevance signals, improving ranking accuracy. The methodology offers a scalable, interpretable, and high-performance retrieval framework that can adapt to evolving models and datasets, representing a significant advancement over existing neural and lexical IR methods.
Novelty
This research is the first to explicitly train a generative keyword reformulation policy using reinforcement learning, integrating it with traditional retrieval engines to improve effectiveness and scalability. Unlike prior work relying on meta-data or document IDs, QueStER generates explicit search specifications, enabling better generalization and interpretability. Its combination of lightweight LLMs, reinforcement learning, and classical IR techniques marks a novel paradigm shift in retrieval research, bridging the gap between neural and lexical methods.
Limitations
- The model’s performance depends heavily on the quality of the reward function and the size of the language model; in low-resource or highly specialized domains, effectiveness may decline. Training complexity and computational costs remain high, especially for larger models and extensive reinforcement learning iterations. The system’s reliance on cross-encoder distillation introduces additional overhead, and its robustness in extremely noisy or ambiguous queries needs further validation. Future work should focus on improving domain adaptation, reducing training costs, and enhancing interpretability.
Future Work
Future research will explore multi-modal inputs, integrating structured knowledge and contextual cues to enrich keyword generation. Adaptive reward mechanisms could further improve robustness across diverse domains. Additionally, extending the framework to multi-turn, interactive retrieval scenarios and incorporating user feedback could enhance personalization and explainability. Efforts to reduce training complexity and improve model efficiency will be prioritized, aiming to deploy this approach in real-world, large-scale search engines and knowledge systems.
AI Executive Summary
Traditional information retrieval systems rely heavily on static inverted indexes, such as BM25, which, despite their efficiency, suffer from vocabulary mismatch and limited flexibility. Neural IR models, including dense and sparse encoders like ANCE and SPLADE, have advanced effectiveness but at the cost of large storage requirements and high update complexity. Generative retrieval (GR) models emerged as an alternative, internalizing the index within model parameters, but they often lack robustness and scalability across domains.
This paper introduces QueStER, a novel framework that bridges the gap between classical lexical retrieval and neural models by learning to generate explicit keyword search specifications from user queries. Using a lightweight language model trained via reinforcement learning (GRPO), QueStER produces keyword queries that are then executed by a standard BM25 engine. The core innovation lies in optimizing the generation policy with a reward based on SoftRank and nDCG metrics, ensuring high-quality ranking results.
Extensive experiments on MS MARCO, TREC DL, and BEIR datasets demonstrate that QueStER consistently outperforms traditional BM25 (+4.0 nDCG@10 in-domain, +5.3 out-of-domain) and rivals neural IR models in effectiveness, while maintaining low latency (~28ms/query). The approach effectively combines the efficiency of lexical methods with the adaptability of generative models, offering a scalable, interpretable, and high-performing retrieval solution.
This work significantly impacts both academia and industry by providing a practical pathway to deploy high-quality, scalable retrieval systems that are less dependent on massive indices and retraining. Future directions include multi-modal integration, personalized retrieval, and further efficiency improvements, promising a new paradigm for large-scale information access.
Deep Dive
Abstract
Generative retrieval (GR) differs from the traditional index-then-retrieve pipeline by storing relevance in model parameters and generating retrieval cues directly from the query, but it can be brittle out of domain and expensive to scale. We introduce QueStER (QUEry SpecificaTion for gEnerative Keyword-Based Retrieval), which bridges GR and query reformulation by learning to generate explicit keyword-based search specifications. Given a user query, a lightweight LLM produces a keyword query that is executed by a standard retriever (BM25), combining the generalization benefits of generative query rewriting with the efficiency and scalability of lexical indexing. We train the rewriting policy with reinforcement learning techniques. Across in- and out-of-domain evaluations, QueStER consistently improves over BM25 and is competitive with neural IR baselines, while maintaining strong efficiency.